If you want to read the PDF, try requesting it from the authors.

Abstract

Due to its fast temporal evolution and its representation and integration among complex and widespread neural networks, the emotion perception process should preferably be examined by means of multimethodological approach. Indeed the indubitable vantage of acquiring both the autonomic (arousal-related) and the central (cortical-related) activities stands in the possibility to better elucidate the reciprocal interplay of the two compartments. In the present study EEG (frequency band analysis), systemic SCR and heart rate (HR) were all recorded simultaneously with hemodynamic (NIRS, Near-Infrared Spectroscopy) measurements as potential biological markers of emotions, related to both central and peripheral systems. These multiple measures were then related to the self-report correlates, that is the subjective appraisal in term of valence (positive vs. negative) and arousal (high vs. low) by using SAM rating. Twenty subjects were submitted to emotional cues processing (IAPS) when fNIRS, frequency bands (alpha, beta, delta, theta), SCR and HR were recorded. As shown by O2Hb increasing within the right hemisphere, the contribution of prefrontal cortex was elucidated, by pointing out a relevant lateralization effect (more right-PFC activity) induced by the specific valence (negative) of the emotional patterns. Secondly, EEG activity (mainly low-frequency theta and delta bands) was intrinsically associated with the cortical hemodynamic responsiveness to the negative emotional patterns, within the right side. Finally SCR increased mainly in response to negative patterns, and the autonomic behavior was related to explicit (SAM) and cortical (NIRS; EEG) activity. The intrinsic relationships between these three different levels are discussed. Copyright © 2015 Elsevier Inc. All rights reserved.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Electroencephalography (EEG) can directly measure neuron activity with high temporal resolution, evaluate the time and frequency of spontaneous brain activity, and characterize brain functional networks (Cao et al., 2021;Duan et al., 2021;Pei et al., 2014;van Diessen et al., 2015). Further, previous studies have found that, in emotion regulation, relevant EEG information is mainly encoded in the low-frequency band, particularly in the theta range (Balconi et al., 2015;Ertl et al., 2013;Qian et al., 2014;Sulpizio et al., 2021), which is associated with emotion and reflects the neural mechanism of emotion regulation (Balconi et al., 2015;Schlumpf et al., 2019;Xing et al., 2019;Zhang et al., 2013). In addition, the event-related potential (ERP) could help evaluate neural activity in a stimulus-response process. ...
... Electroencephalography (EEG) can directly measure neuron activity with high temporal resolution, evaluate the time and frequency of spontaneous brain activity, and characterize brain functional networks (Cao et al., 2021;Duan et al., 2021;Pei et al., 2014;van Diessen et al., 2015). Further, previous studies have found that, in emotion regulation, relevant EEG information is mainly encoded in the low-frequency band, particularly in the theta range (Balconi et al., 2015;Ertl et al., 2013;Qian et al., 2014;Sulpizio et al., 2021), which is associated with emotion and reflects the neural mechanism of emotion regulation (Balconi et al., 2015;Schlumpf et al., 2019;Xing et al., 2019;Zhang et al., 2013). In addition, the event-related potential (ERP) could help evaluate neural activity in a stimulus-response process. ...
... After the completion of the emotion-regulation task for all pictures, participants entered the phase of image rating. They rated their level of valence, followed by their level of arousal (Balconi et al., 2015;Doré et al., 2017;Füstös et al., 2013;Langeslag and Surti, 2017;Pollatos and Gramann, 2012). In each block, the image rating task had a total of 45 trials, with an image for each trial. ...
Article
Reappraisal is an effective emotion regulation strategy which can be divided into self- and situation-focused subtypes. Previous studies have produced inconsistent findings on the moderating effects and neural mechanisms of reappraisal; thus, further research is necessary to clarify these inconsistencies. In this study, a total of 44 participants were recruited and randomly assigned to two groups. 23 participants were assigned to the self-focused group, while 21 participants were assigned to the situation-focused group. The participants' resting EEG data were collected for 6 minutes before the experiment began, followed by an emotional regulation task. During this task, participants were asked to view emotion-provoking images under four emotion regulation conditions (View, Watch, Increase, and Decrease). Late positive potential (LPP) was obtained when these emotional images were observed. LPP is an effective physiological indicator of emotion regulation, enabling this study to explore emotion regulation under different reappraisal strategies, as well as the functional connectivity and node efficiency within the brain. It was found that, in terms of the effect on emotion regulation, situation-focused reappraisal was significantly better than self-focused reappraisal at enhancing the valence of negative emotion, while self-focused reappraisal was significantly better than situation-focused reappraisal at increasing the arousal of negative emotion. In terms of neural mechanisms, multiple brain regions such as the anterior cingulate cortex, the frontal lobe, the parahippocampal gyrus, parts of the temporal lobe, and parts of the parietal lobe were involved in both reappraisal processes. In addition, there were some differences in brain regions associated with different forms of cognitive reappraisal. Self-focused reappraisal was associated with the posterior cingulate gyrus, fusiform gyrus, and lingual gyrus, and situation-focused reappraisal was associated with the parietal lobule, anterior central gyrus, and angular gyrus. In conclusion, this research demonstrates that self- and situation-focused reappraisal are not homogenous in terms of their effects and neural mechanisms and clarifies the uncertainties over their regulatory effects. Different types of reappraisal activate different brain regions when used, and the functional connectivity or node efficiency of these brain regions seems to be a suitable indicator for assessing the effects of different types of reappraisal.
... fNIRS allows the determination of relative changes in the concentration of oxygenated (oxy-Hb) and deoxygenated hemoglobin (deoxy-Hb), while simultaneous functional magnetic resonance imaging (fMRI) and fNIRS recordings have shown that the blood oxygen level-dependent (BOLD) and fNIRS signals are strongly correlated to each other (Cui, Bray, Bryant, Glover & Reiss, 2011;Strangman, Culver, Thompson & Boas, 2002). Previous fNIRS studies have shown a right lateralized activation of prefrontal cortex for negative and left lateralized activation for positive emotions (Balconi, Grippa & Vanutelli, 2015b;Everhart & Harrison, 2000;Tanida, Katsuyama & Sakatani, 2007). Hoshi, Huang, Kohri et al. (2011) using event-related fNIRS showed that negative emotions increased oxy-Hb in bilateral ventrolateral prefrontal cortex (PFC), and positive emotions decreased it in the left dorsolateral PFC. ...
... Moreover, electrodermal activity (EDA) has been thought as a potential peripheral biomarker of central affective processes and has been shown, among other things, to reflect limbic and ventral prefrontal activations (Furmark, Fischer, Wik, Larsson & Fredrikson, 1997;Lang, Davis & Ohman, 2000). Previous fNIRS studies have reported correlation between PFC activation and sympathetic activity in response to mental tasks (Balconi et al., 2015b;Tanida et al., 2007) and other studies have shown that EDA responses differ depending on the kind of the underlying affective process (Balconi, Brambilla & Falbo, 2009;Balconi et al., 2015b;Cuthbert, Schupp, Bradley, Birbaumer & Lang, 2000;Tupak et al., 2014). EDA is thus considered an important complementary method of studying the peripheral autonomic activity in parallel with that of central nervous system activity. ...
... Moreover, electrodermal activity (EDA) has been thought as a potential peripheral biomarker of central affective processes and has been shown, among other things, to reflect limbic and ventral prefrontal activations (Furmark, Fischer, Wik, Larsson & Fredrikson, 1997;Lang, Davis & Ohman, 2000). Previous fNIRS studies have reported correlation between PFC activation and sympathetic activity in response to mental tasks (Balconi et al., 2015b;Tanida et al., 2007) and other studies have shown that EDA responses differ depending on the kind of the underlying affective process (Balconi, Brambilla & Falbo, 2009;Balconi et al., 2015b;Cuthbert, Schupp, Bradley, Birbaumer & Lang, 2000;Tupak et al., 2014). EDA is thus considered an important complementary method of studying the peripheral autonomic activity in parallel with that of central nervous system activity. ...
Article
Full-text available
Affective processing, including induction and regulation of emotion, activates neural networks, induces physiological responses, and generates subjective experience. Dysregulation of these processes can lead to maladaptive behavior and even psychiatric morbidity. Multimodal studies of emotion thus not only help elucidate the nature of emotion, but also contribute to important clinical insights. In the present study, we compared the induction (EI) and effortful regulation (ER) with reappraisal of fear and disgust in healthy subjects using functional near infrared spectroscopy (fNIRS) in conjunction with electrodermal activity (EDA). During EI, there was significant activation in medial prefrontal cortex (PFC) for fear and more widespread activation for disgust, with right lateral PFC significantly more active during disgust compared to fear. ER was equally effective for fear and disgust reducing subjective emotion rating by roughly 45%. Compared to baseline, there was no increased PFC activity for fear during ER, while for disgust lateral PFC was significantly more active. Significant differences between the two negative emotions were also observed in sympathetic nerve activity as reflected in EDA during EI, but not during ER. Lastly, compared to men, women had higher emotion rating for both fear and disgust without corresponding differences in EDA. In conclusion, in the present study we show that emotion induction was associated with differential activation in both PFC and sympathetic nerve activity for fear and disgust. These differences were however less prominent during emotion regulation. We discuss the potential interpretation of our results and their implications regarding our understanding of negative emotion processing.
... Therefore, thirdly, a more suitable approach could combine EEG/fNIRS measurements that allow for the complementary examination of neural as well as hemodynamic aspects of brain activation with a good temporal and spatial resolution (Biallas et al., 2012). Indeed, fNIRS is particularly well-suited for evaluating the prefrontal cortex (PFC) activity in terms of hemodynamic variations related to emotional processing (Balconi et al., 2015), and empathy for pain (Balconi et al., 2020;Xie et al., 2018). Despite the low time resolution if compared to an electroencephalogram (EEG), the fNIRS system has a good spatial resolution, a low sensitivity to body movements, it is portable, easy to use, non-invasive, and makes possible for participants to view stimuli in a more naturalistic way. ...
... Specifically, the theta band positively correlated with subjective ratings of perceived pain and self-unpleasantness, suggesting that theta oscillations are involved in emotional sharing during empathy for pain (Mu et al., 2008). Theta rhythm modulation was also associated with affective valence discrimination of visual displays (Aftanas et al., 2001(Aftanas et al., , 2003Balconi et al., 2015;Balconi & Lucchiari, 2006;Balconi & Pozzoli, 2005, 2007), activation of memory and emotion regulation systems ( Knyazev, 2007). In contrast, exiguous data concern the modulation of the delta and beta bands when considering the emotional significance of a stimulus (Karakaş et al., 2000). ...
... Theta band underlies mechanisms for cognitive control over the situation (Cavanagh & Frank, 2014), alertness, attention, and readiness to process the presence of emotional information (Aftanas et al., 2001(Aftanas et al., , 2003Balconi et al., 2015;Balconi & Lucchiari, 2006;Balconi & Pozzoli, 2007), activation of memory and emotion regulation systems (Knyazev, 2007). Also, theta duration was interpreted as a correlate of increased attention and arousal due to the emotional content of the stimulus (Balconi & Lucchiari, 2006). ...
Article
We examined whether the modulation of Interoceptive Attentiveness (IA) influences the cortical correlates of observation of pain in others. Healthy participants observed painful/non-painful stimuli while brain response [oxygenated and deoxygenated hemoglobin (O2Hb; HHb), and electroencephalographic cortical oscillations] was measured. Participants were divided into experimental (EXP) and control group: EXP group was required to focus on its interoceptive correlates during the task. Interpersonal Reactivity Index (IRI) scale was administered to test empathic traits. Focusing on prefrontal cortex activity, theta band and O2Hb in the right frontal hemisphere while observing painful stimuli positively correlates in the EXP. Delta band and O2Hb in left frontal hemisphere for non-painful stimuli positively correlates in controls. IRI (Perspective Taking and Empathic Concern subscales) and right frontal activity for painful stimuli negatively correlates in the EXP. Findings were discussed in light of the modulating role of IA in enhancing the negative experience of observing pain in others.
... Evidence from literature has historically associated different neurophysiological processes to these patterns. In fact, lowfrequency bands (delta and theta) have been linked to the processing of emotional and mnestic information (Balconi et al., 2015;Khader et al., 2010). Instead, alpha desynchronization and beta synchronization are associated respectively with general cognitive activation and selective attentional processing for specific stimuli within the environment (Runnova et al., 2021) Another methodological issue to address when considering our research object is how to make two training sessions comparable to each other, with a satisfying degree of reliability, while preserving a high level of ecological validity. ...
... Regarding theta activity, we conjectured that the face-to-face condition presents increased activation in the power compared to the remote one. In fact, we expect higher emotional engagement, marked by a theta synchronization (Balconi et al., 2015) in the first condition where nonverbal language is also part of the participants' communication. ...
... As expected, we detected a general increased activity in the theta power in the face-to-face condition compared to the remote setting. Cognitive neuroscience has consistently highlighted the role of theta rhythm in emotional regulation functioning (e.g., Uusberg et al., 2014;Balconi et al., 2015). Its inherent role as a foundation for interindividual communication and as a social connector from an emotional perspective is known. ...
Article
The digitalization of learning in the organization represents both a necessity and an opportunity. Little to no research explored how distance training affects cognitive and affective processes in individuals and workgroups. For this reason, in this work, we propose an hyperscanning research design where conversational analysis is used to compare neurophysiological measures (frequency band analysis: delta, theta, alpha, and beta) between an equivalent training session carried out in two conditions (face-to-face and remote), by collecting electroencephalographic data (EEG) on a trainer and three groups of trainees. We theoretically describe the protocol, and we further report initial explorative results. Data showed a significant effect of the condition on both theta and beta waves, with higher synchronization for the face-to-face setting. Also, trainees seem more impacted by the delivery modality compared to the trainer. This work highlights the relevance of neurophysiological measures to test e-learning efficacy.
... eta rhythm modulation was also associated with affective valence discrimination of visual displays [44][45][46][47][48][49][50], activation of memory, and emotion regulation systems [51]. ...
... Starting with the first evidence of this study, for the EXP group, significantly higher theta power was found in the following two areas of the right hemisphere in response to painful stimuli: (i) in the frontal area for the stimuli presented in the social condition and (ii) in the parieto-occipital area for stimuli displayed in the individual condition. e theta band underlies the mechanisms for cognitive control over the situation [60], alertness, attention, and readiness to process the presence of emotional information [44][45][46][47][48][49][50], activation of memory, and emotion regulation systems [51]. In addition, the theta duration was interpreted as a correlate of increased attention and arousal due to the emotional content of the stimulus [48]. ...
... In addition, the theta duration was interpreted as a correlate of increased attention and arousal due to the emotional content of the stimulus [48]. Moreover, regarding the electrodes where these effects occur, it is worth noting that the modulation of the theta band by prefrontal and frontal cortex regions (DLPFC) has been previously shown to be related to cognitive control over salient emotional stimuli [47] and to the affective and motivational evaluation of pain in others [29][30][31]. Instead, the theta band over parieto-occipital areas was detected in response to the visual aspects of the stimuli, to the observation (rather than the direct stimulation and induction) of a painful condition or to the arousing levels of the emotional visual stimuli presented [61]. ...
Article
Full-text available
Objective: This research demonstrates that interoceptive attentiveness (IA) can modulate cortical oscillations related to the emotional and cognitive representations of observing pain in others. Methods: Twenty participants were required to observe painful/nonpainful stimuli in an individual versus the interactive condition during the recording of the electroencephalogram. The sample was divided into experimental (EXP) and control (CTR) groups, and the EXP group was explicitly required to direct the attention on its interoceptive correlates while observing the stimuli. Results: Mixed repeated measures, analyses of variance, were applied to each EEG frequency band. Significant findings were obtained mainly for theta and beta bands for the two groups. A hemispheric lateralisation effect was found, with right lateralisation of the theta band for the EXP group when observing painful stimuli and enhanced left activation of theta and beta bands for the CTR group when observing nonpainful stimuli. For both groups, frontal cortical regions were significantly sensitive to social scenarios, while posterior parietal activation was found for stimuli depicting the individual condition. Conclusions: The results suggest that IA might enhance the emotional representation of painful stimuli, highlighting their negative and unpleasant features in the EXP group, while the attention of the CTR group was mainly drawn to nonpainful stimuli in social and individual conditions, with a positive valence. The role of frontal regions in the processing of social stimuli through social cognition, inducing emotional mirroring and requiring deeper analysis of the social context, was underlined. We propose that IA could be trained for promoting emotion regulation and empathic response.
... The use of large datasets will change the performance of the support vector machine classifier because the training process must deal with more parameters so the speed of convergence become slow. Nowadays the Shuffled Frog Leaping Algorithm (SFLA) [8][9][10][11][12][13] is receiving more and more attention in project optimization. SFLA has a powerful global optimal search capability and is more efficient than existing optimization algorithms. ...
... IOP Publishing doi:10.1088/1742-6596/1792/1/012002 and they will easily converge with good RBM initialization. ...
Article
Full-text available
This paper studies the random optimization algorithm to reorganize the leapfrog algorithm. The big data challenge requires an effective optimization algorithm to explore potential data structures using deep neural networks. At first, we introduce the neural network classifier and compare it with the support vector machine. Neural networks are suitable for large data sets and have the complex ability to extract high-level abstract data. And then we have to introduce a large dataset covering cancer data and voice data. Both datasets have large numbers of samples with complex low-level variance. At last we have to use the reorganized leapfrog algorithm to optimize neural network parameters. The random leapfrog algorithm is efficient and robust to a local minimum. The experimental results show that the algorithm has extensive application prospects and is suitable for the classification of big dataset. The neural network parameters can effectively optimized by the improved shuffled frog leaping algorithm.
... In line with this, Knyazev (2007) showed that delta power depends on the activity of motivational systems and participate in stimulus salience detection (Knyazev, 2007). Moreover, Balconi et al. (2015) found that delta modulations were found to be related to arousing power of stimuli in right and left frontal localizations, regardless of the stimulus valence. Therefore, we concluded that, taken together, Theta and Delta increased activity may be responsive to a process of signal detection of the stimuli encountered during the store navigation; however, the absence of tactile contact could have given rise to a situation of incomplete environmental perception that alerted the consumers on the need for cognitive control on their experience. ...
... Moreover, further studies will be necessary to investigate the lateralization effect we found in relation to the different frequency bands and the possible specific role of Delta and Theta oscillations in left frontal structures as a possible marker of processing the emotional valence of consumers' experience. Indeed, frontal and prefrontal cortex lateralization has been previously related to cognitive control over emotional stimuli and emotional behavior in basic research and in studies on cross-modal integration of emotional cues (Balconi et al., 2015;Balconi and Vanutelli, 2016). In past neuromarketing studies, a left prefrontal cortex activation toward commercial advertising was interpreted as an index of positive emotions and consumers' preference (Balconi et al., 2014b;Leanza and Balconi, 2017). ...
Article
Full-text available
To gain a deeper understanding of consumers' brain responses during a real-time in-store exploration could help retailers to get much closer to costumers' experience. To our knowledge, this is the first time the specific role of touch has been investigated by means of a neuroscientific approach during consumer in-store experience within the field of sensory marketing. This study explores the presence of distinct cortical brain oscillations in consumers' brain while navigating a store that provides a high level of sensory arousal and being allowed or not to touch products. A 16-channel wireless electroencephalogram (EEG) was applied to 23 healthy participants (mean age = 24.57 years, SD = 3.54), with interest in cosmetics but naive about the store explored. Subjects were assigned to two experimental conditions based on the chance of touching or not touching the products. Cortical oscillations were explored by means of power spectral analysis of the following frequency bands: delta, theta, alpha, and beta. Results highlighted the presence of delta, theta, and beta bands within the frontal brain regions during both sensory conditions. The absence of touch was experienced as a lack of perception that needs cognitive control, as reflected by Delta and Theta band left activation, whereas a right increase of Beta band for touch condition was associated with sustained awareness on the sensory experience. Overall, EEG cortical oscillations' functional meaning could help highlight the neurophysiological implicit responses to tactile conditions and the importance of touch integration in consumers' experience.
... The findings revealed increased recruitment of PFC during the viewing of COVID-19-related commercials compared to ads unrelated to COVID-19, providing support for our first hypothesis. Indeed, a substantial body of literature shows that several cortical areas within the PFC contribute to the neural network that controls attention shifting towards emotional elements and the processing of emotional information, such as the anterior cingulate cortex (ACC), the orbitofrontal cortex (OFC), and the dorsolateral (DLPFC) and medial prefrontal cortices (mPFC) [57][58][59][60]. This finding might imply that the message conveyed by COVID-19-related advertisements is effective in prompting a deeper elaboration of the affective information. ...
... For instance, electroencephalography (EEG) has widely shown its valuable contribution in the practice of affective computing, including in consumer neuroscience studies [26], and has recently proved the potential of its combined use with fNIRS [73,74]. In addition, autonomic indices, such as electrodermal activity and heart rate, could provide supplementary insights into the role of other emotional response components (such as physiological arousal) on consumer decisions [58]. Moreover, the strength of the statistical analysis in the present study may have been mitigated by the small size of the recruited sample. ...
Article
Full-text available
In pandemic times, taking advantage of COVID-19-elicited emotions in commercials has been a popular tactic employed by corporations to build successful consumer engagement and, hopefully, increase sales. The present study investigates whether COVID-19-related emotional communication affects the consumer’s emotional response and the approach/avoidance motivation toward the brand—measured as a function of brain hemodynamic changes—as well as the purchase intentions. The functional Near-Infrared Spectroscopy (fNIRS) was employed to record neural correlates from the prefrontal cortex while the experimental and control groups were observing respectively COVID-19-related and unrelated advertisements (ads). The hemodynamic patterns suggest that COVID-19-related ads may promote deeper emotional elaboration, shifting consumers’ attention from the semantic meaning to the affective features and perhaps supporting a more favorable brand evaluation. Conversely, purchase intentions were only related to the pre-existing level of brand engagement. The findings suggest that leveraging the negative emotional potential of COVID-19 may not shift the explicit purchase intentions but could nonetheless boost emotional engagement, benefitting the final evaluation of the brand at an implicit level.
... To compare the present results to the ones of other works, only three publicly available emotion EEG datasets can be found. The public datasets are MAHNOB HCI (Balconi et al., 2015), DEAP (Koelstra et al., 2012) and SEED (Petrantonakis et al., 2009). In this study, an emotion recognition system based on several recursive identification subsystems is conceived, and the DEAP dataset is chosen to validate the effectiveness of our results. ...
... Electroencephalogram (EEG), a brain imaging tool, was used to measure these neural oscillations during the stress period (Cohen, 2017;Sabow et al., 2018). Many studies show the relationship between stress and EEG activity in human (Abdul Hamid et al., 2010;Balconi et al., 2015;Hou et al., 2015). The analysis of the frequency from the EEG recording uses the Fast Fourier Transform (FFT) methods to obtain the signals' spectrum (Ang et al., 2017;Singh and Kanda, 2017). ...
Article
Full-text available
A comprehensive stress assessment is vital in understanding the impact of the pre-slaughter procedure on animal welfare. The transportation and handling process was commonly reported to cause stress in animals. This research utilises electroencephalography (EEG) as an alternative stress indicator to non-painful acute stress measurement. EEG has been proved to be instantaneous and sensitive with specific results. Therefore, this study was aimed to determine the stress level of goats subjected to two different transportation duration and the effect of lairage based on their EEG activities and blood parameters changes. Eighteen adult male goats were divided into two transportation stress groups based on the transport duration: the two-hour (TS2) and six-hour (TS6) groups. Then, each group was then again divided into three smaller groups according to the lairage duration, which was three-hour (L3), six-hour (L6), and overnight (L12) groups. Blood was sampled before transport, after transport, and during slaughter while EEG was recorded before transport, after transport, after lairage, and during slaughter. Results revealed that there was a significant decrease in beta wave activity compared to baseline in TS2 goats (P
... Emotion expression production and recognition play a decisive and central role in individuals' life. The consideration and the investigation of emotions result to be especially important allowing to comprehend individuals' emotional experiences and empathic mechanisms, representing driving knowledge for brain-computer interfaces (BCI), through the implementation of emotional patterns into artificial intelligence tools and computers, and for in-deep comprehension of psychopathology (Balconi et al., 2015a). ...
... Moreover, an increase in alpha and delta intra-brain connectivity was also observed in temporo-parietal areas in association with informative gestures. That different increases in the intra-brain connectivity depend on the gesture category might hint at the mirroring mechanisms of some cerebral regions that appear to be involved in action observation and execution [56][57][58][59][60]. The frontal and posterior regions appear to be involved in the mirroring mechanisms activated during action execution and the observation of the same action, creating an interdependent connection between encoder and decoder [17,18]. ...
Article
Full-text available
Recently, the neurosciences have become interested in the investigation of neural responses associated with the use of gestures. This study focuses on the relationship between the intra-brain and inter-brain connectivity mechanisms underlying the execution of different categories of gestures (positive and negative affective, social, and informative) characterizing non-verbal interactions between thirteen couples of subjects, each composed of an encoder and a decoder. The study results underline a similar modulation of intra- and inter-brain connectivity for alpha, delta, and theta frequency bands in specific areas (frontal or posterior regions) depending on the type of gesture. Moreover, taking into account the gestures’ valence (positive or negative), a similar modulation of intra- and inter-brain connectivity in the left and right sides was observed. This study showed congruence in the intra-brain and inter-brain connectivity trend during the execution of different gestures, underlining how non-verbal exchanges might be characterized by intra-brain phase alignment and implicit mechanisms of mirroring and synchronization between the two individuals involved in the social exchange.
... Differential lateral activation has been observed in the PFC, with right PFC activation associated with unpleasant emotional stimuli or the withdrawal system, and left PFC activation associated with pleasant emotional stimuli or the approach system (Davidson et al., 1990). This is known as the valence asymmetry hypothesis and has been observed in activation of the PFC through a variety of imaging modalities including EEG (Davidson, 1992), fMRI (Canli et al., 1998;Herrington et al., 2005), and fNIRS (Morinaga et al., 2007;Marumo et al., 2009;Tuscan et al., 2013;Balconi et al., 2015). ...
Article
Full-text available
Brain-computer interfaces (BCIs) are being investigated as an access pathway to communication for individuals with physical disabilities, as the technology obviates the need for voluntary motor control. However, to date, minimal research has investigated the use of BCIs for children. Traditional BCI communication paradigms may be suboptimal given that children with physical disabilities may face delays in cognitive development and acquisition of literacy skills. Instead, in this study we explored emotional state as an alternative access pathway to communication. We developed a pediatric BCI to identify positive and negative emotional states from changes in hemodynamic activity of the prefrontal cortex (PFC). To train and test the BCI, 10 neurotypical children aged 8–14 underwent a series of emotion-induction trials over four experimental sessions (one offline, three online) while their brain activity was measured with functional near-infrared spectroscopy (fNIRS). Visual neurofeedback was used to assist participants in regulating their emotional states and modulating their hemodynamic activity in response to the affective stimuli. Child-specific linear discriminant classifiers were trained on cumulatively available data from previous sessions and adaptively updated throughout each session. Average online valence classification exceeded chance across participants by the last two online sessions (with 7 and 8 of the 10 participants performing better than chance, respectively, in Sessions 3 and 4). There was a small significant positive correlation with online BCI performance and age, suggesting older participants were more successful at regulating their emotional state and/or brain activity. Variability was seen across participants in regards to BCI performance, hemodynamic response, and discriminatory features and channels. Retrospective offline analyses yielded accuracies comparable to those reported in adult affective BCI studies using fNIRS. Affective fNIRS-BCIs appear to be feasible for school-aged children, but to further gauge the practical potential of this type of BCI, replication with more training sessions, larger sample sizes, and end-users with disabilities is necessary.
... Regarding theta activity, instead, a greater theta brain responsiveness can be related to an increase of individuals' motivational and emotional response and affective arousal experienced during hotel environments exploration (Adolphs, 1999;Balconi & Vanutelli, 2016Balconi et al., 2015;Leanza et al., 2015). This result is supported by previous studies that have demonstrated the involvement of the theta band in the processes of emotional evaluation and emotional engagement in response to salient stimuli Knyazev, 2007;Leanza et al., 2015). ...
Article
Full-text available
Hoteliers and companies are asked to optimize their budgets and, at the same time, gifting unforgettable experiences to customers who are freely able to express their feedbacks, readable by thousands of potential guests. In addition, the rise of corporate social responsibility (CSR), sustainability, environmental and social issues leads to the necessity to investigate more systematically new business models that respond to these emerging concerns. In order to investigate the impact of sustainability in tourism marketing, a multimethod neuroscientific approach was used to record individuals’ neural activity (electroencephalography, EEG) and autonomic system responses during the exploration of four different environments of a green hotel adopting a sustainability approach. EEG findings showed an increase of beta and theta frequency band activity during individuals’ environments exploration, underlying the presence of cognitive and emotional elaboration processes. Moreover, from autonomic activity, an increase of pulse volume amplitude (PVA) and heart rate (HR) response emerged during the exploration of hotel environments, highlighting emotional engagement and positive emotional response. This research, therefore, contributes to show how responsiveness to sustainability concept within consumer services can improve individuals’ experience enjoyment and customers' well-being condition.
... Tragbare Biosensoren registrieren darüber hinaus in Echtzeit physiologische Daten über die neurale Aktivität (etwa durch in ein Kopfband eingebettete Elektroden zur Aufzeichnung mit Elektroenzephalografie ([EEG]), die Herzfrequenzvariabilität (HRV), die elektrodermale Aktivität (etwa der Füße, gemessen durch Sensoren in Socken oder Schuhen), oder den Hautleitwert, um sie mit mentalen Zuständen zu korrelieren und so Selbstberichte zu validieren [17]. Kombinierte KI-Technologien, die gleichzeitig Daten über die Gehirnaktivität und andere biologische Werte mittels Sprache, Mimik, Gestik und über elektronische Selbstberichte erfassen, verarbeiten, interpretieren und daraus Handlungsoptionen generieren, sind in Forschung und Praxis zukünftig durchaus vorstellbar [19][20][21]. ...
Article
Objective: AI-based applications are increasingly developed to support users to digitally record, manage and change their emotions, beliefs and behavior patterns. Such forms of self-tracking in the mental sphere are accompanied by a variety of medical benefits in diagnostics, prevention, and therapy. This article pursues the question of which philosophical-ethical implications must be taken into account when dealing with these advantages. Methods: First, some AI-based applications for self-tracking of mental characteristics and processes are outlined. Subsequently, relevant philosophical-ethical implications are presented. Results: The following aspects prove to be normatively relevant: improvement versus reduction of self-determination; improvement of self-knowledge versus alienation; positive versus negative aspects of self-responsible health care; epistemic challenges of AI applications; difficulties of conceptual and normative definitions in the applications.
... A seminal work [9] showed that fNIRS is sensitive towards changes in task difficulty in both real-life (flight simulator) and laboratory settings (executive functions test batteries), where increased concentration of oxygenated haemoglobin (O2Hb) and decreased concentration of deoxygenated haemoglobin (HHb) were observed as the tasks became more complex. Another study [10] combining EEG, fNIRS and skin conductance response (SCR) measures showed that these physiological signals correlated with self-reported valence and arousal levels of emotion. fNIRS showed increased O2Hb activity in right pre-frontal cortex (PFC) induced by negative emotions while EEG showed delta and theta band activity was correlated with cortical haemodynamic responsiveness to negative emotions. ...
Article
Full-text available
The effect of stress on task performance is complex, too much or too little stress negatively affects performance and there exists an optimal level of stress to drive optimal performance. Task difficulty and external affective factors are distinct stressors that impact cognitive performance. Neuroimaging studies showed that mood affects working memory performance and the correlates are changes in haemodynamic activity in the prefrontal cortex (PFC). We investigate the interactive effects of affective states and working memory load (WML) on working memory task performance and haemodynamic activity using functional near-infrared spectroscopy (fNIRS) neuroimaging on the PFC of healthy participants. We seek to understand if haemodynamic responses could tell apart workload-related stress from situational stress arising from external affective distraction. We found that the haemodynamic changes towards affective stressor- and workload-related stress were more dominant in the medial and lateral PFC, respectively. Our study reveals distinct affective state-dependent modulations of haemodynamic activity with increasing WML in n-back tasks, which correlate with decreasing performance. The influence of a negative effect on performance is greater at higher WML, and haemodynamic activity showed evident changes in temporal, and both spatial and strength of activation differently with WML.
... Therefore, any multimodal fNIRS study should first describe the motivations to combine fNIRS with another modality, which are usually falling within one of the four main categories listed below: (1) providing improved quantification of brain hemodynamics and oxygenation (e.g., combining hemoglobin measurements with fMRI, DCS, or CCO measurements to provide quantitative measurements of physiologically interpretable parameters such as CMRO 2 and hemoglobin 22,[140][141][142][143][144][145] ), (2) assessing brain activity at the time of complex or transient events, usually monitored and detected using scalp EEG 146 (e.g., prolonged recordings to characterize hemodynamic responses to epileptic discharges, [136][137][138][147][148][149] sleep physiology 150,151 and sleep disorders, 152 or resting-state fluctuations 153 ), (3) monitoring brain activity in real time for brain-computer interfaces 154 and during noninvasive brain stimulation, 155 or (4) when experimental designs involving complex cognitive processes can benefit from simultaneous recordings to better explore the underlying complex neural processes (e.g., language, learning, attention, intention, emotion). [156][157][158] When reporting fNIRS multimodal studies, the set up of the acquisition should be carefully described, especially the methods considered to synchronize the different modalities in time. To fully benefit from the added value of multimodal approaches, accurate sensor localization and coregistration are also helpful, 35 notably through the use of neuro-navigation tools. ...
... The social neurosciences are now greatly benefiting from hyperscanning of socially interacting people [26]. Social perception and interaction studies in adults and children are being conducted to identify brain-behavior relations, such as imitation behavior [27], emotion perception [28], affective touch [29] and deceptive [30] and cooperative games [31]. Wearable systems also enable unconstrained studies in clinical settings, as needed for rehabilitation, neurological diseases [32], developmental neuroscience (typical, mental retardation [33], autism, attention deficit hyperactivity disorder, traumatic brain injury, neurodevelopment in rural areas [34]), psychiatric disorders (bipolar disorder, schizophrenia, and game addiction) [35], and intraoperative studies. ...
Article
Functional Near-Infrared Spectroscopy (fNIRS) assesses human brain activity by noninvasively measuring changes of cerebral hemoglobin concentrations caused by modulation of neuronal activity. Recent progress in signal processing and advances in system design, such as miniaturization, wearability and system sensitivity, have strengthened fNIRS as a viable and cost-effective complement to functional Magnetic Resonance Imaging (fMRI), expanding the repertoire of experimental studies that can be performed by the neuroscience community. The availability of fNIRS and Electroencephalography (EEG) for routine, increasingly unconstrained, and mobile brain imaging is leading towards a new domain that we term “Neuroscience of the Everyday World” (NEW). In this light, we review recent advances in hardware, study design and signal processing, and discuss challenges and future directions towards achieving NEW.
... Therefore, any multimodal fNIRS study should first describe the motivations to combine fNIRS with another modality, which are usually falling within one of the four main categories listed below: (1) providing improved quantification of brain hemodynamics and oxygenation (e.g., combining hemoglobin measurements with fMRI, DCS, or CCO measurements to provide quantitative measurements of physiologically interpretable parameters such as CMRO 2 and hemoglobin 22,[140][141][142][143][144][145] ), (2) assessing brain activity at the time of complex or transient events, usually monitored and detected using scalp EEG 146 (e.g., prolonged recordings to characterize hemodynamic responses to epileptic discharges, [136][137][138][147][148][149] sleep physiology 150,151 and sleep disorders, 152 or resting-state fluctuations 153 ), (3) monitoring brain activity in real time for brain-computer interfaces 154 and during noninvasive brain stimulation, 155 or (4) when experimental designs involving complex cognitive processes can benefit from simultaneous recordings to better explore the underlying complex neural processes (e.g., language, learning, attention, intention, emotion). [156][157][158] When reporting fNIRS multimodal studies, the set up of the acquisition should be carefully described, especially the methods considered to synchronize the different modalities in time. To fully benefit from the added value of multimodal approaches, accurate sensor localization and coregistration are also helpful, 35 notably through the use of neuro-navigation tools. ...
Article
Full-text available
The application of functional near-infrared spectroscopy (fNIRS) in the neurosciences has been expanding over the last 40 years. Today, it is addressing a wide range of applications within different populations and utilizes a great variety of experimental paradigms. With the rapid growth and the diversification of research methods, some inconsistencies are appearing in the way in which methods are presented, which can make the interpretation and replication of studies unnecessarily challenging. The Society for Functional Near-Infrared Spectroscopy has thus been motivated to organize a representative (but not exhaustive) group of leaders in the field to build a consensus on the best practices for describing the methods utilized in fNIRS studies. Our paper has been designed to provide guidelines to help enhance the reliability, repeatability, and traceability of reported fNIRS studies and encourage best practices throughout the community. A checklist is provided to guide authors in the preparation of their manuscripts and to assist reviewers when evaluating fNIRS papers.
... Partial changes in HbO signals are common parameters of NIRS and are closely related to changes in brain nerve activity [28] that indirectly measure changes in brain nerve activity by detecting hemodynamic changes in the cerebral cortex [29]. It is generally accepted that hemodynamic and electroneurographic signals (e.g., EEG) provide supplementary information about the underlying neural mechanisms in various cognitive functions [30]. Using fMRI (functional magnetic resonance imaging) or fNIRS (functional near-infrared spectroscopy) technology, many studies have demonstrated identifiable hemodynamic response data between positive and negative emotions [31]. ...
Article
Full-text available
This article aimed to explore whether there are emotional conflicts and gender differences in the cognitive processing of user interface layouts. By taking the basic layout framework of a shopping interface as the research object and using behavioral experiments and NIRS (Near-Infrared Spectroscopy) technology, this study evaluated the emotional conflict response of men and women to 9 interface layouts formed by the combination of different numbers of layout elements. The results showed that there is a gender difference (p < 0.05) in cognition of the interface layout with combinations of 3 and 4 layout elements in the behavioral experiment. Behavioral response was positively correlated with brain function. Oriented by positive emotions, the HbO (oxygenated hemoglobin in the brain) concentration in the cerebral cortex is significantly activated (p < 0.05) with gender differences in the case of interfaces combining four elements in the H-type layout frame and three elements in the left-I-type layout frame. These research conclusions can be extended to the application of personalized design and evaluation process of the interface and provide a reference for style changes to digital interface products in actual projects.
... While considerable progress has been made in understanding the principles of brain functions and organization that underlie affective processing [13,14], there is currently a gap between research in the fields of affective neuroscience and HCI. Most studies primarily use well-established and standardized stimulus material such as pictures, sounds or movies to investigate the brain mechanisms of affect [15][16][17][18][19]. However, those materials may lack ecological validity [20] to investigate affective user reactions in real-world HCI applications. ...
Article
Full-text available
Affect monitoring is being discussed as a novel strategy to make adaptive systems more user-oriented. Basic knowledge about oscillatory processes and functional connectivity underlying affect during naturalistic human–computer interactions (HCI) is, however, scarce. This study assessed local oscillatory power entrainment and distributed functional connectivity in a close-to-naturalistic HCI-paradigm. Sixteen participants interacted with a simulated assistance system which deliberately evoked positive (supporting goal-achievement) and negative (impeding goal-achievement) affective reactions. Electroencephalography (EEG) was used to examine the reactivity of the cortical system during the interaction by studying both event-related (de-)synchronization (ERD/ERS) and event-related functional coupling of cortical networks towards system-initiated assistance. Significantly higher α-band and β-band ERD in centro-parietal and parieto-occipital regions and β-band ERD in bi-lateral fronto-central regions were observed during impeding system behavior. Supportive system behavior activated significantly higher γ-band ERS in bi-hemispheric parietal-occipital regions. This was accompanied by functional coupling of remote β-band and γ-band activity in the medial frontal, left fronto-central and parietal regions, respectively. Our findings identify oscillatory signatures of positive and negative affective processes as reactions to system-initiated assistance. The findings contribute to the development of EEG-based neuroadaptive assistance loops by suggesting a non-obtrusive method for monitoring affect in HCI.
... Electroencephalogram (EEG), a brain imaging tool, was used to measure these neural oscillations during the stress period (Cohen, 2017;Sabow et al., 2018). Many studies show the relationship between stress and EEG activity in human (Abdul Hamid et al., 2010;Balconi et al., 2015;Hou et al., 2015). The analysis of the frequency from the EEG recording uses the Fast Fourier Transform (FFT) methods to obtain the signals' spectrum (Ang et al., 2017;Singh & Kanda, 2017). ...
Article
Full-text available
This preliminary trial investigated the effect of transportation and lairage periods on physiological parameters of goats subjected to slaughter. Nine male Boer cross goats aged 8-12 mo were transported for 6 h and kept at lairage for 3, 6, or 16 h (n=3). Blood samples were collected at pre- (Pre-T) and post-transportation (Post-T), and post-slaughter (Post-S) for determination of hematological parameters, serum enzyme, protein, and cortisol concentrations. Electroencephalogram readings were taken at Pre-T, Post-T, pre-slaughter (Pre-S) and Post-S to determine the median frequency (F50) and total power (Ptot) values. At Post-T, there were manifestations of stress leukogram, increase in hematocrit, total protein and muscle enzyme concentrations, and decrease in Ptot (p<0.05). The high Pre-T cortisol concentration suggests that the goats were already under stress before transportation. Stress leukogram became less evident after lairage, indicating that the goats had recovered from the stress of transportation. Although the Ptot increased at Post-S especially following 3 h of lairage, F50 values at Post-S did not differ from Pre-L, suggesting that the pre-slaughter stress may have affected the pain threshold. It is suggested that after 6 h of transportation, goats should ideally be placed in lairage for a minimum period of 3 h before slaughter.
... To address these concerns, adequate basic research is required before the application of NIRS in clinical practice. Complementary use of other techniques, such as fMRI (Kirilina et al., 2012;Sato et al., 2013;Haeussinger et al., 2014) and physiological measurements (Balconi et al., 2015;Pinti et al., 2015), can be beneficial. ...
Article
Full-text available
This perspective article discusses the importance of evidence-based psychotherapy and highlights the usefulness of near-infrared spectroscopy (NIRS) in assessing the effects of psychotherapeutic interventions as a future direction of clinical psychology. NIRS is a safe and non-invasive neuroimaging technique that can be implemented in a clinical setting to measure brain activity via a simple procedure. This article discusses the possible benefits and challenges of applying NIRS for this purpose, and the available methodology based on previous studies that used NIRS to evaluate psychotherapeutic effects. Furthermore, this perspective article suggests alternative methodologies that may be useful, namely, the single- and multi-session evaluations using immediate pre- and post-intervention measurements. These methods can be used to evaluate state changes in brain activity, which can be derived from a single session of psychotherapeutic interventions. This article provides a conceptual schema important in actualizing NIRS application for evidence-base psychotherapy.
... Wallois et al. used EEG-NIRS to synchronously collect information to study the brain activation mechanism [14]. In 2015, Balconi et al. studied the connection between EEG-NIRS and emotions, and the findings revealed the correlation between cortical forward networks and emotional stimuli [15]. Fazli et al. used two devices to synchronize information and improved the recognition accuracy of motion imagination to 90% [16]. ...
Article
Most studies on drug addiction degree are made based on statistical scales, addicts' account, and subjective judgement of rehabilitation doctors. No objective, quantified evaluation has been made. This paper uses devises the synchronous bimodal signal collection and experimentation paradigm with electroencephalogram (EEG) and forehead high-density near-infrared spectroscopy (NIRS) device. The drug addicts are classified into mild, moderate and severe groups with reference to the suggestions of researchers and medical experts. Data of 45 drug addicts (mild: 15; moderate: 15; and severe: 15) is collected, and then used to design an addiction degree testing algorithm based on decision fusion. The algorithm is used to classify mild, moderate and severe addiction. This paper pioneers to use two types of Convolutional Neural Network (CNN) to abstract the EEG and NIR data of drug addicts, and introduces batch normalization to CNN, thus accelerating training process, reducing parameter sensitivity, and enhancing system robustness. The characteristics output by two CNNs are transformed into dimensions. Two new characteristics are assigned with a weight of 50% each. The data is used for decision fusion. In the networks, 27 subjects are used as training sets, 9 as validation sets, and 9 as testing sets. The 3-class accuracy remains to be 63.15%, preliminarily justifying this method as an effective approach to measure drug addiction degree. And the method is ready to use, objective, and offers results in real time.
... We study multi-class EEG-connectome emotion regulation tasks and analyze the effect of different frequency bands of EEG signals. In emotion regulation, studies have shown that relevant EEG information is primarily encoded in the low frequency bands [18]. Thus, we analyze the EEGconnectome data in 5 frequency bands: Delta (1-3 Hz), Theta (4-7 Hz), Alpha (8-12 Hz), Beta (13-30 Hz), for relative power, as well as the total power of the EEG (1-30 Hz) [19]. ...
Preprint
This paper presents a novel graph-based kernel learning approach for connectome analysis. Specifically, we demonstrate how to leverage the naturally available structure within the graph representation to encode prior knowledge in the kernel. We first proposed a matrix factorization to directly extract structural features from natural symmetric graph representations of connectome data. We then used them to derive a structure-persevering graph kernel to be fed into the support vector machine. The proposed approach has the advantage of being clinically interpretable. Quantitative evaluations on challenging HIV disease classification (DTI- and fMRI-derived connectome data) and emotion recognition (EEG-derived connectome data) tasks demonstrate the superior performance of our proposed methods against the state-of-the-art. Results showed that relevant EEG-connectome information is primarily encoded in the alpha band during the emotion regulation task.
... Regardless of the issue investigated, little conceptual work is exploited to predict or account for the patterns reported. This point might seem unfortunate given the considerable potential of bimodal EEG-fNIRS in moving neuroergonomics research forward at both fundamental and practical levels (Pfurtscheller et al., 2012;Wallois et al., 2012;Balconi et al., 2015;Pinti et al., 2018b). The development of advanced tools for investigating the neurocognitive correlates of behavior does not replace careful theoretical analysis of the phenomenon addressed (Krakauer et al., 2017). ...
Article
Full-text available
Neuroergonomics focuses on the brain signatures and associated mental states underlying behavior to design human-machine interfaces enhancing performance in the cognitive and physical domains. Brain imaging techniques such as functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) have been considered key methods for achieving this goal. Recent research stresses the value of combining EEG and fNIRS in improving these interface systems' mental state decoding abilities, but little is known about whether these improvements generalize over different paradigms and methodologies, nor about the potentialities for using these systems in the real world. We review 33 studies comparing mental state decoding accuracy between bimodal EEG-fNIRS and unimodal EEG and fNIRS in several subdomains of neuroergonomics. In light of these studies, we also consider the challenges of exploiting wearable versions of these systems in real-world contexts. Overall the studies reviewed suggest that bimodal EEG-fNIRS outperforms unimodal EEG or fNIRS despite major differences in their conceptual and methodological aspects. Much work however remains to be done to reach practical applications of bimodal EEG-fNIRS in naturalistic conditions. We consider these points to identify aspects of bimodal EEG-fNIRS research in which progress is expected or desired.
... The mean of the fNIRS signal intensities during the rest interval was subtracted from that during each task interval. Then, the signal intensities during each interval were converted to Z-scores by dividing them by the standard deviation of the signal intensity during the rest interval [58][59][60][61] . The channels whose mean signal intensities exceeded the mean ± two standard deviations (SD) of all channel data were excluded from the analysis. ...
Article
Full-text available
The effects of physical exercise on cognitive tasks have been investigated. However, it is unclear how different exercise intensities affect the neural activity. In this study, we investigated the neural activity in the prefrontal cortex (PFC) by varying the exercise intensity while participants performed a dual task (DT). Twenty healthy young adults performed serial subtraction while driving a cycle ergometer. Exercise intensity was set to one of three levels: low, moderate, or high intensity. We did not find any significant change in PFC activity during DT under either the control (no exercise) or low-intensity conditions. In contrast, we observed a significant increase in PFC activity during DT under moderate- and high-intensity conditions. In addition, we observed complex hemodynamics after DT. PFC activity decreased from baseline after DT under the control condition, while it increased under the low-intensity condition. PFC activity remained higher than the baseline level after DT under the moderate-intensity condition but returned to baseline under the high-intensity condition. The results suggest that moderate-intensity exercise with a cognitive load effectively increases PFC activity, and low-intensity exercise may increase PFC activity when combined with a cognitive load.
... For each stimulus trial, a hemodynamic effect size metric named Cohen's D was computed by subtracting the mean of the signal in the [0-3] s pre-stimulus time range before the stimulus onset from the mean of the signal in the [1-4] s duration after the stimulus onset (Balconi and Molteni, 2016). Stimulus induced amplitude change with respect to the baseline was normalized by dividing the difference between the mean baseline amplitude and task mean amplitude with the standard deviation of the [1-3] s pre-stimulus baseline (Balconi et al., 2015;Vanutelli, 2016, 2017;Mutlu et al., 2020), (Figure 3). ...
Article
Full-text available
Functional near infrared spectroscopy (fNIRS) has been gaining increasing interest as a practical mobile functional brain imaging technology for understanding the neural correlates of social cognition and emotional processing in the human prefrontal cortex (PFC). Considering the cognitive complexity of human-robot interactions, the aim of this study was to explore the neural correlates of emotional processing of congruent and incongruent pairs of human and robot audio-visual stimuli in the human PFC with fNIRS methodology. Hemodynamic responses from the PFC region of 29 subjects were recorded with fNIRS during an experimental paradigm which consisted of auditory and visual presentation of human and robot stimuli. Distinct neural responses to human and robot stimuli were detected at the dorsolateral prefrontal cortex (DLPFC) and orbitofrontal cortex (OFC) regions. Presentation of robot voice elicited significantly less hemodynamic response than presentation of human voice in a left OFC channel. Meanwhile, processing of human faces elicited significantly higher hemodynamic activity when compared to processing of robot faces in two left DLPFC channels and a left OFC channel. Significant correlation between the hemodynamic and behavioral responses for the face-voice mismatch effect was found in the left OFC. Our results highlight the potential of fNIRS for unraveling the neural processing of human and robot audio-visual stimuli, which might enable optimization of social robot designs and contribute to elucidation of the neural processing of human and robot stimuli in the PFC in naturalistic conditions.
... A digital band-pass filter was adopted to filter the raw data at 0.01-0.3 Hz (Balconi et al., 2015;Pinti et al., 2019). To detect noisy channels due to motion artifacts or amplitude changes raw time-series were visually inspected subject-by-subject both during the experimental phase and the signal analysis. ...
Article
Full-text available
This research explored the effect of explicit Interoceptive Attentiveness (IA) manipulation on hemodynamic brain correlates during a task involving interpersonal motor coordination framed with a social goal. Participants performed a task requiring interpersonal movement synchrony with and without a social framing in both explicit IA and control conditions. Functional Near-Infrared Spectroscopy (fNIRS) was used to record oxygenated (O2Hb) and deoxygenated hemoglobin (HHb) changes during the tasks. According to the results, the prefrontal cortex (PFC), which is involved in high-order social cognition and interpersonal relations processing, was more responsive when inducing the explicit focus (IA) on the breath during the socially framed motor task requiring synchronization, as indicated by increased O2Hb. In the absence of a broader social frame, this effect was not significant for the motor task. Overall, the present study suggests that when a joint task is performed and the individual focuses on his/her physiological body reactions, the brain hemodynamic correlates are “boosted” in neuroanatomical regions that support sustained attention, reorientation of attention, social responsiveness, and synchronization. Furthermore, the PFC responds significantly more as the person consciously focuses on physiological interoceptive correlates and performs a motor task requiring synchronization, particularly when the task is socially framed.
... Here, valence was represented from negative to positive, and arousal was represented from calm to excitement. It is necessary to divide the international affective picture system (IAPS) images into valence and arousal to induce positive or negative emotions in participants [40]. Based on this model, 100 positive and 100 negative images were chosen from the IAPS image dataset. ...
Article
Full-text available
A stress group should be subdivided into eustress (low-stress) and distress (high-stress) groups to better evaluate personal cognitive abilities and mental/physical health. However, it is challenging because of the inconsistent pattern in brain activation. We aimed to ascertain the necessity of subdividing the stress groups. The stress group was screened by salivary alpha-amylase (sAA) and then, the brain’s hemodynamic reactions were measured by functional near-infrared spectroscopy (fNIRS) based on the near-infrared biosensor. We compared the two stress subgroups categorized by sAA using a newly designed emotional stimulus-response paradigm with an international affective picture system (IAPS) to enhance hemodynamic signals induced by the target effect. We calculated the laterality index for stress (LIS) from the measured signals to identify the dominantly activated cortex in both the subgroups. Both the stress groups exhibited brain activity in the right frontal cortex. Specifically, the eustress group exhibited the largest brain activity, whereas the distress group exhibited recessive brain activity, regardless of positive or negative stimuli. LIS values were larger in the order of the eustress, control, and distress groups; this indicates that the stress group can be divided into eustress and distress groups. We built a foundation for subdividing stress groups into eustress and distress groups using fNIRS.
... On the whole, we suggest that the integration of multilevel measures could be useful to unveil the neurophysiological complexity of normal and abnormal decision-making processes with regard to emotional and rational components in gambling and moral behaviors, as previously demonstrated even by basic research on emotions (Balconi et al., 2015;Balconi & Terenzi, 2012). ...
Article
Within the neuroeconomics field, there are two evident situations in which decisionmaking process do not respect the rule of expected utility: gambling and moral behaviors. In the case of gambling behavior, a tendency to engage in risky decision-making could lead to choose disadvantageous options (loss vs gain) and long-term negative economic consequences. Regarding moral behavior, subjects prefer options not always related to their expected utility, but more to their social and ethical significance (fair vs unfair). This commentary discusses both the theoretical and empirical basis of these behaviors, focusing on neurophysiological methods adopted to investigate commonalities and differences in physiological and behavioral subjects’ responses. The dichotomy between emotions and rationality will be explored considering two popular economics games, Iowa Gambling Task and Ultimatum Game, and will be discussed in the light of somatic marker hypothesis frame. We propose a multidimensional approach to describe more in-depth real-world decision-making situations in neuroeconomics.
... Based on PSD topography of competing dyads, it seems that W won due to their ability to detach, by maintaining RA at baseline level, as well as because L decreased RA during the game. From literature on the relation between theta activity and emotional processing (Yun et al., 2012;Balconi et al., 2015;Balconi and Vanutelli, 2018), W group might have also been less involved emotionally in the game than the latter. However, these remain speculations, as NASA-TLX results did not indicate significant differences between the groups. ...
Article
Full-text available
EEG hyperscanning during multiuser gaming offers opportunities to study brain characteristics of social interaction under various paradigms. In this study, we aimed to characterize neural signatures and phase-based functional connectivity patterns of gaming strategies during collaborative and competitive alpha neurofeedback games. Twenty pairs of participants with no close relationship took part in three sessions of collaborative or competitive multiuser neurofeedback (NF), with identical graphical user interface, using Relative Alpha (RA) power as a control signal. Collaborating dyads had to keep their RA within 5% of each other for the team to be awarded a point, while members of competitive dyads scored points if their RA was 10% above their opponent's. Interbrain synchrony existed only during gaming but not during baseline in either collaborative or competitive gaming. Spectral analysis and interbrain connectivity showed that in collaborative gaming, players with higher resting state alpha content were more active in regulating their RA to match those of their partner. Moreover, interconnectivity was the strongest between homologous brain structures of the dyad in theta and alpha bands, indicating a similar degree of planning and social exchange. Competitive gaming emphasized the difference between participants who were able to relax and, in this way, maintain RA, and those who had an unsuccessful approach. Analysis of interbrain connections shows engagement of frontal areas in losers, but not in winners, indicating the formers' attempt to mentalise and apply strategies that might be suitable for conventional gaming, but inappropriate for the alpha neurofeedback-based game. We show that in gaming based on multiplayer non-verbalized NF, the winning strategy is dependent on the rules of the game and on the behavior of the opponent. Mental strategies that characterize successful gaming in the physical world might not be adequate for NF-based gaming.
... Moreover, combined technologies making use of brain data as well as other data about a person via, e.g. recording speech, facial expressions, or gestures, are tested in research (Balconi et al., 2015;Dijksterhuis et al., 2013;Mohamad et al., 2015). ...
Article
Recently, the epistemic quality of algorithms and its normative implications have come under scrutiny. While general questions of justice have been addressed in this context, specific issues of epistemic (in)justice have so far been neglected. We aim to fill this gap by analyzing some potential implications of behavioral intelligent neurotechnology (B-INT). We claim that B-INT exhibits a number of epistemic features implying the potential for certain epistemic problems, which, in turn, are likely to result in instances of epistemic injustice. To support this claim, we will first introduce and specify the terminology and technology behind B-INT. Second, we will present four fictitious scenarios of using B-INT and highlight a number of epistemic issues that might arise. Third, we will discuss their relation to the concept of epistemic justice, as well as potential instances thereof. Thus, we will show some important and morally relevant implications of the epistemic properties of INT.
Article
A controversial issue in artificial intelligence is human emotion recognition. This paper presents a fuzzy parallel cascades (FPC) model for predicting the continuous subjective emotional appraisal of music by time-varying spectral content of electroencephalogram (EEG) signals. The EEG, along with an emotional appraisal of 15 subjects, was recorded during listening to seven musical excerpts. The emotional appraisement was recorded along the valence and arousal emotional axes as a continuous signal. The FPC model was composed of parallel cascades with each cascade containing a fuzzy logic-based system. The FPC model performance was evaluated by comparing with linear regression (LR), support vector regression (SVR), and Long-Short-Term-Memory recurrent neural network (LSTM-RNN) models by 4 fold cross-validation. The root mean square error (RMSE) of the FPC was lower than other models in the estimation of both valence and arousal of all musical excerpts. The lowest obtained RMSE was 0.082, which was acquired by the FPC model. The analysis of mutual information of frontal EEG with the valence confirms the role of frontal channels in the theta frequency band in emotion recognition. Considering the dynamic variations of musical features during songs, employing a modeling approach to predict dynamic variations of the emotional appraisal can be a plausible substitute for the classification of musical excerpts into predefined labels.
Chapter
While psycho-physiological changes have extensively been characterized from a single-system viewpoint, cross-system correlates of emotion especially linked to a functional BHI are yet unknown.
Chapter
Advances in neuroimaging and electrophysiology have fostered great achievements in many heterogeneous research and clinical domains, including cardiology, neurology, psychiatry, and neuroscience. These techniques, however, have primarily focused on ad hoc physiological systems and domain-specific conclusions, and cross-system exchange and cross-modality investigations have been almost completely neglected. For example, although neural disorders have very notable effects on cardiovascular control and, likewise, heart diseases strongly impact on brain functioning, to-date standard of care refers to system-specific epidemiology only, thus failing to account for a whole-body, hyper-dimensional systemic responsiveness. In this chapter, the anatomical and especially the functional links between the central and autonomic nervous systems are reported, picturing the clinical and neurophysiological bases of functional BHI physiology. Knowledge highlighted in this chapter serves as fundamental basis for the understanding of the signal processing methods and applications reported in the next chapters.
Thesis
Hintergrund: Das Catechol-O-Methyltransferase-Gen (COMT) ist ein vielversprechendes Kandidatengen zur Untersuchung kognitiver und emotionaler Funktionen sowie deren pathologischer Veränderungen. Ein einzelner Basenaustausch in diesem Gen führt zu einer 3-4fach höheren COMT-Aktivität der Val Variante. Ein dadurch vermitteltes dopaminerges Defizit wird als relevanter Faktor für eine veränderte Hirnfunktion angenommen. Mit dem kognitiven Stroop-Paradigma wurden kognitive Verarbeitungsprozesse bisher gut erforscht. Zur Erfassung emotionaler Verarbeitungsprozesse wurde eine emotionale Variante entwickelt, deren neurale Grundlagen bislang weniger gut bekannt sind. Ziel: Unsere imaging genetics-Arbeit untersucht den Einfluss genetischer Varianten auf die neurale Funktion. Ziel dieser experimentellen Arbeit war es, den Einfluss des COMT-Polymorphismus (COMT-PM) auf die Frontalkortex-Funktion in ausgewählten Regionen von Interesse (ROI) zu erfassen und der Frage nachzugehen, ob das Val-Allel als Risiko-Allel zur Pathogenese einer Angststörung (AS) beitragen könnte. Zudem sollte die Tauglichkeit des emotionalen Stroop- Paradigmas als angstsensibles Messinstrument zur Untersuchung dieser Fragestellung geprüft werden. Demgegenüber steht die Annahme, das emotionale Stroop-Paradigma könnte lediglich eine Arbeitsgedächtnis (AG)-Aufgabe darstellen. Methoden: Mittels funktioneller Nahinfrarotspektroskopie (fNIRS) und ereigniskorrelierter Potentiale untersuchten wir 121 gesunde nach dem COMT- Val158Met-PM stratifizierte Probanden während eines kombiniert emotional- kognitiven Stroop-Paradigmas. Als neurale Korrelate von Exekutivfunktionen und AG-Aufgaben waren die ROI dabei der laterale präfrontale und inferiore Kortex, die auch mit emotionaler Regulation in Verbindung gebracht werden. Als Parameter der Reaktion des autonomen Nervensystems (ANS) diente die Erfassung der elektrodermalen Aktivität sowie die kontinuierliche Messung von Blutdruck, Herzfrequenz und Herzratenvariabilität. Ergebnisse: Bei allen drei COMT Varianten zeigte sich ein kognitiver Stroop-Effekt mit verlängerter Reaktionszeit und erhöhter Fehleranzahl während der Präsentation inkongruenter Farbworte. Als Reaktion des ANS stellte sich eine erhöhte elektrodermale Aktivität bei inkongruenten Farbworten dar. Die funktionelle Bildgebung ließ in den analysierten Regionen eine erhöhte präfrontale Aktivierung während der Verarbeitung inkongruenter Farbworte nachweisen. Es fanden sich keine Gruppenunterschiede im kognitiven Stroop-Paradigma. Der einzige emotionale Stroop-Effekt zeigte sich in der P300. Der einzig nachweisbare Gruppeneffekt stellte sich im emotionalen Stroop-Paradigma als höhere Fehleranzahl bei Met-Homozygoten verglichen mit Heterozygoten dar. Schlussfolgerung: Genetische Information und funktionelle Bildgebung kombiniert sollten ermöglichen, neurale Mechanismen zu definieren, die mit genetischen Varianten verlinkt sind. Die Ergebnisse bezogen auf die analysierten Regionen liefern keinen Hinweis auf ein Val-Allel assoziiertes Risiko für die Entwicklung einer AS. Damit gelingt es nicht, bisher gewonnene Ergebnisse zum Einfluss des COMT-PM auf die präfrontale Funktion zu replizieren. Fraglich ist jedoch, ob sich das emotionale Stroop-Paradigma zur Untersuchung dieser Frage eignet, da weder in den fNIRS-, noch in den autonomen oder Verhaltensdaten ein emotionaler Stroop-Effekt nachgewiesen werden konnte.
Article
Full-text available
Functional neuroimaging provides an avenue for earlier diagnosis and tailored treatment of psychological disorders characterized by emotional impairment. Near-infrared spectroscopy offers ecological advantages compared to other neuroimaging techniques and suitability of measuring regions involved in emotion functions. A systematic review was conducted to evaluate the capacity of NIRS to detect activation during emotion processing and to provide recommendations for future research. Following a comprehensive literature search, we reviewed 85 journal articles which compared activation during emotional experience, regulation or perception with either a neutral condition or baseline period among healthy participants. The quantitative synthesis of outcomes was limited to thematical analysis, owing to the lack of standardisation between studies. Although most studies found increased prefrontal activity during emotional experience and regulation, the findings were more inconsistent for emotion perception. Some researchers reported increased activity during the task, some reported decreases, some no significant changes, and some reported mixed findings depending on the valence and region. We propose that variations in the cognitive task and stimuli, recruited sample, and measurement and analysis of data are the primary causes of inconsistency. Recommendations to improve consistency in future research by carefully considering the choice of population, cognitive task and analysis approach are provided.
Article
Objective: The aim of this study was to test the feasibility of a functional near infrared spectroscopy (fNIRS) system for quantification of the similarities and differences in the spatial localization of cerebral hemodynamic activation induced by visual presentation of neutral, negative and positive valence emotional stimuli. Method: 13 healthy subjects viewed neutral, pleasant and unpleasant pictures from the International Affective Picture System (IAPS) database in a block design experiment while the prefrontal cortical hemodynamic changes induced by emotional stimuli were continuously recorded with a 20 channel fNIRS system that covered the forehead region. Results: Negative valence pictures induced higher hemodynamic activity in right lateralized regions involving dorsolateral and oribtofrontal cortex when compared to neutral and positive valence stimuli (pFDR
Article
Full-text available
Introduction: Many cognitive and social deficits in autism are caused by abnormal functional connections between brain networks, which are manifested by impaired integrity of white matter tracts. White matter tracts are like the "highways" of the brain, which allow fast and efficient communication in different areas of the brain. The purpose of this article is to review the results of autism studies that have used diffusion tensor images (DTI). Diffusion tensor images is a neuroimaging technique to examine the integrity of tracts. Conclusion: The results of these studies suggest that neural tracts can be abnormal in people with Autism spectrum disorder (ASD) due to impaired white matter integrity. Thus, changes in these tracts in the brains of people with ASD are helpful in identifying individual differences. Although most studies have reported decreased FA and increased MD, RD, and AD in white matter tracts, some studies have reported increased FA or no significant difference between the control and autistic groups.
Chapter
Mental disorders caused by chronic stress are difficult to identify, and colleagues in the work environment may suddenly report symptoms. Social barriers exist including the financial cost of medical services and the lack of a perceived need for treatment even if potential patients have a desire to receive mental healthcare. Self-report inventories such as the Beck Depression Inventory (BDI-II) and State-Trait Anxiety Inventory (STAI) can assess the emotional valence for mental health assessment, but medical expertise may be required for interpretation of the results. Contingency plans for clinical supervision and referral sources are necessary for sufficient mental healthcare using self-report inventories. On the other hand, the laterality index at rest (LIR) has been proposed for evaluation of the mental stress level from near-infrared spectroscopy (NIRS) data in the prefrontal cortex in the resting state. However, the potential for long-term monitoring has not been investigated with sufficient evaluation results. In this study, feature values were extracted from both NIRS and EEG signals each week for 10 weeks in four young participants with an average BDI-II score of 17.7, i.e., indicative of mild depression. Temporal changes in LIR and heart rate (HR) were compared with STAI-Y1 and BDI-II scores. We found cross-correlations between the time series of LIR and STAI-Y1 within one-week delay. In addition, the time series of LIR was also correlated with BDI-II with one-week delay. Importantly, by annotating the larger changes in LIR and HR on daily life events, the changes in LIR and HR were different depending on the type of life event that affected these moods.
Article
Neurovascular coupling is a key physiological mechanism that occurs in the healthy human brain, and understanding this process has implications for understanding the aging and neuropsychiatric populations. Combined electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) has emerged as a promising, noninvasive tool for probing neurovascular interactions in humans. However, the utility of this approach critically depends on the methodological quality used for multimodal integration. Despite a growing number of combined EEG–fNIRS applications reported in recent years, the methodological rigor of past studies remains unclear, limiting the accurate interpretation of reported findings and hindering the translational application of this multimodal approach. To fill this knowledge gap, we critically evaluated various methodological aspects of previous combined EEG–fNIRS studies performed in healthy individuals. A literature search was conducted using PubMed and PsycINFO on June 28, 2021. Studies involving concurrent EEG and fNIRS measurements in awake and healthy individuals were selected. After screening and eligibility assessment, 96 studies were included in the methodological evaluation. Specifically, we critically reviewed various aspects of participant sampling, experimental design, signal acquisition, data preprocessing, outcome selection, data analysis, and results presentation reported in these studies. Altogether, we identified several notable strengths and limitations of the existing EEG–fNIRS literature. In light of these limitations and the features of combined EEG–fNIRS, recommendations are made to improve and standardize research practices to facilitate the use of combined EEG–fNIRS when studying healthy neurovascular coupling processes and alterations in neurovascular coupling among various populations.
Article
Full-text available
The COVID-19 pandemic has prompted the production of a vast amount of COVID-19-themed brand commercials, in an attempt to exploit the salience of the topic to reach more effectively the consumers. However, the literature has produced conflicting findings of the effectiveness of negative emotional contents in advertisings. The present study aims at exploring the effect of COVID-19-related contents on the hemodynamic brain correlates of the consumer approach or avoidance motivation. Twenty Italian participants were randomly assigned to two different groups that watched COVID-19-related or non-COVID-19-related commercials. The hemodynamic response [oxygenated (O2Hb) and deoxygenated hemoglobin modulations] within the left and right prefrontal cortices (PFC) was monitored with Functional Near-Infrared Spectroscopy (fNIRS) while brand commercials were presented, as the prefrontal lateralization was shown to be indicative of the attitude toward the brand and of the approach-avoidance motivation. First, the findings showed that the COVID-19-related contents were able to prompt emotional processing within the PFC to a higher extent compared to contents non-related to COVID-19. Moreover, the single-channel analysis revealed increased O2Hb activity of the left dorsolateral PFC compared to the left pars triangularis Broca’s area in the group of participants that watched the COVID-19-related commercials, suggesting that the commercials may have driven participants to dedicate more attention toward the processing of the emotional components compared to the semantic meaning conveyed by the ad. To conclude, despite expressing unpleasant emotions, commercials referring to the highly emotional pandemic experience may benefit the advertising efficacy, increasing the capability to reach customers.
Thesis
Full-text available
Malgré l’émergence rapide des systèmes automatisés et le développement de l’informatique affective, très peu d’études considèrent la charge mentale de travail dans la conception de scénarios de formation en RV. Cette thèse a pour objectif de contribuer au développement des systèmes adaptatifs en RV, basés sur la charge mentale de travail des utilisateurs. Nous proposons 3 axes de recherches : induction, reconnaissance, et exploitation de la charge mentale de travail en RV, ainsi qu'une définition de la « Réalité Virtuelle Affective et Cognitive ». Dans un premier temps, nous étudierons l’impact du port de casque de RV sur l’effort mental des utilisateurs. De plus, l’influence potentielle de la marche et de l’effet d’accommodation en RV seront analysés. Puis, nous proposerons une approche méthodologique pour introduire l’évaluation de la charge mentale de travail dans la conception de scénarios de formation en RV. Cette méthodologie permettra notamment de moduler le niveau de charge mentale de travail des utilisateurs au cours du temps. Des études utilisateurs seront menées dans un simulateur de vol en RV afin d'évaluer cette approche. Finalement, nous proposerons une solution tout-en-un afin d'estimer la charge mentale de travail des utilisateurs en temps-réel en utilisant des capteurs intégrés aux casques de RV. Cette configuration sera comparée aux systèmes plus répandus dans le commerce vis-à-vis des performances de prédiction. Les influence du types de mesures, des capteurs, et des méthodes de normalisation des signaux seront également analysées.
Article
Full-text available
In each culture, non-verbal communication is characterized by the use of conventional gestures. The present study aimed to investigate the neural correlates underlying a non-verbal interaction between encoder and decoder characterized by the use of different gestures in Italian and French cultures. Specifically, interagents’ cortical activity was recorded with the use of electroencephalography (EEG) in hyperscanning. From results, different cortical modulation and inter-brain connectivity emerged regarding the types and valence of gestures, the interagents’ role and the culture of belonging. The present research has shown cultural differences concerning gestures’ type and valence, highlighting brain synchronization in same culture’s individuals.
Article
Gestural communication allows providing information about thoughts and feelings, characterizing face-to-face interactions, also during non-verbal exchanges. In the present study, the autonomic responses and peripheral synchronization mechanisms of two individuals (encoder and decoder) were recorded simultaneously, through the use of biofeedback in hyperscanning, during two different experimental phases consisting in the observation (watching videos of gestures) and reproduction of positive and negative different types of gestures (affective, social and informative) supported by linguistic contexts. Therefore, the main aim of this study was focused on the analysis of simultaneous individuals' peripheral mechanisms during the performing of complex joint action, consisting of the observation (watching videos) and the reproduction of positive and negative social, affective, and informative gestures each supported by a linguistic script. Single-subject and inter-subject correlation analyses were conducted to observe individuals' autonomic responses and physiological synchronization. Single-subject results revealed an increase in emotional arousal, indicated by an increase in electrodermal activity (skin conductance level - SCL and response - SCR), during both the observation (watching videos) and reproduction of negative social and affective gestures contextualized by a linguistic context. Moreover, an increase of emotional engagement, expressed by an increase in heart rate (HR) activity, emerged in the encoder compare to the decoder during gestures reproduction (simulation of gestures). Inter-subject correlation results showed the presence of mirroring mechanisms, indicated by an increase in SCL, SCR, and HR synchronization, during the linguistic contexts and gesture observation (watching videos). Furthermore, an increase in SCL and SCR synchronization emerged during the observation (watching videos) and reproduction of negative social and affective gestures. Therefore, the present study allowed to obtain information on the mirroring mechanisms and physiological synchronization underlying the linguistic and gesture system during non-verbal interaction.
Chapter
In recent years, sustainability has received significant interest from companies and structures that have adopted new marketing and business models to improve the living conditions of the environment and consumers. Given the growing interest and attention, the issue of sustainability has been investigated by different disciplines, such as neuroscience, which have proved useful for investigating the emotional responses and cognitive processes of individuals, allowing us to understand the relationship between consumers and sustainability better. In this regard, this paper offers an overview on the topic of sustainability in a specific sector, such as tourism marketing, on the use of neuroscience to investigate sustainability and on the application of a multimethodological paradigm, involving the use of electroencephalography (EEG) and biofeedback, to measure individuals’ electroencephalographic and autonomic activity during the exploration of a green hotel.
Article
As a relatively new field of neurology and computer science, brain computer interface (BCI) has many established and burgeoning applications across scientific disciplines. Many neural monitoring technologies have been developed for BCI studies. Combining multiple monitoring technologies provides a new approach that synthesizes the advantages and overcomes the limitations of each technology. This article presents a systematic review on the applications, limitations, and future directions for the hybridization of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) into one synchronous multimodality. This review investigated research questions on design and usability of hybrid EEG-fNIRS studies. In this article, 765 papers were included in the initial search and 128 papers were selected through the PRISMA protocol. The review results show the possibility of improving the performance of hybrid EEG-fNIRS by optimizing the feature extraction algorithms and physical designing as well as expending more possible applications in information processing related fields.
Article
Full-text available
Prefrontal cortex provides both inhibitory and excitatory input to distributed neural circuits required to support performance in diverse tasks. Neurological patients with prefrontal damage are impaired in their ability to inhibit task-irrelevant information during behavioral tasks requiring performance over a delay. The observed enhancements of primary auditory and somatosensory cortical responses to task-irrelevant distractors suggest that prefrontal damage disrupts inhibitory modulation of inputs to primary sensory cortex, perhaps through abnormalities in a prefrontal-thalamic sensory gating system. Failure to suppress irrelevant sensory information results in increased neural noise, contributing to the deficits in decision making routinely observed in these patients. In addition to a critical role in inhibitory control of sensory flow to primary cortical regions, and tertiary prefrontal cortex also exerts excitatory input to activity in multiple sub-regions of secondary association cortex. Unilateral prefrontal damage results in multi-modal decreases in neural activity in posterior association cortex in the hemisphere ipsilateral to damage. This excitatory modulation is necessary to sustain neural activity during working memory. Thus, prefrontal cortex is able to sculpt behavior through parallel inhibitory and excitatory regulation of neural activity in distributed neural networks.
Article
Full-text available
Despite having relatively poor spatial and temporal resolution, near-infrared spectroscopy (NIRS) has several methodological advantages compared with other non-invasive measurements of neural activation. For instance, the unique characteristics of NIRS give it potential as a tool for investigating the role of the prefrontal cortex (PFC) in emotion processing. However, there are several obstacles in the application of NIRS to emotion research. In this mini-review, we discuss the findings of studies that used NIRS to assess the effects of PFC activation on emotion. Specifically, we address the methodological challenges of NIRS measurement with respect to the field of emotion research, and consider potential strategies for mitigating these problems. In addition, we show that two fields of research, investigating (i) biological predisposition influencing PFC responses to emotional stimuli and (ii) neural mechanisms underlying the bi-directional interaction between emotion and action, have much to gain from the use of NIRS. With the present article, we aim to lay the foundation for the application of NIRS to the above-mentioned fields of emotion research.
Article
Full-text available
Efficient emotion regulation is essential for social interaction and functioning in human society and often happens without direct intention and conscious awareness. Cognitive labeling of stimuli based on certain characteristics has been assumed to represent an effective strategy of implicit emotional regulation whereas processing based on simple perceptual characteristics (e.g., matching) has not. Evidence exists that the ventrolateral prefrontal cortex (VLPFC) might be of functional relevance during labeling by down-regulating limbic activity in the presence of threatening stimuli. However, it remained unclear whether this VLPFC activation was particularly specific to threat because previous studies focused exclusively on threatening stimuli. In the current study, 35 healthy participants labeled or matched both threatening and neutral pictures while undergoing 52-channel functional near-infrared spectroscopy. Results showed increased VLPFC activation during labeling of threatening but not neutral pictures. No increase in prefrontal activation was detected during matching. Moreover, skin conductance increased equally for both valence conditions during initial phases of labeling whereas during matching stronger increases were found for threatening stimuli. Although a general inverse relationship between VLPFC function and skin conductance was not confirmed, both were negatively correlated during matching of threatening pictures in subjects with high state anxiety. It was concluded that the VLPFC plays an essential role during implicit emotion regulation. Further, even simple perceptual processing seems to engage regulatory top-down activation in anxious individuals.
Article
Full-text available
Factor-analytic evidence has led most psychologists to describe affect as a set of dimensions, such as displeasure, distress, depression, excitement, and so on, with each dimension varying independently of the others. However, there is other evidence that rather than being independent, these affective dimensions are interrelated in a highly systematic fashion. The evidence suggests that these interrelationships can be represented by a spatial model in which affective concepts fall in a circle in the following order: pleasure (0), excitement (45), arousal (90), distress (135), displeasure (180), depression (225), sleepiness (270), and relaxation (315). This model was offered both as a way psychologists can represent the structure of affective experience, as assessed through self-report, and as a representation of the cognitive structure that laymen utilize in conceptualizing affect. Supportive evidence was obtained by scaling 28 emotion-denoting adjectives in 4 different ways: R. T. Ross's (1938) technique for a circular ordering of variables, a multidimensional scaling procedure based on perceived similarity among the terms, a unidimensional scaling on hypothesized pleasure–displeasure and degree-of-arousal dimensions, and a principal-components analysis of 343 Ss' self-reports of their current affective states. (70 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
In this study we analyze whether facial expression recognition is marked by specific event-related potential (ERP) correlates and whether conscious and unconscious elaboration of emotional facial stimuli are qualitatively different processes. ERPs elicited by supraliminal and subliminal (10ms) stimuli were recorded when subjects were viewing emotional facial expressions of four emotions or neutral stimuli. Two ERP effects (N2 and P3) were analyzed in terms of their peak amplitude and latency variations. An emotional specificity was observed for the negative deflection N2, whereas P3 was not affected by the content of the stimulus (emotional or neutral). Unaware information processing proved to be quite similar to aware processing in terms of peak morphology but not of latency. A major result of this research was that unconscious stimulation produced a more delayed peak variation than conscious stimulation did. Also, a more posterior distribution of the ERP was found for N2 as a function of emotional content of the stimulus. On the contrary, cortical lateralization (right/left) was not correlated to conscious/unconscious stimulation. The functional significance of our results is underlined in terms of subliminal effect and emotion recognition. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Facial stimulus processing is an important topic to explain how people comprehend affective disposition in others. The effect of attentive and pre-attentive elaboration of emotional facial expression was explored in the present research by using backward masking procedure. Specifically, unconscious mental process of emotion comprehension was analyzed: pictures presenting a happy, sad, angry, fearful, disgusted, surprised expressions were submitted to 21 subjects in both attentive and pre-attentive conditions and event-related potentials (ERPs) were registered in the two conditions. The two processes, attentive and pre-attentive, seem to be similar in their nature, since they are marked by analogous ERP deflections. In fact, two ERP effects were found, a positive (P300) deflection, maximally distributed on the parietal regions, and a negative (N200) deflection, more localized on the frontal sites. Nevertheless, some differences between the two conditions were found in terms of quantitative modulations of the two peaks. The N200 effect, ampler in attentive condition, may be considered such as an index of conscious processing of emotional faces, whereas the P3 (P3a) effect, higher in pre-attentive condition, was considered a specific marker of the automatic, unconscious process during the emotional face comprehension.
Article
Full-text available
This paper applies the behavior systems approach to fear and defensive behavior, examining the neural circuitry controlling fear and defensive behavior from this vantage point. The defensive behavior system is viewed as having three modes that are activated by different levels of fear. Low levels of fear promote pre-encounter defenses, such as meal-pattern reorganization. Moderate levels of fear activate post-encounter defenses. For the rat, freezing is the dominant post-encounter defensive response. Since this mode of defense is activated by learned fear, forebrain structures such as the amygdala play a critical role in its organization. Projections from the amygdala to the ventral periaqueductal gray activate freezing. Extremely high levels of fear, such as those provoked by physical contact, elicit the vigorous active defenses that compose the circa-strike mode. Midbrain structures such as the dorsolateral periaqueductal gray and the superior colliculus play a crucial role in organizing this mode of defense. Inhibitory interactions between the structures mediating circa-strike and post-encounter defense allow for the rapid switching between defensive modes as the threatening situation varies.
Article
Full-text available
Facial expressions contain both motor and emotional components. The inferior frontal gyrus (IFG) and posterior parietal cortex have been considered to compose a mirror neuron system (MNS) for the motor components of facial expressions, while the amygdala and insula may represent an "additional" MNS for emotional states. Together, these systems may contribute to our understanding of facial expressions. Here we further examine this possibility. In three separate event-related fMRI experiment, subjects had to (1) observe (2) discriminate and (3) imitate facial expressions. Stimuli were dynamic neutral, happy, fearful and disgusted facial expressions, and in Experiments 1 and 2, an additional pattern motion condition. Importantly, during each experiment, subjects were unaware of the nature of the next experiments. Results demonstrate that even passive viewing of facial expressions activates a wide network of brain regions that were also involved in the execution of similar expressions, including the IFG/insula and the posterior parietal cortex. Only a subset of these regions responded more during the observation of facial than pattern motion (bilateral ventral IFG, bilateral STS/MTG, bilateral amygdala, SMA). While the viewing of facial expressions recruited similar brain regions in all three experiments, adding an active task (discrimination, imitation) augmented the magnitude of these activations. Brain activations reflected differences in observed facial expressions, with emotional expressions activating relatively more the insula/frontal operculum, and neutral ones (blowing up the cheeks) the somatosensory cortices (SII). Using movies, fear activated the amygdala and disgust the insula, but other emotions activated these structures to a similar degree.
Article
Full-text available
Empathizing is a quantitative trait involving understanding another's mental state (including their emotion) and responding to this with an appropriate emotion. A reliable, behaviorally validated self-report questionnaire measure of this is the Empathy Quotient (EQ), which is continuously distributed across the general population. The "discrete emotions" model posits that each "basic" emotion has a relatively independent evolutionary antecedent and social-communicative function and is subserved by a discrete neural system. In this study, we investigate if and how empathy influences the perception of basic emotions. Twenty-five volunteers (13 female, 12 male) selected across EQ space participated in a correlational design 3T fMRI study. The stimuli were presented in a box-car design, where 5 blocks (each containing 4 video clips of any one of happy, sad, angry, disgust or neutral expressions from different actors) and a low-level baseline were presented in pseudo-random order. Using an exploratory analysis, we found different brain regions correlated with EQ, depending on which emotion was being perceived. In particular, the ventral striatal response to happy faces correlated positively with EQ, while the ventral striatal response to sad faces was negatively correlated with EQ. The precuneus and lateral prefrontal cortical response to angry faces correlated positively with EQ. The response of the insula and the superior temporal gyrus cortex to disgust faces were negatively correlated with EQ. These results are discussed in the light of the postulated evolutionary function of each emotion. Using a hypothesis-driven conjunction analysis, we found that a region in the left dorsal inferior frontal gyrus/premotor cortex was positively correlated to the EQ across all four emotions. This region could therefore constitute a biomarker for trait empathy across emotions. We conclude that there are common regions underlying empathy across different emotions, and there are regions that show an emotion-specific correlation with empathy. This pattern of results is interpreted using a modification of Haxby et al.'s model of face perception.
Article
At the heart of emotion, mood, and any other emotionally charged event are states experienced as simply feeling good or bad, energized or enervated. These states - called core affect - influence reflexes, perception, cognition, and behavior and are influenced by many causes internal and external, but people have no direct access to these causal connections. Core affect can therefore be experienced as free-floating (mood) or can be attributed to some cause (and thereby begin an emotional episode). These basic processes spawn a broad framework that includes perception of the core-affect-altering properties of stimuli, motives, empathy, emotional meta-experience, and affect versus emotion regulation; it accounts for prototypical emotional episodes, such as fear and anger, as core affect attributed to something plus various nonemotional processes.
Article
Introduction The study of the neurobiological bases of social cognition is currently one of the most prominent areas of research in behavioural neurology and in neuropsychology. Similar to many other areas of contemporary neuropsychological research, it was first characterized by single case reports of patients suffering from brain damage. One of the first descriptions of impaired social cognition following brain damage was provided by Harlow (1868). In his famous case report, Harlow portrays the case of Phineas Gage, a railroad employee, who suffered severe frontal lobe injury due to an iron bar that penetrated his frontal lobes. Although he survived, recovered physically and had many preserved cognitive abilities, his social behaviour was so impaired that his acquaintances said he was ‘no longer Gage’. Harlow does not refer directly to Gage's empathic ability yet he describes him as ‘… fitful, irreverent, indulging at times in the grossest profanity, manifesting but little deference for his fellow …’ (Harlow, 1868). In the following years, similar clinical reports have offered accumulating evidence regarding the role of the frontal lobes in emotions and behaviour regulation, and the past 10 years have seen tremendous resurgence of interest in this area. Studies have consistently suggested that acquired damage to the prefrontal cortex may result in severe impairment in interpersonal behaviour (Damasio et al., 1991; Mesulam, 1985; Stuss and Benson, 1986; Stuss et al., 2001). © Cambridge University Press 2007 and Cambridge University Press, 2009.
Article
The simultaneous application of different neuroimaging methods combining high temporal and spatial resolution can uniquely contribute to current issues and open questions in the field of pragmatic language perception. In the present study, comprehension of novel metaphors was investigated using near-infrared spectroscopy (NIRS) combined with the simultaneous acquisition of event-related potentials (ERPs). For the first time, we investigated the effects of figurative language on early electrophysiological markers (P200, N400) and their functional relationship to cortical haemodynamic responses within the language network (Broca’s area, Wernicke’s area). To this end, 20 healthy subjects judged 120 sentences with respect to their meaningfulness, whereby phrases were either literal, metaphoric, or meaningless. Our results indicated a metaphor specific P200 reduction and a linear increase of N400 amplitudes from literal over metaphoric to meaningless sentences. Moreover, there were metaphor related effects on haemodynamic responses accessed with NIRS, especially within the left lateral frontal cortex (Broca’s area). Significant correlations between electrophysiological and haemodynamic responses indicated that P200 reductions during metaphor comprehension were associated with an increased recruitment of neural activity within left Wernicke’s area, indicating a link between variations in neural activity and haemodynamic changes within Wernicke’s area. This link may reflect processes related to interindividual differences regarding the ability to classify novel metaphors. The present study underlines the usefulness of simultaneous NIRS-EEG measurements in language paradigms – especially for investigating the functional significance of neurophysiological markers that have so far been rarely examined – as these measurements are easily and efficiently realizable and allow for a complementary examination of neural activity and associated metabolic changes in cortical areas.
Article
Anxiety behavior showed a consistent attentional bias toward negative and aversive memories, induced by a right frontal cortical superiority, based on an unbalance effect between the two hemispheres. The aim of the present study was to explore the role of the left dorsolateral prefrontal cortex (DLPFC) in the memory retrieval process of positive versus negative emotional stimulus, as a function of anxiety level. A repeated transcranial magnetic stimulation (rTMS) paradigm was used to induce cortical activation of the left DLPFC. Subjects (n = 27; age range, 21–36 years), who were divided into two different groups (high/low anxiety; State–Trait Anxiety Inventory), were required to perform a task consisting of two experimental phases: an encoding phase (lists composed of positive and negative emotional words); and a retrieval phase (old stimuli and new stimuli to be recognized). Moreover, new stimuli (distractors) semantically related or unrelated to the old stimuli were used to test a possible interference effect induced by the semantic association. rTMS over the left DLPFC affects memory retrieval. High-anxiety subjects benefited in greater measure from frontal left stimulation with a reduced negative bias (increased accuracy and reduced response time for the positive stimuli) and a significant increased performance for the semantically related distractors (reduced interference effect). Left DLPFC activation favors the memory retrieval of positive emotional information and might limit the unbalance effect induced by right hemispheric superiority in high levels of anxiety.
Article
This experiment assessed the components of Baddeley's working memory system impaired by anxiety during performance of the Corsi Blocks Test. The Corsi task was performed concurrently with different secondary tasks (i.e., articulatory suppression; counting backwards; spatial tapping; simple tapping). Results showed Corsi performance depended mainly on the central executive and visuospatial sketchpad components of working memory. Adverse effects of trait anxiety on the Corsi task were observed on the central executive but not on the phonological loop or the visuospatial sketchpad. These effects were not mediated by state anxiety. The findings indicate for the first time that trait anxiety impairs central executive functioning on a nonverbal task, and that anxiety does not impair functioning of the “slave” systems (i.e., phonological loop; visuospatial sketchpad). Theoretical implications of these findings are discussed.
Article
presented an overview of recent research on anterior [cortical function] asymmetries associated with emotion and individual differences in emotional reactivity, psychopathologic states, dispositional mood, and temperament sketch the major elements of the theoretical model that motivates the research to be presented / the methods that are common to our studies and the unique methodologic requirements of this research are then described / research on anterior asymmetries associated with the phasic arousal of emotion are presented, followed by a summary of our findings on the relations between individual differences in baseline asymmetry and affective reactivity (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
To develop a noninvasive method for psychophysiological assessment of posttraumatic stress disorder (PTSD), 34 victims of the Tokyo Subway Sarin Attack in 1995 including 8 diagnosed as PTSD and 12 controls were examined by a multichannel near-infrared spectroscopy (NIRS) system. Hemodynamic response in the prefrontal cortex was monitored during the presentation of trauma-related and control stimuli by video images. Skin conductance response (SCR) was also examined. Oxygenated hemoglobin significantly increased during the trauma-related image in the victims with or without PTSD. Deoxygenated hemoglobin significantly decreased only in victims with PTSD. No significant alteration was found in controls. Significantly enhanced SCR was also observed in the victims with PTSD during trauma-related stimuli. The findings suggest that measurement of cerebral hemodynamic response by NIRS is useful for psychophysiological assessment of PTSD.
Article
The aims were (1) to determine the sensitivity and reproducibility to detect the hemodynamic responses and optical neuronal signals to brain stimulation by near-infrared spectroscopy (NIRS) and evoked potentials by electroencephalography (EEG) and (2) to test the effect of novel filters on the signal-to-noise ratio. This was achieved by simultaneous NIRS and EEG measurements in 15 healthy adults during visual stimulation. Each subject was measured three times on three different days. The sensitivity of NIRS to detect hemodynamic responses was 55.2 % with novel filtering and 40 % without. The reproducibility in single subjects was low. For the EEG, the sensitivity was 86.4 % and the reproducibility 57.1 %. An optical neuronal signal was not detected, although novel filtering considerably reduced noise.
Article
Known to be involved in emotional processing the human prefrontal cortex (PFC), can be non-invasively monitored using near-infrared spectroscopy (NIRS). As such, PFC NIRS can serve as a means for studying emotional processing by the PFC. Identifying patterns associated with emotions in PFC using NIRS may provide a means of bedside emotion identification for nonverbal children and youth with severe physical disabilities. In this study, NIRS was used to characterize the PFC hemodynamic response to emotional arousal and valence in a music-based emotion induction paradigm in 9 individuals without disabilities or known health conditions. In particular, a novel technique based on wavelet-based peak detection was used to characterize chromophore concentration patterns. The maximum wavelet coefficients extracted from oxygenated hemoglobin concentration waveforms from all nine recording locations on the PFC were significantly associated with emotional valence and arousal. Specifically, high arousal and negative emotions were associated with larger maximum wavelet coefficients.
Article
The hypotheses of this investigation were based on conceiving of facial mimicry reactions in face-to-face interactions as an early automatic component in the process of emotional empathy. Differences between individuals high and low in emotional empathy were investigated. The parameters compared were facial mimicry reactions, as represented by electromyographic (EMG) activity, when individuals were exposed to pictures of angry or happy faces. The present study distinguished between spontaneous facial reactions and facial expressions associated with more controlled or modulated emotions at different information processing levels, first at a preattentive level and then consecutively at more consciously controlled levels: 61 participants were exposed to pictures at three different exposure times (17, 56, and 2350 ms). A significant difference in facial mimicry reactions between high- and low-empathy participants emerged at short exposure times (56 ms), representing automatic, spontaneous reactions, with high-empathy participants showing a significant mimicking reaction. The low-empathy participants did not display mimicking at any exposure time. On the contrary, the low-empathy participants showed, in response to angry faces, a tendency to an elevated activation in the cheek region, which often is associated with smiling.
Chapter
Near-infrared spectroscopy (NIRS), which was originally designed for clinical monitoring of tissue oxygenation, has recently been receiving increasing attention as a useful tool for neuroimaging studies. This technique is completely noninvasive, does not require strict motion restriction, and can be used in a daily-life environment. It is expected that NIRS will provide a new direction for cognitive neuroscience research, more so than other neuroimaging techniques, although several problems with NIRS remain to be explored. This chapter describes the basic theory of NIRS, its potential and limitations, and the future prospects of this technique.
Article
According to a recent hypothesis, the prefrontal cortex has been proposed as the site of emotional memory integration, because it is sensitive to the recognition of emotional contents. In the present research, we explored the role of the dorsolateral prefrontal cortex (DLPFC) in memory recognition processes for positive versus negative emotional stimuli when old (target) and new (distractor, either semantically related or unrelated to the target) stimuli were presented. The role of the DLPFC was analysed using an rTMS (repeated transcranial magnetic stimulation) paradigm that induced increased cortical activation of the left DLPFC. The subjects were required to perform a task that consisted of two experimental phases (i.e., an encoding and a recognition phase) in which the targets and the distractors were presented and recognition performance was measured. rTMS stimulation was provided over the left DLPFC during the recognition phase. We found that the rTMS stimulation affected the memory recognition of positive emotional material. Moreover, related and unrelated distractors were discarded better when they were positively valenced, and a more significant effect (i.e., increased performance) was produced in response to related distractors. This result suggests that the activation of the left DLPFC favours the memory recognition of positive emotional information, and that such activation is able to induce a more appropriate selective process to distinguish target from distractor stimuli in the presence of more complex processes (related distractors). The valence model of emotional cue processing may explain this increased performance by demonstrating the distinct role of the left hemisphere in the retrieval of positive emotional information.
Article
We examine the effect of individual psychological differences on network structures, proposing several hypotheses about how individual differences might predispose actors to structure their social environment by seeking network closure or by sustaining structural holes. We introduce a new triad census method to examine personal networks of strong and weak ties. For 125 egocentric networks we correlated the triad census results with several extensively researched psychological instruments. The triad census reduced to three principal components, describing central aspects of strength-of-weak-ties and structural holes theories. Psychological predispositions explained a significant proportion of the variance in each of these components. Our results suggest that people who see themselves vulnerable to external forces tend to inhabit closed networks of weak connections. On the other hand, people who seek to keep their strong tie partners apart, and thereby bridge structural holes, tend to be individualists, to believe that they control the events in their lives, and to have higher levels of neuroticism. Finally, people with strong network closure and “weak” structural holes (as with the “strength of weak ties”) tend to categorize themselves and others in terms of group memberships. They also tend to be more extraverted and less individualistic.
Article
Alpha brain oscillation modulation was analyzed in response to masked emotional facial expressions. In addition, behavioural activation (BAS) and behavioural inhibition systems (BIS) were considered as an explicative factor to verify the effect of motivational significance on cortical activity. Nineteen subjects were submitted to an ample range of facial expressions of emotions (anger, fear, surprise, disgust, happiness, sadness, and neutral). The results demonstrated that anterior frontal sites were more active than central and posterior sites in response to facial stimuli. Moreover, right-side responses varied as a function of emotional types, with an increased right-frontal activity for negative emotions. Finally, whereas higher BIS subjects generated a more right hemisphere activation for some negative emotions (such as fear, anger, and surprise), Reward-BAS subjects were more responsive to positive emotion (happiness) within the left hemisphere. Valence and potential threatening power of facial expressions were considered to elucidate these cortical differences.