Article

The mental and subjective skin: Emotion, empathy, feelings and thermography

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Cruz-Albarran et al. (2017) presented a noninvasive biological thermal imaging technique that offers excellent insights for emotional diagnosis. In addition, Salazar-López et al. (2015) extensively studied the cognitive neuropsychology of emotions by employing thermography as a somatic marker in emotional tasks. Their research uncovered statistically significant associations between fluctuations in facial temperature and mental states, hence strengthening the potential of thermal imaging as a tool for com prehending emotional experiences. ...
... Although youthful faces showed a similar pattern, the results did not approach statistical significance. The findings align with previous research, emphasizing that emotions cause temperature fluctuations in particular areas of the face (Cruz-Albarran et al., 2017;Salazar-López et al., 2015). The influence of stimuli depicting angry expressions of elderly individuals was clearly observed in the maxillary region, suggesting minor changes in biomarkers linked to the aging process. ...
... Angry female faces exhibited greater temperatures compared to neutral female faces. The findings highlight the varied impacts of various emotional expressions on facial temperature, which are consistent with previous research (Jaramillo-Quintanar et al., 2022;Cruz-Albarran et al., 2017;Salazar-López et al., 2015;Neta et al., 2009). ...
Article
Full-text available
The aim of the study This study examines how exposure to angry versus neutral facial expressions affects facial temperature, aiming to validate thermograms as reliable measures of emotional responses. We focus on Saudi youth and explore the impact of different stimuli categories (young vs. old, male vs. female) on subjective feelings recorded by thermal cameras. Material and methods The research involved a sample of 40 male participants selected from King Fahd University of Petroleum and Minerals (KFUPM) in Saudi Arabia, with an average age of 21.8 years (SD = 1.3). All participants had normal or corrected-to-normal visual acuity. The study comprised two experiments to examine the impact of exposure to angry versus neutral faces on physiological responses. Thermal data were collected using infrared thermography, focusing on variations in facial temperature as a key indicator of physiological reactions. Participants were exposed to stimuli consisting of angry and neutral faces from various demographic categories, including males, females, young individuals, and elderly adults. This comprehensive methodology combined behavioral analysis and infrared thermography technique Results The behavioral findings indicated that individuals exhibited increased response times (RTs) when classifying angry stimuli compared to neutral face stimuli. Infrared thermography revealed a notable impact of anger, particularly in Region of Interest 5 (the area around the upper jaw). This effect was especially pronounced in individuals exposed to angry facial expressions, but not in those analyzing angry expressions of younger individuals. Conclusions These findings provide insight into the unique physiological responses triggered by angry facial expressions, specifically identifying temperature rises in specific facial areas. The results contribute to our understanding of the physiological mechanisms involved in the processing of facial expression recognition.
... These studies have generally used a highly controlled experimental setting. For instance, the participants in [3] were exposed to certain images on a computer screen meant to elicit specific emotions; their temperature in different regions of interest (ROIs) were then compared to that at a baseline level. The researchers could determine that the participants' nose became warmer after they saw a positive-valence and high-arousal picture, and, in another context, that it became colder during experiences of empathy (of both joy and pain). ...
... For example, [5] have argued that thermal increases in the nose are linked to sexual arousal, which could be interpreted in this case more generally as a type of platonic "social" arousal. In fact, following Salazar and colleagues' [3] interpretation, the temperature increase could be related more generally to higher emotional arousal. Interestingly, though, the frequency of thermal decreases in the nose also tended to be higher when the participant reported higher likelihood of future friendship, despite this effect not reaching statistical significance (see Figure 3). ...
... Interestingly, though, the frequency of thermal decreases in the nose also tended to be higher when the participant reported higher likelihood of future friendship, despite this effect not reaching statistical significance (see Figure 3). This effect would be in line with [4], who reported lower nose temperatures during experiences of joy, or with [3], who associated this pattern with empathy feelings. These nuanced findings highlight the complex relationship between emotion and skin temperature. ...
Conference Paper
Full-text available
This study investigates f0 adaptation, skin temperature change, and the relationship between the two. While a growing number of studies have demonstrated that emotional reactions in humans lead to changes in their facial skin temperature, none of them have studied temperature change in conversational contexts. Here, we have tested whether a conversation's degree of intimacy influences emotion such that it affects facial temperature and f0 adaptation-in terms of entrainment to interlocutor and f0 change due to the conversation topic. We also ask whether temperature change and f0 adaptation are related. In our data set of 38 participants in a between-subjects design, few speakers aligned on f0 to their partner, with no identifiable patterns. Regardless of their interlocutor, however, the speakers' f0 median and standard deviation tended to decrease when they spoke about more personal topics. This adds to previous literature describing emotional speech prosody. The participants' nose temperature was modulated by social emotion, but there was no relationship between temperature change and f0 adaptation. This suggests that, although the participants' nose temperature was sensitive to the social dynamic, the emotional reactions driving thermal change do not seem to be the same leading to prosodic adaptation.
... Разгледани са методите за диагностициране чрез анализ на данните, предоставени от термична камера [9][10][11][12][13][14][15][16][17][18][19][20][21]. Основният анализ, успореден на този, извършен чрез инфрачервено измерване, е ЕЕГ [22][23][24]. ...
... В [19] използват ИРТ технология, като неинвазивен и непрекъснат метод за оценка на стреса при животни. В [23] се използва термография за изследване на когнитивната невропсихология на емоциите, използвайки я като соматичен маркер на субективно преживяване по време на емоционални задачи. Получените резултати показват значителна корелация между промените в температурата на лицето и психическото настроение. ...
... В [23] се използва термография за изследване на когнитивната невропсихология на емоциите, използвайки я като соматичен маркер на субективно преживяване по време на емоционални задачи. Получените резултати показват значителна корелация между промените в температурата на лицето и психическото настроение. ...
... According to schemes used in previous stress and pain research, thermal signatures were analyzed in a baseline phase and a stress (hunger) phase. [13][14][15][16][17][18][19][20]23,26,[33][34][35][36][37][38] Furthermore, the endpoints "relaxed face" and "facial expression" were considered for each phase of the analysis. This was done to identify the facial action and associated muscle activity as a potential influencing factor of thermal variation. ...
... The ROIs were chosen taking into account previous IRT-based emotion and stress research. [13][14][15][16][17][18][19][20]23,26,[33][34][35][36][37][38] The ROI "T1" (nose tip) was defined by a circle in the area of the major alar cartilage between the nostrils, excluding the nostrils themselves. The orbicularis oculi muscle served as an anatomical landmark for the periorbital ROI "T2," with a special focus on the anastomosis area of the dorsal nasal artery and the angular artery. ...
... When it comes to measuring facial thermal response, the nose tip has repeatedly been identified as the most stress-sensitive region of the face. 15,20,[33][34][35]37 The results of our study confirm these observations. In our study, the nose tip was the ROI with the most important hunger-related rise in temperature, which was shown to be highly significant. ...
Article
Full-text available
Background: This study is a preliminary clinical investigation with the objective to evaluate the facial thermal response of premature and term neonates to a non-painful stressor (hunger) using infrared thermography (IRT). The development of objective and reliable parameters to monitor pain and stress is of relevance for optimal neonatal outcome and achieving a better management of patient comfort. Methods: We enrolled 12 neonates ranging from 27 to 39 weeks gestation (median: 34) and aged 3-79 days (median: 13). Recordings were performed before and after feeding, with and without hunger. Six regions of interest were chosen for evaluation (nose tip, periorbital and corrugator region, forehead, perioral and chin region). Results: There was an increase in the facial temperature in infants immediately prior to their next feed relative to infants who were not hungry, with the nasal tip being the facial evaluation site with the greatest temperature change. Conclusions: The IRT appears to be a feasible and suitable method to detect changes in the neonatal patient. The thermal variations observed seem to reflect an arousal mediated by the parasympathetic nervous system, which has been described in existing infant stress research. Impact: This is the first study to examine the use of infrared thermography (IRT) in monitoring the facial thermal response to a mild stressor (hunger) in premature and term neonates. Hunger as a mild, non-pain-associated stressor showed a significant effect on the facial temperature. The thermal signature of the regions of interest chosen showed hunger-related thermal variations. Results suggest the feasibility and suitability of IRT as an objective diagnostic tool to approach stress and changes in the condition of the neonatal patient.
... In the present research, we tested Infrared thermography as a physiological index of social anxiety (Nikolić, 2020). This technique has been used as a biological marker of several subjective experiences such as emotions, stress, feelings, deception detection, empathy and anxiety (Abdelrahman et al., 2017;Clay-Warner & Robinson, 2015;Engert et al., 2014;Gálvez-García et al., 2021;Ioannou et al., 2014a;Ito et al., 2018;Moliné et al., 2017;2018a, 2018bSalazar-López et al., 2015;Shastri et al., 2009). ...
... Other studies report that thermal rise in the forehead is related to mental workload and anxiety, and thermal rise in the cheeks to social desirability (Gálvez-García et al., 2021). Thermal changes measured by infrared thermography (IRT) have some advantages over other physiological indexes such as heart rate variability or skin conductance (Salazar-López et al., 2015). First, IRT is noninvasive and ecological; a person can be observed from a distance, in natural settings, without annoying the participant with tools such as electrodes or cables on the skin, and even being registered without conscious realization. ...
... To ensure consistent registration of the facial ROIs, we applied the same square size (3 × 3 pixels) for each ROI (forehead, cheeks, and nose) in all thermograms. These ROIs can be easily delimited in line with previous research (Ammer & Formenti, 2016;Gálvez-García et al., 2021;Salazar-López et al., 2015). For cheeks and forehead, both left and right sides were recorded. ...
Article
Previous research on physiological indices of social anxiety has offered unclear results. In this study, participants with low and high social anxiety performed five social interaction tasks while being recorded with a thermal camera. Each task was associated with a dimension assessed by the Social Anxiety Questionnaire for Adults (1 = Interactions with strangers. 2 = Speaking in public/Talking with people in authority, 3 = Criticism and embarrassment, 4 = Assertive expression of annoyance, disgust or displeasure, 5 = Interactions with the opposite sex). Mixedeffects models revealed that the temperature of the tip of the nose decreased significantly in participants with low (vs. high) social anxiety (p < 0.001), while no significant differences were found in other facial regions of interest: forehead (p = 0.999) and cheeks (p = 0.999). Furthermore, task 1 was the most effective at discriminating between the thermal change of the nose tip and social anxiety, with a trend for a higher nose temperature in participants with high social anxiety and a lower nose temperature for the low social anxiety group. We emphasize the importance of corroborating thermography with specific tasks as an ecological method, and tip of the nose thermal change as a psychophysiological index associated with social anxiety.
... To evaluate the dimensional approach and the combination of the two approaches (modular and dimensional), studies of emotions have been developed based on infrared thermography, which has gained considerable interest as a non-invasive technique. In this method, infrared radiation from the skin surface is captured, which, in turn, is associated with blood perfusion changes that are controlled by the autonomic nervous system [2], [14], [15] and which allows for thermal changes in the face that can be associated with an emotional response to be identified [15]- [19]. Moreover, it is a low-noise measurement [14]. ...
... In this method, infrared radiation from the skin surface is captured, which, in turn, is associated with blood perfusion changes that are controlled by the autonomic nervous system [2], [14], [15] and which allows for thermal changes in the face that can be associated with an emotional response to be identified [15]- [19]. Moreover, it is a low-noise measurement [14]. ...
... The room had no exposed heating pipes or electrical wiring because these elements could have interfered with the thermographic measurement, as they are sources of heat. The thermographic images were obtained using a FLIR A655SC thermographic camera with a wavelength in the far infrared band (7.5-14 µm), a spatial resolution of 640 × 480 pixels, an accuracy of ±2 °C, and a NETD (Noise Equivalent Temperature Difference, also known as thermal sensitivity) over 30 mK (0.03 °C); the software used for the acquisition was the FLIR R&D version 3.3 [14]. ...
Preprint
Full-text available
The objective of the work was to realize a preliminary study of proof-of-concept to evaluate emotions using thermographic images and blood perfusion algorithm; the images were obtained for baseline and positive and negative valence according to the protocol of the Geneva Affective Picture Database. The blood perfusion algorithm is based on the heat transport equation. The average temperature and blood perfusion in forehead, periorbital eyes, cheeks, nose and upper lips were determined. Absolute and percentage differences between the valences and the baseline were calculated. For negative valence, a decrease in temperature and blood perfusion was observed in the ROIs, and the effect was greater on the left side than on the right side. In positive valence, the temperature and blood perfusion increased in some cases, showing a complex pattern. The temperature and perfusion of the nose was reduced for both valences, which is indicative of the arousal dimension. The blood perfusion images were found to be greater contrast; the percentage differences in the blood perfusion images are greater than those obtained in thermographic images. Moreover, the blood perfusion images, and vasomotor answer are consistent, therefore, they can be a better biomarker than thermographic analysis in identifying emotions.
... To evaluate the dimensional approach and the combination of the two approaches (modular and dimensional), studies of emotions have been developed based on infrared thermography, which has gained considerable interest as a non-invasive technique. In this method, infrared radiation from the skin surface is captured, which, in turn, is associated with blood perfusion changes that are controlled by the autonomic nervous system [2], [14], [15] and which allows for thermal changes in the face that can be associated with an emotional response to be identified [15]- [19]. Moreover, it is a low-noise measurement [14]. ...
... In this method, infrared radiation from the skin surface is captured, which, in turn, is associated with blood perfusion changes that are controlled by the autonomic nervous system [2], [14], [15] and which allows for thermal changes in the face that can be associated with an emotional response to be identified [15]- [19]. Moreover, it is a low-noise measurement [14]. ...
... The room had no exposed heating pipes or electrical wiring because these elements could have interfered with the thermographic measurement, as they are sources of heat. The thermographic images were obtained using a FLIR A655SC thermographic camera with a wavelength in the far infrared band (7.5-14 µm), a spatial resolution of 640 × 480 pixels, an accuracy of ±2 °C, and a NETD (Noise Equivalent Temperature Difference, also known as thermal sensitivity) over 30 mK (0.03 °C); the software used for the acquisition was the FLIR R&D version 3.3 [14]. ...
Article
Full-text available
In this work, a preliminary study of proof-of-concept was conducted to evaluate the performance of the thermographic and blood perfusion data when emotions of positive and negative valence are applied, where the blood perfusion data are obtained from the thermographic data. The images were obtained for baseline, positive, and negative valence according to the protocol of the Geneva Affective Picture Database. Absolute and percentage differences of average values of the data between the valences and the baseline were calculated for different regions of interest (forehead, periorbital eyes, cheeks, nose and upper lips). For negative valence, a decrease in temperature and blood perfusion was observed in the regions of interest, and the effect was greater on the left side than on the right side. In positive valence, the temperature and blood perfusion increased in some cases, showing a complex pattern. The temperature and perfusion of the nose was reduced for both valences, which is indicative of the arousal dimension. The blood perfusion images were found to be greater contrast; the percentage differences in the blood perfusion images are greater than those obtained in thermographic images. Moreover, the blood perfusion images, and vasomotor answer are consistent, therefore, they can be a better biomarker than thermographic analysis in identifying emotions.
... IRT is able to measure the superficial temperature of a body at a distance. In the biomedical field, this technique is employed to measure the modulation of skin temperature, which is an indicator of a subject's psychophysiological condition [25]. In fact, stress, anxiety, or weariness, but also happiness, may cause changes in superficial microcirculation that induces fluctuations in skin temperature. ...
... Hence, IRT has already been used to investigate the workload [30], the learning process [31], and the subject's emotional state [25,32,33]. Furthermore, this technology has been used to assess autonomic correlates during the administration of clinical tests, for instance for Alzheimer's disease patients [34] or during the recovery from breast cancer treatment [35]. ...
... Regarding the IRT signals data analysis, visual inspection was used to assess captured thermal signals' quality first; no video was disallowed. Two regions of inte (ROI) were chosen on the nose tip and forehead (corrugator) of each subject (Figure These regions were selected; hence, they are highly indicative of the autonomic nerv system activity [25]. The position of these ROIs was tracked across all the frames of thermal video employing a tracking algorithm [41]. ...
Article
Full-text available
Cerebral palsy (CP) is a non-progressive neurologic pathology representing a leading cause of spasticity and concerning gait impairments in children. Robotic-assisted gait training (RAGT) is widely employed to treat this pathology to improve children’s gait pattern. Importantly, the effectiveness of the therapy is strictly related to the engagement of the patient in the rehabilitation process, which depends on his/her psychophysiological state. The aim of the study is to evaluate the psychophysiological condition of children with CP during RAGT through infrared thermography (IRT), which was acquired during three sessions in one month. A repeated measure ANOVA was performed (i.e., mean value, standard deviation, and sample entropy) extracted from the temperature time course collected over the nose and corrugator, which are known to be indicative of the psychophysiological state of the individual. Concerning the corrugator, significant differences were found for the sample entropy (F (1.477, 5.907) = 6.888; p = 0.033) and for the mean value (F (1.425, 5.7) = 5.88; p = 0.047). Regarding the nose tip, the sample entropy showed significant differences (F (1.134, 4.536) = 11.5; p = 0.041). The findings from this study suggests that this approach can be used to evaluate in a contactless manner the psychophysiological condition of the children with CP during RAGT, allowing to monitor their engagement to the therapy, increasing the benefits of the treatment.
... Chung et al. demonstrate the use of compact, wearable devices that interface gently and noninvasively with the skin of neonates for the measurement of the full suite of vital signs in the intensive care unit (ICU), including clinical grade thermometry on the skin of as a surrogate for measuring core body temperature. 81,82 Wearable temperature sensors can also be used to monitor mental and physical activity, 76 emotions, 117 sleep cycles, and menstrual cycles. 86,118,119 Furthermore, skin temperature has applications in monitoring of skin injury, such as wound healing, viability of skin flaps, and ulcer formation (e.g., diabetic ulcers, bed sores), and the onset of disease, such as breast cancer. ...
... 76 Another study highlights changes in facial temperature using infrared imaging (n ¼ 40) while presenting subjects with various positive, neutral, and negative visual stimuli [ Fig. 5(b)]. 117 A significant change in temperature (À0.85 C) appears on the tip of the nose compared to the neutral condition (p < 0.001) for images with negative, low emotional impact, while a significant increase in temperature (þ 0.96 C) occurs for images with negative, high emotional impact (p < 0.003). Images with strong, positive emotional arousal leads to a þ 1.66 C change in nasal temperature (p < 0.001). ...
... Similar procedures can assess other emotions, such as empathy and love, of interest in applications in mental health. 117 ...
Article
Full-text available
Measurements of the thermal properties of the skin can serve as the basis for a noninvasive, quantitative characterization of dermatological health and physiological status. Applications range from the detection of subtle spatiotemporal changes in skin temperature associated with thermoregulatory processes, to the evaluation of depth-dependent compositional properties and hydration levels, to the assessment of various features of microvascular/macrovascular blood flow. Examples of recent advances for performing such measurements include thin, skin-interfaced systems that enable continuous, real-time monitoring of the intrinsic thermal properties of the skin beyond its superficial layers, with a path to reliable, inexpensive instruments that offer potential for widespread use as diagnostic tools in clinical settings or in the home. This paper reviews the foundational aspects of the latest thermal sensing techniques with applicability to the skin, summarizes the various devices that exploit these concepts, and provides an overview of specific areas of application in the context of skin health. A concluding section presents an outlook on the challenges and prospects for research in this field.
... Infrared thermography enables remote temperature measurement applicable to multiple people regardless of lighting conditions [1]. The use of thermography has also been explored beyond temperature measurement for diagnosing illnesses [2,3] and estimating emotions [4][5][6][7] based on the fact that human skin temperature changes via blood circulation due to heartbeat, respiration, sweating, and muscular activity [8]. Its non-contact nature offers advantages over methods like electromyography or electroencephalography, which require physical contact. ...
... lect ROIs on the forehead and nasal tip vessels and nonreproducible, and the variation in the definition of ROIs across different studies could lead to inconsistency [5]. ...
Preprint
Infrared thermography, which has widely spread particularly during the COVID-19 period, has been effectively used for research on health monitoring and emotion estimation. Nevertheless, detecting minute temperature changes with thermography is challenging as it is disturbed by not only noise but also outside temperature surrounding the object. In this study, we demonstrate detecting face temperature variation by implementing lock-in thermography using heartbeat signals as a reference. It allows us to detect minute temperature changes, as low as \sim10 mK, on the forehead with a commercially available thermal camera. The proposed approach enables stable measurement of body temperature variation, showing potential for non-contact emotion estimation.
... Although there is no standard model of thermal analysis that enables human emotions to be recognized (Fu and Frasson 2016;Filippini et al. 2020), monitoring the change in temperature of regions of interest (ROI) on the face has allowed correlations with specific emotions and twodimensional models of emotion to be identified (Merla and Romani 2007;Robinson et al. 2012;Ioannou et al. 2014;Salazar-López et al. 2015;Diaz-Piedra et al. 2019). ...
... Infrared Thermography-a Flir E60 thermal imager (19,200 pixels and 320 × 240 resolution) and FLIR Tools software were used. The regions of interest (ROIs) chosen to analyze the participants' thermal changes were the tip of the nose, cheeks, and chin, as they are considered important facial reference points for capturing emotions (Nhan and Chau 2010; Robinson et al. 2012;Ioannou et al. 2014;Legrand et al. 2015;Salazar-López et al. 2015;Cruz-Albarran et al. 2017;Kosonogov et al. 2017;Goulart et al. 2019;Zhang et al. 2019). ...
Article
Full-text available
Identifying users’ experience when using products is one of the major challenges for design. Analyzing users’ psychophysiological reactions to an experience using biofeedback can produce more reliable results than using subjective evaluations, such as structured interviews and questionnaires. Two case studies were conducted to identify emotions users actually felt and to check whether there is some correspondence with what they reported after using two computational systems. The first system investigated users’ emotions during training on a vehicle driving simulator, and the second analyzed the emotions experienced during a car racing game, both in a virtual reality environment. User’s opinions about their emotional state were obtained using self-report techniques (using the Geneva Emotions Wheel—GEW and Positive and Negative Affective Schedule—PANAS questionnaires) and applying EEG (brain activity with Frontal Alpha Asymmetry Index—FAAI) and infrared thermography (facial thermograms). The training experiment presented the greater concordance between the psychophysiological and the self-report responses. Results evidenced the importance of undertaking multimodal studies in design research to determine users’ emotional experiences in a virtual reality context.
... Donors who experienced VVR symptoms during the donation tend to show higher levels, or greater velocity of, thermal fluctuations around the nose, chin, left cheek and forehead areas. This is in line with previous studies who found decreases in nose tip temperature related to emotional arousal 36,37 and stress 38 and perinasal differences related to stress and perspiration 39 or fear 36 , decrease in forehead temperature associated with fear 40 , stress 38 and emotional states with low power 41 and that of cheeks related to stress 38 . ...
... Furthermore, the results corroborate the idea that ITI may be a valid tool for the study of emotions such as arousal, fear and stress [36][37][38][39][40][41] , especially by looking at features that reflect thermal fluctuations, such as derivatives, mean, median, or standard deviation, rather than considering only increases or decreases in temperature. That said, future studies could assess whether adding other streams of data, including demographic risk factors (e.g. ...
Article
Full-text available
Around one-third of adults are scared of needles, which can result in adverse emotional and physical responses such as dizziness and fainting (e.g. vasovagal reactions; VVR) and consequently, avoidance of healthcare, treatments, and immunizations. Unfortunately, most people are not aware of vasovagal reactions until they escalate, at which time it is too late to intervene. This study aims to investigate whether facial temperature profiles measured in the waiting room, prior to a blood donation, can be used to classify who will and will not experience VVR during the donation. Average temperature profiles from six facial regions were extracted from pre-donation recordings of 193 blood donors, and machine learning was used to classify whether a donor would experience low or high levels of VVR during the donation. An XGBoost classifier was able to classify vasovagal groups from an adverse reaction during a blood donation based on this early facial temperature data, with a sensitivity of 0.87, specificity of 0.84, F1 score of 0.86, and PR-AUC of 0.93. Temperature fluctuations in the area under the nose, chin and forehead have the highest predictive value. This study is the first to demonstrate that it is possible to classify vasovagal responses during a blood donation using temperature profiles.
... Peripheral skin temperature is a common measure of stress and can be used to distinguish between emotional states (Collet, 1997;Salazar-López 2015;Karthikeyan, 2012). Generally, the theory of blood flow constriction due to stress causing a decrease in skin temperature and blood flow dilation due to happiness causing an increase in skin temperature is used (Karthikeyan, 2012). ...
... Generally, the theory of blood flow constriction due to stress causing a decrease in skin temperature and blood flow dilation due to happiness causing an increase in skin temperature is used (Karthikeyan, 2012). But there are conflicting reports of skin temperature response to emotional states, where few report the temperature of the wrist or hand and instead focus on either monitoring facial temperature with a thermal camera (Salazar-López, 2015) or will place their temperature probe in an area that is free from thermal environmental disturbances, such as under the armpit (Karthikeyan, 2012). For studies that place the temperature probe on the hand (Collet, 1997), a decrease in skin temperature is observed during happy states and an increase in skin temperature is observed for anger. ...
Thesis
Full-text available
When applying machine learning to clinical practice, a major hurdle that will be encountered is the lack of available data. While the data collected in clinical therapies is suitable for the types of analysis that are needed to measure and track clinical outcomes, it may not be suitable for other types of analysis. For instance, video data may have poor alignment with behavioral data, making it impossible to extract the videos frames that directly correlate with the observed behavior. Alternatively, clinicians may be exploring new data modalities, such as physiological signal collection, to research methods of improving clinical outcomes that are incompatible with their existing tools. Both problems warrant the exploration of improving the tools available to clinicians and developing them in a way that allows future customization for future research and clinical needs. This thesis covers the development and user perceived usability of a custom software tool, called cometrics, that was designed to address this data gap and be accessible enough to cover future use cases. Various use cases of this software are explored and a survey from existing users was conducted to provide comparison to existing tools. Additionally, the free and open-source nature of the software not only ensures confidence in the handling of private health information, but also allows anyone to inspect and modify the source code for their specific needs. After deploying this software tool to two independent clinical spaces within the Munroe-Meyer Institute at the University of Nebraska Medical Center, two novel datasets were collected. A physiological dataset focused on expressions of emotion in children was analyzed using statistical models and benchmark machine learning techniques. Secondly, a video-based dataset focused on the expression of severe behavior in children was used for detecting instances of hitting. Both datasets demonstrate the rudimentary capability to train machine learning models to automate annotation of clinical data for both efficiency and early warning use cases. These advances are accelerated by the intersection of clinical practice and engineering, which is made easier using a tool that is made for both parties.
... To have a more suitable appreciation of how the data obtained behave, the box-andwhisker plot of the entire database obtained is shown ( Figure 3). According to what is shown in Figure 3, the trends can be seen in terms of the temperature shown in the different regions of interest evaluated, the one with the greatest variation being the nose, which is one of the most studied and in which other authors have already found that they present consistent results [8,36]. It is interesting to remark that temperatures in other regions of interest are clustered with minor variations as in the case of supraorbitals. ...
... One of the comparisons that comes to mind when it is necessary to identify differ- According to what is shown in Figure 3, the trends can be seen in terms of the temperature shown in the different regions of interest evaluated, the one with the greatest variation being the nose, which is one of the most studied and in which other authors have already found that they present consistent results [8,36]. It is interesting to remark that temperatures in other regions of interest are clustered with minor variations as in the case of supraorbitals. ...
Article
Full-text available
There has been a wide use of thermal images of the human body in recent years, specifically images with thermal information of regions of interest (ROI) in the face; this information can be used for epidemiological, clinical, and/or psychological purposes. Due to this, it is important to have plenty of information on temperature in these ROIs in the basal state that allows their use as a reference in terms of their thermal analysis. In this work, a face thermal map of the Mexican population in the basal state (n = 196) is created, adding the comparison between different population groups, such as gender, age, and clinical status, obtaining results of great interest for future research. The t-test for independent samples was applied to the ROIs with normal distribution and Mann–Whitney u-test to the ones that did not present normal distribution. Statistically significant differences were found in some of the ROI comparisons like the corrugator, the supraorbitals, and the chin between the control and clinical groups, as well as in the differentiation by age (p < 0.05).
... However, before the potential of BSMs can be exploited and its theoretical and practical contributions further elaborated, a critical advancement is required. Previous BSMs are so far based only on the activity dimension (García-Magariño et al., 2018;Hietanen et al., 2016;Nakul et al., 2020;Nummenmaa et al., 2014;Sachs et al., 2019;Salazar-López et al., 2015;Torregrossa et al., 2018;Volynets et al., 2019), thereby completely neglecting the valence dimension. There is now a general agreement that both arousal/activity and valence are the most prominent underlying dimensions of every emotional experience (Colibazzi et al., 2010;Lang et al., 1993;Posner et al., 2005Posner et al., , 2009, and some authors emphasized that both these experiences depend on feedback from the whole body (Russell, 2003;Wiens, 2005). ...
... The aim of this series of experiments was to advance the theoretical and practical value of bodily sensation mapping, a recently developed promising tool for emotional assessment. This tool, up to now, only captured sensations of activity/ arousal (García-Magariño et al., 2018;Hietanen et al., 2016;Lyons et al., 2021;Nummenmaa et al., 2014;Sachs et al., 2019;Salazar-López et al., 2015;Torregrossa et al., 2018;Volynets et al., 2019), and we argued that further capturing sensations of valence-related bodily sensations is crucial for a comprehensive emotional assessment. ...
Article
Full-text available
Bodily sensation mapping (BSM) is a recently developed self-report tool for the assessment of emotions in which people draw their sensations of activation in a body silhouette. Following the circumplex model of affect, activity and valence are the underling dimensions of every emotional experience. The aim of this study was to introduce the neglected valence dimension in BSM. We found that participants systematically report valence-related sensations of bodily lightness for positive emotions (happiness, love, pride), and sensations of bodily heaviness in response to negative emotions (e.g., anger, fear, sadness, depression) with specific body topography (Experiment 1). Further experiments showed that both computers (using a machine learning approach) and humans recognize emotions better when classification is based on the combined activity-and valence-related BSMs compared to either type of BSM alone (Experiments 2 and 3), suggesting that both types of bodily sensations reflect distinct parts of emotion knowledge. Importantly, participants found it clearer to indicate their bodily sensations induced by sadness and depression in terms of bodily weight than bodily activity (Experiment 2 and 4), suggesting that the added value of valence-related BSMs is particularly relevant for the assessment of emotions at the negative end of the valence spectrum.
... Events deemed as competitive (aggression, copulations, dominant arrival, display by another male, hearing screams and feeding with others) were associated with lower temperatures, probably due to a general increase in stress levels (see comparable effects in humans 36 , macaque monkeys 37 and common marmosets 38 ), whereas events regarded as cooperative (grooming 20,28 , playing 29 , patrolling 21,39 and inspecting females) were associated with higher temperatures, probably due a general decrease in stress levels 40,41 . These patterns are consistent with previous findings by Brügger and colleagues 42 , who found nose temperature increases in marmosets exposed to cooperative vocalizations and decreases in response to aggressive vocalisations, similar to what has been reported in humans 43 . Of particular interest were the observed intermediate temperatures when chimpanzees were solicited for cooperation, such as to hand over food, being solicited for sex (young females often solicit although they are not interesting for males 44 ) and encountering a snake, which require lengthy guarding and warning others 16 . ...
... Our study suggested that chimpanzees cognitively categorise most social events as either cooperative or competitive, but that these perceptions are moderated by audience composition, which may explain some inconsistent results in previous studies. Indeed, previous researches hypothesised before that facial temperature changes were linked with arousal 38,49 , valence 43 , or cognitive workload 52 . ...
Article
Full-text available
Accessing animal minds has remained a challenge since the beginnings of modern science. Here, we used a little-tried method, functional infrared thermal imaging, with wild chimpanzees during common social interactions. After removing confounds, we found that chimpanzees involved in competitive events had lower nose skin temperatures whereas those involved in cooperative events had higher temperatures, the latter more so in high- than low-ranking males. Temperatures associated with grooming were akin to those of cooperative events, except when males interacted with a non-reciprocating alpha male. In addition, we found multiple audience effects. Notably, the alpha male’s presence reduced positive effects associated with cooperation, whereas female presence buffered negative effects associated with competition. Copulation was perceived as competitive, especially during furtive mating when other males were absent. Overall, patterns suggest that chimpanzees categorise ordinary social events as cooperative or competitive and that these perceptions are moderated by specific audiences.
... Over the last decade, however, this picture has become more detailed and diverse. For example, FTI has unveiled emotion-associated temperature changes in the cheeks, forehead, and oro-facial area (Salazar-López et al., 2015). Other work has pointed to even larger-scale regional differences. ...
Chapter
Full-text available
What can facial thermography tell us about emotions? The notion that we can leverage measures of bodily responses to reveal something about our inner emotional experiences harks back to William James’ ground-breaking theory on the nature of emotions (James, 1884). At the same time, this quest has met with many criticisms along the way (see Cannon, 1927; Levenson, 2003). Deciphering the relationships between bodily signals and human emotions has fascinated generations of scholars (Levenson, 2003). The face, in particular, continues to attract a lot of interest (Kappas et al., 2013), as researchers hope that subtle and almost invisible changes in facial expressions might provide a key to unveiling someone’s genuine emotions. Where only a few specifically trained experts were previously believed to be able to see and interpret subtle cues of deception in facial muscle movements (Ekman et al., 1999; Frank & Svetieva, 2015), recent technical advances in computational imaging (Nowara et al., 2022) and machine learning may be on the verge of changing this picture (Bian et al., 2024). Perhaps even more stunning, however, is the flexibility with which machine learning methods are beginning to integrate biosignals from different modalities and devices. For example, traditional RGB image data can now be fused with data from the invisible thermal spectrum (Chen et al., 2018; Wang et al., 2014, 2018), thereby allowing researchers to indeed look beneath the skin of facial movements. Could this mean that we may be close to the holy grail of reliably revealing the truth about someone’s emotions and thus revealing the face behind the mask?
... Correlations between infrared thermography read-outs and emotions have been studied in humans (Pavlidis et al. 2002;Esposito et al. 2015) and animals (Stewart et al. 2008a;Kuraoka & Nakamura 2011;Proctor & Carder 2015;Lecorps et al. 2016). In humans, changes in temperature patterns of different regions of interest (ROIs) of the face have been associated with the arousal and valence dimension of emotions (Salazar-López et al. 2015). In animals, changes in peripheral temperature associated with arousal levels have been studied using infrared thermography (Stewart et al. 2008a,b;Kano et al. 2016), Stewart et al. found that cows showed a drop and a subsequent gradual rise in the temperature of the inner corner of the eye in response to negative handling (Stewart et al. 2008a), hot-iron disbudding and surgical castration (Stewart et al. 2008b(Stewart et al. , 2010a; changes mediated by the autonomic nervous system (Stewart et al. 2010b). ...
Article
Full-text available
The emotional valence hypothesis suggests an increased left-brain hemisphere activation during positive situations and vice versa during negative situations. Since facial thermal asymmetries may reflect lateralised brain activity, we investigated this in dairy calves subjected to hot-iron disbudding (Disbudded; n = 12) as a model of negative affective states. As affective responses can vary due to previous experiences, we examined whether calves that had (ExpObs; n = 12) and had not (InexObs; n = 12) experienced disbudding differed in their thermal response to a conspecific being disbudded, and whether calf response to the researcher (approaching, moving away, not moving) was associated with thermal asymmetries. We made thermographic recordings of each calf on three days: Day before disbudding (D1); Disbudding day (D2); and Day after disbudding (D3), and at two different times: Disbudding time/1400(T1) and Afternoon/1700(T2). Data were analysed using multilevel models. Calves had warmer left ears on D2 compared to higher temperatures on the right ear on D1, suggesting higher right-hemisphere activity on D2. ExpObs calves had higher left-eye temperatures when observing a conspecific being disbudded (D2×T1) than InexObs calves that had warmer right eyes, but this reversed on the following day (D3×T1). Calves avoiding the researcher had warmer left eyes whereas those approaching him had warmer right eyes, suggesting greater activation of the right hemisphere in the former. This study provides initial evidence of temperature asymmetries when observing or experiencing a negative event. Further work is required to confirm and build upon these early findings. The study highlights the potential for future development of methods using infrared thermography as a proxy measure of affective valence.
... When a blood vessel is constricted, blood volume is less in that region, which decreases skin temperature. On the contrary, if the person experiences the positive affect of low physiological arousal, such as calm, interest, and curiosity, the blood vessels dilate and increase the temperature in these same regions [31,32]. ...
Article
Full-text available
Previous studies have suggested that mindfulness programs can be useful, in a significant sector of the population, to reduce stress when practiced for at least 8 weeks. The objective of the present investigation was to explore the effect of a single session of mindfulness practice in reducing stress in female cancer survivors. Two repeated measures studies were applied; in the first one, it was performed individually, while in the second one, it was performed in a group. Psychosocial measures were administered, and skin temperature was recorded as a marker of autonomic nervous activity. The results indicate that only when the mindfulness exercise was presented did the skin temperature increase (p < 0.05), with a large effect size (d > 0.8) during compassion, suggesting sympathetic decline. Furthermore, the psychosocial functioning of the group of female cancer survivors was like that of the non-clinical population. The data are discussed in the context of Polyvagal Theory, a theoretical model of biopsychosocial functioning, and evidence is provided on the effect of mindfulness and compassion on reducing stress and inducing positive affect in female cancer survivors.
... The 4 Hz infrared thermopile (TMP) measured skin temperature via changes in infrared radiation emitted by the skin. Peripheral skin temperature is a common measure of stress and has been shown to distinguish between emotional states(Collet et al., 1997;Salazar-López et al., 2015). The typical human temperature is 97.7-99.5°F. ...
Article
Full-text available
This study's purpose is to introduce a new measurement system that objectively assesses the social validity of behavioral feeding intervention from the child's perspective via the concurrent measurement of two dimensions of emotion. To date, the primary measures of social validity for behavioral feeding intervention are caregiver treatment acceptability and satisfaction surveys. This is the first study to objectively measure two dimensions of emotion, physiological arousal and emotional valence, while children received behavioral intervention for feeding difficulties. Data collectors used a new open‐source data collection software, cometrics, developed to synchronize and record physiological and observational data. Physiological data was collected using a wearable biosensor and observers recorded an index of child happiness, unhappiness, or neither using definitions by Phipps et al. (2022) for six children with a pediatric feeding disorder. Two out of six children needed programmed habituation to the device before data collection. All children assented to wearing the device during data collection. Recorded indices mapped to separable physiological states using electrodermal activity and its constituent components, skin conductance responses and skin conductance level, in addition to skin temperature. Findings demonstrated the feasibility of measuring two dimensions of child emotions during behavioral feeding intervention and revealed that children's physiological responses were distinctly different during intervals scored as happy, unhappy, or neither. This new data collection system has implications for future research on the child's emotional experience of behavioral feeding treatment and increases the possibilities for improving clinical practice.
... This suggests that facial skin temperature captures characteristics related to stress-coping responses. The relationship between facial skin temperature and psychology has been reported [8][9][10]. The relationship between nasal skin temperature and stress has also been reported [11]. ...
Article
Individuals exhibit two types of responses when exposed to external stimuli. These are called stress-coping responses, or active and passive coping responses, respectively. These stress-coping responses are discriminated by differences in the fluctuations of hemodynamic parameters, such as cardiac output (CO), total peripheral resistance (TPR), and mean blood pressure (MBP), and others. However, the existing method for measuring hemodynamic parameters is contact measurement, which involves wearing a continuous blood pressure cuff; thus, a remote measurement method is required. Therefore, we focused on facial thermal imaging, remotely measurable indicator of the cardiovascular system. We constructed a model to estimate stress-coping responses from the spatial characteristics of facial thermal images using a CNN and sparse coding. However, the standard spatial distribution of facial thermal images of stress-coping response states has not yet been examined. Therefore, in this study, we analyzed the standard spatial distribution of facial thermal images of stress-coping response states. To elicit each stress-coping response, a cold pressure task and a game task were performed. Facial thermal images and hemodynamic parameters were recorded during the experiments. The measured hemodynamic parameters confirmed the elicitation of a stress-coping response. Additionally, using the measured facial thermal images, we evaluated the deviation of the stress-coping response states from a person’s normal state and the standard spatial distribution of each stress-coping response. The results showed that the stress-coping response states deviated from a person’s normal state. In addition, the standard spatial distribution differed for each stress-coping response.
... In the dimensions of valence and arousal, thermal images were used to mark physical changes during emotional tasks, revealing a link between nose temperature and emotions, particularly valence. Positive valence and arousal patterns led to an increase in nose temperature, while negative valence triggered a decrease [30]. Machine learning has also been incorporated into thermographic emotional studies, demonstrating high accuracy. ...
Article
Full-text available
The acquisition of physiological signals for analyzing emotional experiences has been intrusive, and potentially yields inaccurate results. This study employed infrared thermal images (IRTIs), a noninvasive technique, to classify user emotional experiences while interacting with business-to-consumer (B2C) websites. By manipulating the usability and aesthetics of B2C websites, the facial thermal images of 24 participants were captured as they engaged with the different websites. Machine learning techniques were leveraged to classify their emotional experiences, with participants’ self-assessments serving as the ground truth. The findings revealed significant fluctuations in emotional valence, while the participants’ arousal levels remained consistent, enabling the categorization of emotional experiences into positive and negative states. The support vector machine (SVM) model performed well in distinguishing between baseline and emotional experiences. Furthermore, this study identified key regions of interest (ROIs) and effective classification features in machine learning. These findings not only established a significant connection between user emotional experiences and IRTIs but also broadened the research perspective on the utility of IRTIs in the field of emotion analysis.
... Emotion recognition can be performed with various types of input, such as facial images or physiological signals. Common signals used include heart rate variability (HRV) [8], electrodermal activity (EDA) [9,10], respiration rate (RR) [11], skin temperature (SKT) [12,13], and electromyography (EMG) [14]. Many existing works also suggest electroencephalography (EEG) as a reliable signal to recognize emotion [15]. ...
Article
Full-text available
Objective In this paper, an around-ear EEG system is investigated as an alternative methodology to conventional scalp-EEG-based systems in classifying human affective states in the arousal-valence domain evoked in response to auditory stimuli. Approach EEG recorded from around the ears is compared to EEG collected according to the international 10-20 system in terms of efficacy in an affective state classification task. A wearable device with eight dry EEG channels is designed for ear-EEG acquisition in this study. Twenty-one subjects participated in an experiment consisting of six sessions over three days using both ear and scalp-EEG acquisition methods. Experimental tasks consisted of listening to an auditory stimulus and self-reporting the elicited emotion in response to the said stimulus. Various features were used in tandem with asymmetry methods to evaluate binary classification performances of arousal and valence states using ear-EEG signals in comparison to scalp-EEG. Main Results We achieve an average accuracy of 67.09% ±6.14 for arousal and 66.61%±6.14 for valence after training a multi-layer extreme learning machine with ear-EEG signals in a subject-dependent context, and 63.74% ±3.84 for arousal and 64.32% ±6.38 for valence in a subject-independent context. The best results show no significant differences between ear-EEG and scalp-EEG signals for classifications of affective states. Significance To the best of our knowledge, this paper is the first work to explore the use of around-ear EEG signals in emotion monitoring. Our results demonstrate the potential use of around-ear EEG systems for the development of emotional monitoring setups that are more suitable for use in daily affective life log systems compared to conventional scalp-EEG setups.
... Observers often need to infer other people's emotional state for effective social interactions. It is well established that adults use a combination of perceptual cues ranging from facial, vocal and body expressions, including body temperature and odour, to perceptually discriminate and recognize other people's emotional state (Belin et al., 2012;de Gelder, 2009;de Groot & Smeets, 2017;Keltner et al., 2016;Robinson et al., 2012;Rosen et al., 2015;Salazar-López et al., 2015). The development of emotion processing is therefore an important function to investigate, particularly during the critical first 2 years of life when social communication is largely reliant on non-verbal means (e.g., Crespo-Llado et al., 2018a;Geangu, 2008Geangu, , 2015Leppänen & Nelson, 2009;Miguel et al., 2019;Quadrelli et al., 2019;Zieber et al., 2014). ...
Article
Full-text available
Human body postures provide perceptual cues that can be used to discriminate and recognize emotions. It was previously found that 7-months-olds’ fixation patterns discriminated fear from other emotion body expressions but it is not clear whether they also process the emotional content of those expressions. The emotional content of visual stimuli can increase arousal level resulting in pupil dilations. To provide evidence that infants also process the emotional content of expressions, we analysed variations in pupil in response to emotion stimuli. Forty-eight 7-months-old infants viewed adult body postures expressing anger, fear, happiness and neutral expressions, while their pupil size was measured. There was a significant emotion effect between 1040 and 1640 ms after image onset, when fear elicited larger pupil dilations than neutral expressions. A similar trend was found for anger expressions. Our results suggest that infants have increased arousal to negative-valence body expressions. Thus, in combination with previous fixation results, the pupil data show that infants as young as 7-months can perceptually discriminate static body expressions and process the emotional content of those expressions. The results extend information about infant processing of emotion expressions conveyed through other means (e.g., faces).
... Metacognition refers to the activities involved in thinking and forming integrated ideas about oneself and others (Bonfils et al., 2019). Empathy is defined as the ability to experience and understand the momentary emotional state of another individual (Salazar-López et al., 2015), and each component in empathy is closely contributed to metacognition. Fantasy is one of the empathy components that can be defined as the mental experiences associated with a tendency to identify with characters in movies, novels, plays, and other fictional situations (Davis, 1980(Davis, , 1983. ...
Article
Full-text available
Fantasy is the experience of identifying with characters in movies, novels, plays, and other fictional situations. In social contexts, individuals take on the perspective of others by sensing their emotions through empathy. During this process, perspective-taking and emotional sharing affect one’s metacognition, which deals with the distinction between and the understanding of one’s emotions (clarity) and their regulation (repair); previous studies have primarily focused on these processes. However, perspective-taking—considering another individual’s viewpoint—requires one to imagine their outlook; it also induces emotional responses. This study examined the role of fantasy in clarity and repair in metacognition, for which data derived from 475 Japanese participants were analyzed. The results of the Interpersonal Reactivity Index showed that fantasy was positively associated with clarity and repair in the Trait Meta-Mood Scale; these relationships were moderated by perspective-taking and personal distress. Our results revealed that the emotions experienced within oneself might be understood as the distinction between “imagining” (through their imagination; e.g., internal or mental pictures) and “imaging” (from an image; e.g., external pictures). Individuals imagine their immersion into others using lower-level automatic body sensations (emotional contagion), and the accompanying negative emotions are regulated by metacognition.
... The temporal modulation of the cutaneous temperature of responsive regions (e.g., nose tip, chin, and perioral regions) have been demonstrated as sensitive to the autonomic state of an individual [28]. Hence, given its sensitivity in detecting stress, anxiety, fatigue, and emotions [29][30][31][32][33], IRT has been widely employed for monitoring the psychophysiological status of a subject and for affective computing applications. ...
Article
Full-text available
Alzheimer’s disease (AD) is characterized by progressive memory failures accompanied by microcirculation alterations. Particularly, impaired endothelial microvascular responsiveness and altered flow motion patterns have been observed in AD patients. Of note, the endothelium influences the vascular tone and also the small superficial blood vessels, which can be evaluated through infrared thermography (IRT). The advantage of IRT with respect to other techniques relies on its contactless features and its capability to preserve spatial information of the peripheral microcirculation. The aim of the study is to investigate peripheral microcirculation impairments in AD patients with respect to age-matched healthy controls (HCs) at resting state, through IRT and machine learning (ML) approaches. Particularly, several classifiers were tested, employing as regressors the power of the nose tip temperature time course in different physiological frequency bands. Among the ML classifiers tested, the Decision Tree Classifier (DTC) delivered the best cross-validated accuracy (accuracy = 82%) when discriminating between AD and HCs. The results further demonstrate the alteration of microvascular patterns in AD in the early stages of the pathology, and the capability of IRT to assess vascular impairments. These findings could be exploited in clinical practice, fostering the employment of IRT as a support for the early diagnosis of AD.
... Different emotions activate or deactivate vasomotor neurons from the medulla oblongata that regulate vasodilation and vasoconstriction of the facial vasculature (Drummond, 2013). The other set is of thermal waves (<0.05 Hz), in which different emotional states are also known to alter different parts of the face under the influence of the autonomic nervous system (Salazar-López et al., 2015). ...
Article
Full-text available
Experiential and behavioural aspects of emotions can be measured readily but developing a contactless measure of emotions’ physiological aspects has been a major challenge. We hypothesised that different emotion-evoking films can produce distinctive facial blood flow patterns that can serve as physiological signatures of discrete emotions. To test this hypothesis, we created a new Transdermal Optical Imaging system that uses a conventional video camera to capture facial blood flows in a contactless manner. Using this and deep machine learning, we analysed videos of the faces of people as they viewed film clips that elicited joy, sadness, disgust, fear or a neutral state. We found that each of these elicited a distinct blood flow pattern in the facial epidermis, and that Transdermal Optical Imaging is an effective contactless and inexpensive tool to the reveal physiological correlates of discrete emotions.
... The study [22] associates this type of facial temperature picture with a more demanding cognitive task. Cooling of the nose and warming of the forehead may indicate joy, stress, anger, or anxiety [20,66,69,70]. By considering the level of SC-R, a moderate feeling of anxiety or agitation can be attributed to the children with ADHD during the more successful test solving performance, and the children without ADHD were feeling more stressed. ...
Article
Full-text available
Children with Attention-Deficit/Hyperactivity Disorder (ADHD) face a range of learning difficulties in the school environment, thus several strategies have been developed to enhance or optimise their performance in school. One possible way is to actively enable appropriate restlessness using dynamic seats. In this paper, an assessment of the efficacy of a dynamic seat while solving school task is presented and compared to classic chair and therapy ball. To test the effectiveness of active seat, a study that examined task solving performance while observing the intensity of movement, in-seat behaviour and psychophysiological responses (electrodermal activity, facial temperature) was designed. A total of 23 school-aged children participated in the study, 11 children with a combined type of ADHD and 12 children without disorders. Children with ADHD achieved the best results when sitting in the active seat, where the most intense movement and best in-seat behaviour was observed. At the same time, psychophysiological parameters indicate that when performing better at the task children with ADHD were not too challenged and were consequently less agitated. Results have suggested that for a better cognitive performance of children with ADHD, it is crucial to provide a comfortable and pleasant workspace that enables them the right amount of restlessness.
... Sadness is characterized by a low-arousal physiological state (Huron, 2018) and a conservation-withdrawal tendency which, however, is not consistently reflected across physiological channels (Kreibig et al., 2007). When observing sad body expressions, the participants in the current study showed an initial drop in their cheek temperature (see Salazar-López et al., 2015 for similar findings on negative images with low arousal). As previous research has shown that watching sad body movements can induce sadness in observers (Shafir et al., 2013), the cheek temperature drop in our study might be the result of induced sadness. ...
Article
Affective states are expressed in an individual’s physical appearance, ranging from facial expressions and body postures, to indicators of physiological arousal (e.g., a blush). Confirming the claimed communicative function of these markers, humans are capable of distinguishing between a variety of discrete emotion displays. In an attempt to explain the underlying mechanism, characteristic bodily changes within the observer, including physiological arousal and mimicry, have been suggested to facilitate the interpretation of an expression. The current study aims to create a holistic picture of emotion perception by (1) using three different sources of emotional information (prototypical facial expressions, bodily expressions, and subtle facial cues) and (2) measuring changes in multiple physiological signals (facial electromyography, skin conductance level, skin temperature, and pupil size). While participants clearly discriminated between perceived emotional expressions, there was no overall 1–1 correspondence with their physiological responses. Some specific but robust effects were observed. Angry facial expressions were consistently responded to with a peak in skin conductance level. Furthermore, sad body expressions were associated with a drop in skin temperature. In addition to being the best recognized expression, viewing happy faces elicited congruent facial muscle responses, which supports the potential role of embodied simulation in emotion recognition. Lastly, tears were not only rated as highly emotional intense but also evoked a peak in skin conductance level in the observer. The absence of distinct physiological responses to other expressions could be explained by the lacking functionality of affect sharing in a non-interactive experimental context. Consequentially, emotional alignment in body and mind might especially take place in real social situations, which should be considered in future research.
... Some studies using two types of probe sounds (standard tones, which are presented frequently or as part of a repetitive background, and oddball tones, which are presented infrequently and thus are considered deviant) have shown that the response to sound deviance is modulated after emotional picture presentation [4][5][6] . In these studies, the auditory stimuli are task-irrelevant. ...
Article
Full-text available
There have been various studies on the effects of emotional visual processing on subsequent non-emotional auditory stimuli. A previous study with EEG has shown that responses to deviant sounds presented after presenting negative pictures collected more attentional resources than those for neutral pictures. To investigate such a compelling between emotional and cognitive processing, this study aimed to examined pupillary responses to an auditory stimulus after a positive, negative, or neutral emotional state was elicited by an emotional image. An emotional image was followed by a beep sound that was either repetitive or unexpected, and the pupillary dilation was measured. As a result, we found that the early component of the pupillary response to the beep sound was larger for negative and positive emotional states than the neutral emotional state, whereas the late component was larger for the positive emotional state than the negative and neutral emotional states. In addition, the peak latency of the pupillary response was earlier for negative than neutral or positive images. Further, to compensate for the disadvantage of low-temporal resolution of the pupillary data, the pupillary responses were deconvoluted and used in the analysis. The deconvolution analysis of pupillary responses confirmed that the responses to beep sound were more likely to be modulated by the emotional state rather than being influenced by the short presentation interval between the images and sounds. These findings suggested that pupil size index modulations in the compelling situation between emotional and cognitive processing.
Article
For many mammals, siblings are an important part of the developmental environment, probably contributing to the formation of individual differences in personality. Specifically, in humans sibling effects are thought to be associated with birth order. However, the findings have been controversial. This might be due in part to the use of self-report methods, which depend on subjective perceptions. Thus, it is necessary to also use physiological measures not under voluntary control. We tested the behavioral and physiological responses of 80 young adults—40 men and 40 women, 20 firstborn and 20 lastborn of each sex—to 3 commonly used psychological stressors during a 13-min experimental session. Participants’ behavioral responses were recorded on video, and their physiological reactions were registered via facial infrared thermography. As predicted by birth order theory, firstborns showed significantly greater signs of stress than lastborns, with lower nose temperatures on all tests and for both sexes. Behavior did not differ between the 2 groups. Thus, contradictory findings among previous studies of the association of birth order with personality differences between siblings might be partly due to the low resolution of behavioral evaluation in contrast to the registration of autonomic nervous system activity as, for example, measured by noninvasive, easily implemented facial infrared thermography.
Article
Full-text available
The real-time contactless perception of human thermal comfort is essential for building energy conservation. Most of the traditional human thermal comfort perception methods are contact-type, which requires the use of sensors and other equipment to collect human physiological signals, and the human discomfort is often caused. The existing contactless methods are difficult to measure in real time and are not universal. An end-to-end non-contact human thermal comfort posture estimation algorithm is proposed in this study, which focuses on solving the accuracy and real-time issues of contactless human thermal comfort perception. This algorithm first trains a network to extract the heat map of human joints based on the ResNet structure, and then obtains the pose vector of the human body through the method of non-maximum suppression, and finally uses the LSTM network to process the extracted human pose vector to obtain the current person's posture actions and determine its current thermal comfort state based on posture actions. At the end of this paper, the effectiveness and robustness of the proposed algorithm are verified through test set verification, real-time detection and other methods.
Article
A BSTRACT Introduction Anastomotic leak (AL) is the Achilles’ heel of colorectal surgery. Adequate perfusion of the bowel segments to be anastomosed is a prerequisite for healing. Traditionally surgeons have assessed bowel perfusion subjectively, and studies indicate that it is not sensitive in predicting anastomotic leak rates. Hence, a need for objective assessment of bowel perfusion was advocated and various techniques have been developed to assess bowel perfusion. The purpose of this review is to study the clinical application of intraoperative infrared thermography (IRT) in assessing bowel perfusion in colorectal surgery and its effect on the rate of AL. Methods Literature search using keywords ‘colo-rectal surgery’ ‘bowel perfusion’, ‘anastomotic leak’, ‘infra-red thermography’ was done. All english language studies investigating role of infra-red thermography in assesing bowel perfusion were included in analysis. Results Nine studies were found where bowel perfusion was assessed using IRT in humans as well as animals. These studies demonstrated that IRT is a sensitive and safe investigation and can be used as an adjunct in colorectal surgery. None of the studies were large enough to demonstrate a statistically significant effect on AL rates. Conclusion IRT appears to be a feasible adjunct in colorectal surgery and appears to be a safe and sensitive method to assess bowel perfusion. Further randomized controlled trials with larger patient population are required to advocate its routine use in colorectal surgery.
Article
Empathy is a complex, multi‐dimensional capacity that facilitates the sharing and understanding of others' emotions. As our closest living relatives, bonobos ( Pan paniscus ) and chimpanzees ( P. troglodytes ) provide an opportunity to explore the origins of hominin social cognition, including empathy. Despite certain assumptions that bonobos and chimpanzees may differ empathically, these species appear to overlap considerably in certain socio‐emotional responses related to empathy. However, few studies have systematically tested for species variation in Pan empathic or socio‐emotional tendencies. To address this, we synthesise the growing literature on Pan empathy to inform our understanding of the selection pressures that may underlie the evolution of hominin empathy, and its expression in our last common ancestor. As bonobos and chimpanzees show overlaps in their expression of complex socio‐emotional phenomena such as empathy, we propose that group comparisons may be as or more meaningful than species comparisons when it comes to understanding the evolutionary pressures for such behaviour. Furthermore, key differences, such as how humans and Pan communicate, appear to distinguish how we experience empathy compared to our closest living relatives.
Article
Multimodal emotion recognition techniques are increasingly essential for assessing mental states. Image-based methods, however, tend to focus predominantly on overt visual cues and often overlook subtler mental state changes. Psychophysiological research has demonstrated that heart rate (HR) and skin temperature are effective in detecting autonomic nervous system (ANS) activities, thereby revealing these subtle changes. However, traditional HR tools are generally more costly and less portable, while skin temperature analysis usually necessitates extensive manual processing. Advances in remote photoplethysmography (r-PPG) and automatic thermal region of interest (ROI) detection algorithms have been developed to address these issues, yet their accuracy in practical applications remains limited. This study aims to bridge this gap by integrating r-PPG with thermal imaging to enhance prediction performance. Ninety participants completed a 20-min questionnaire to induce cognitive stress, followed by watching a film aimed at eliciting moral elevation. The results demonstrate that the combination of r-PPG and thermal imaging effectively detects emotional shifts. Using r-PPG alone, the prediction accuracy was 77% for cognitive stress and 61% for moral elevation, as determined by a support vector machine (SVM). Thermal imaging alone achieved 79% accuracy for cognitive stress and 78% for moral elevation, utilizing a random forest (RF) algorithm. An early fusion strategy of these modalities significantly improved accuracies, achieving 87% for cognitive stress and 83% for moral elevation using RF. Further analysis, which utilized statistical metrics and explainable machine learning methods including SHapley Additive exPlanations (SHAP), highlighted key features and clarified the relationship between cardiac responses and facial temperature variations. Notably, it was observed that cardiovascular features derived from r-PPG models had a more pronounced influence in data fusion, despite thermal imaging’s higher predictive accuracy in unimodal analysis.
Chapter
Uncovering 23 Facial Locations Involved in Emotional Transitions: One of the most interesting findings from thermal marker research on emotion is that the thermal expression of emotions is not always consistent. For example, someone who is feeling angry may not always show a red face. This is because the amount of blood flow to the face is also affected by other factors, such as stress, fatigue, and medication. This finding has important implications for the development of emotion recognition systems. It suggests that these systems need to be able to take into account a variety of factors in order to accurately identify emotions. The research is also shedding light on the neural mechanisms of emotion. We have found that the areas of the brain that are involved in processing emotions are also involved in regulating blood flow to the face. This suggests that there is a close connection between the two processes. We are continuing to research the complexity of human expression. We believe that our findings have the potential to revolutionize the way we understand and interact with emotions. In a departure from conventional studies, we present a novel perspective on emotional states through thermal imaging, revealing distinct and individualized transitions in humans. Thermal imaging unveils a fascinating pattern: As people shift from anger, upset, and aggression to states of bliss, happiness, and relaxation, specific regions of their faces respond uniquely. These twelve key regions include the nose, eye corners, eye junctions, forehead vertical lines, forehead horizontals, lips, ears, eyebrows, cheeks, chin, mustaches, and beard, as well as vertical lines parallel to the nose. These regions exhibit temperature changes of 2–7 ℃ during emotional transitions, with each person displaying a unique combination of affected areas in their facial expressions. By applying geometric analysis to individuals’ logical responses to asymmetrical images, we discern that introverts and extroverts employ their eyes differently during emotional transitions. Similarly, individuals categorized as aggressive, subdued, or calm and composed in psychological tests use distinct facial regions to express their emotions. These findings hint at the existence of twelve facial expressions originating from two fundamental categories of human character. We present a study with 287 individuals disproves the notion that all human faces, regardless of race, ethnicity, or beliefs, share a common thermal facial expression for specific emotions. Even people from similar backgrounds show distinct thermal expressions when experiencing the same emotion. We propose using dynamic thermal images instead of static ones as they reveal the geometric patterns of emotions on a person’s face. By observing which facial parts are active during emotional transitions, human subjects can be classified into six main categories and 23 subcategories. The analysis of dynamic thermal images also uncovered time conservation rules governing the latency, duration, and transition modes of human emotions. This finding has the potential to classify individuals into at least 23 different categories and assist in quantitatively estimating and treating mental disorders.
Preprint
Full-text available
Identifying users' experience when using products is one of the major challenges for design. Analyzing users’ psychophysiological reactions to an experience using biofeedback can be more reliable than using subjective evaluations, such as structured interviews and questionnaires. Two case studies were conducted to identify emotions users actually felt and to correlate these to what they reported after using two computational systems. The first system investigated users’ emotions during training on a vehicle driving simulator and the second analyzed the emotions experienced during a car racing game, both in a Virtual Reality environment. Their opinions about their emotional state were obtained using self-report techniques (using the Geneva Emotions Wheel-GEW and Positive, and Negative Affective Schedule-PANAS questionnaires) and applying EEG (brain activity with Frontal Alpha Asymmetry Index – FAAI) and infrared thermography (facial thermograms). The training experiment presented the greater correlation between the psychophysiological and the self-report responses. Results evidenced the importance of undertaking multimodal studies in design research to determine users’ emotional experiences in a virtual reality context.
Article
Most studies have been conducted on the impact of indoor environmental quality (IEQ) on occupants' task performance. However, most studies are based on the assumption that IEQ directly affects task performance. Different hypotheses have been proposed that contradict this conventional belief. The objective of this study is to examine the indirect effects of IEQ on task performance. To identify unknown mediating variables between IEQ and task performance, two research hypotheses were set up: (i) Emotion is the result of a physiological response change in the body and would be influenced by the IEQ in the built environment; and (ii) Task performance would be influenced by IEQ-evoked emotion of occupants. The partial least squares structural equation modeling (PLS-SEM) was employed to verify two interrelated research hypotheses and conduct mediating effect analysis using SmartPLS 3.3.1. The results of this study demonstrated the following: (i) IEQ has no direct effect on task performance but has an indirect effect mediated by dominance resulting from skin conductance; (ii) hot and humid thermal environment and unpleasant indoor air quality increase human skin conductance, which in turn leads to passive emotions. The passive emotions that are not under direct human control result in the degradation of task performance. Using the findings of this study, future smart offices need to collect human bio-signals and emotions and manage IEQ based on these data unlike past smart offices that considered only environmental factors.
Article
Full-text available
Body expressions provide important perceptual cues to recognize emotions in others. By adulthood, people are very good at using body expressions for emotion recognition. Thus an important research question is: How does emotion processing of body expressions develop, particularly during the critical first 2-years and into early childhood? To answer this question, we conducted a meta-analysis of developmental studies that use body stimuli to quantity infants' and young children's ability to discriminate and process emotions from body expressions at different ages. The evidence from our review converges on the finding that infants and children can process emotion expressions across a wide variety of body stimuli and experimental paradigms, and that emotion-processing abilities do not vary with age. We discuss limitations and gaps in the literature in relation to a prominent view that infants learn to extract perceptual cues from different sources about people's emotions under different environmental and social contexts, and suggest naturalistic approaches to further advance our understanding of the development of emotion processing of body expressions.
Article
Full-text available
Purpose: Non-contact measurement of facial temperature using infrared thermography has been used for mass screening of body temperature during a pandemic. We investigated the relative stability of temperature measurement in different facial regions of healthy individuals. Methods: Twenty healthy subjects underwent two experiments. In the first experiment, subjects washed their faces with a 20°C wet towel for 1 minute. Temperature changes compared to baseline in the forehead, cornea, inner canthus, and outer canthus were determined using an infrared camera for 10 minutes. In the second experiment, lubricating eye drops at 20°C were instilled over one eye. Temperature changes in the same regions of interest were monitored for 5 minutes. Results: Baseline temperatures before face washing in the forehead and cornea, inner canthus, and outer canthus of the right eye were 33.4°C ± 0.8°C (mean ± SD), 33.3°C ± 0.8°C, 34.3°C ± 0.7°C, and 32.8°C ± 0.7°C, respectively. Reductions in temperature due to face washing were most significant for the forehead and least significant for the cornea. One minute after face washing, the corresponding changes were -2.8°C ± 0.6°C, -0.3°C ± 0.6°C, -0.6°C ± 0.7°C, and -0.9°C ± 0.7°C for the forehead, cornea, inner canthus, and outer canthus, respectively. After administering the eye drops, no significant temperature changes were observed. Conclusions: When facial temperature was exogenously cooled, the cornea had the most stable temperature readings. Translational relevance: When using infrared thermography to screen facial temperature, the measurement of corneal temperature is probably a better representative if the stability of temperature readings is critical.
Article
Schizophrenia affects patients and their families and society because of chronic impairments in cognition, behavior, and emotion. However, its clinical diagnosis mainly depends on the clinicians’ knowledge of the patient’s symptoms. Other auxiliary diagnostic methods such as MRI and EEG are cumbersome and time-consuming. Recently, the convolutional neural network (CNN) has been applied to the auxiliary diagnosis of psychiatry. Hence, in this study, a method based on deep learning and facial videos is proposed for the rapid detection of schizophrenia. Herein, 125 videos from 125 schizophrenic patients and 75 videos from 75 healthy controls based on emotional stimulation tasks were obtained. The video preprocessing included the experiment clips extraction, face detection, facial region cropping, resizing to 500 × 500 pixel size, and uniform sampling of 100 frames. The preprocessed facial videos were used to train the Resnet18_3D. We utilized ten-fold cross-validation, and held-out testing set to evaluate the model with the accuracy, the precision, the sensitivity, the specificity, and the AUC. The Resnet18_3D trained on Film_order achieved the best performance with accuracy, sensitivity, specificity, and AUC of 89.00%, 96.80%, 76.00%, and 0.9397. The neural network model indeed recognizes healthy controls and schizophrenic patients through the changes in the area of the face. The results show that facial video under emotional stimulation can be used to classify schizophrenic patients and help clinicians with diagnosis in the clinical environment. Among the different types of stimuli, the video stimuli with fixed emotional order showed the best classification performance.
Article
Full-text available
Understanding students’ emotional states during the learning process is one of the important aspects to improve learning quality. Measurements of emotion in an academic setting can be performed manually or automatically using a computer. However, developing an emotion recognition method using an imaging modality that is contactless, harmless, and illumination-independent is challenging. Thermography, as a non-invasive emotion recognition method, can recognize emotion variance during learning by observing the temperature distributions in a facial region. Deep learning models, such as convolutional neural networks (CNNs), can be used to interpret thermograms. CNNs can automatically classify emotion thermograms into several emotional states, such as happiness, anger, sadness, and fear. Despite their promising ability, CNNs have not been widely used in emotion recognition. In this study, we aimed to summarize the previous works and progress in emotion recognition in academic settings based on thermography and CNN. We first discussed the previous works on emotion recognition to provide an overview of the availability of modalities with their advantages and disadvantages. We also discussed emotion thermography potential for the academic context to find if there is any information in the available emotion thermal datasets related to the subjects’ educational backgrounds. Emotion classification using the proposed CNN model was described step by step, including the feature learning illustration. Lastly, we proposed future research directions for developing a representative dataset in the academic settings, fed the segmented image, assigned a good kernel, and built a CNN model to improve the recognition performance.
Article
Full-text available
Thermal cameras capture the infrared radiation emitted from a body in a contactless manner and can provide an indirect estimation of the autonomic nervous system (ANS) dynamics through the regulation of the skin temperature. This study investigates the contribution given by thermal imaging for an effective automatic stress detection with the perspective of a contactless stress recognition system. To this aim, we recorded both ANS correlates (cardiac, electrodermal, and respiratory activity) and thermal images from 25 volunteers under acute stress induced by the Stroop test. We conducted a statistical analysis on the features extracted from each signal, and we implemented subject-independent classifications based on the support vector machine model with an embedded recursive feature elimination algorithm. Particularly, we trained three classifiers using different feature sets: the full set of features, only those derived from the peripheral autonomic correlates, and only those derived from the thermal images. Classification accuracy and feature selection results confirmed the relevant contribution provided by the thermal features in the acute stress detection task. Indeed, a combination of ANS correlates and thermal features achieved 97.37% of accuracy. Moreover, using only thermal features we could still successfully detect stress with an accuracy of 86.84% in a contact-free manner.
Chapter
O livro Semântica cognitiva sócio-histórica: estudos sobre o significado, que ora apresentamos, oferece ao(à) seu(ua) leitor(a) resultados de pesquisas que foram elaboradas tendo como norte teórico a Linguística Cognitiva. Concebida como proposta conciliadora para promover estudos sobre a linguagem e como fonte de frutíferos diálogos entre dimensões da vida humana antes separadas, como a cognitiva e a social, a Linguística Cognitiva foi adotada por diferentes pesquisadores.
Article
Full-text available
Emotions are action dispositions—states of vigilant readiness that vary widely in reported affect, physiology, and behavior. They are driven, however, by only 2 opponent motivational systems, appetitive and aversive—subcortical circuits that mediate reactions to primary reinforcers. Using a large emotional picture library, reliable affective psychophysiologies are shown, defined by the judged valence (appetitive/pleasant or aversive/unpleasant) and arousal of picture percepts. Picture-evoked affects also modulate responses to independently presented startle probe stimuli. In other words, they potentiate startle reflexes during unpleasant pictures and inhibit them during pleasant pictures, and both effects are augmented by high picture arousal. Implications are elucidated for research in basic emotions, psychopathology, and theories of orienting and defense. Conclusions highlight both the approach's constraints and promising paths for future study.
Article
Full-text available
The affective experience of a stimulus has traditionally been studied by statistically correlating the Electroencephalogram (EEG) and Affective Self Report (ASR). Here, this method is extended into a three-way correlation by including measurement of changes in forehead temperature on the right and left sides using Infrared Thermography (IRT). Sixteen male undergraduate designers were given a cognitive task whilst simultaneous IRT and EEG measurements were conducted. Measures of Arousal and Valence were recorded along with an additional post-test measure of Task Engagement. Using a Pearson product-moment correlation coefficient, the initial results exhibited inconclusive evidence of triangulation between the three methods, although a strong positive association was established between changes in forehead temperature and changes in total EEG activity. Further analysis revealed that the sample group was complex: half displayed higher temperatures on the right side and half displayed higher temperatures on the left side throughout the test. Analysis of these smaller groups revealed significant correlations between IRT, EEG, and ASR. The results support the view that IRT has potential use in the measurement of cognitive work and affective state changes during user-product interactions and suggest that further work is required to establish a more definitive relationship between forehead temperature dynamics and affect.
Article
Full-text available
The research aimed at developing a non-intrusive physiological measure for mental workload using human facial skin temperature change. It demonstrated initial results in two driving experiments that showed the potential of using this physiological parameter to infer mental workload. Participants completed driving tests in a simulator in the first experiment. Results of simulator and real vehicle testing were used in a second experiment. Forehead and nose temperature were obtained via thermography. Nose temperature dropped significantly after the drives for all conditions in the simulator tests. A secondary task during driving led to higher subjective workload score and a greater nose temperature drop. Simulator drives led to a higher subjective workload score and a greater nose temperature drop than the real driving task. A significant correlation between the nose skin temperature change and the subjective workload score was yielded in both experiments. Potential applications of this research include real-time, non-intrusive, and automated mental workload assessment for advanced human-system interface development and performance prediction.
Article
Full-text available
Purpose – This paper proposes a new procedure for measuring affective responses during social interaction using facial thermographic imaging. Methodology – We first describe the results of several small pilot experiments designed to develop and refine this new measure that reveal some of the methodological advantages and challenges offered by this measurement approach. We then demonstrate the potential utility of this measure using data from a laboratory experiment (N=114) in which we used performance feedback to manipulate identity deflection and measured several types of affective responses – including self-impressions and emotions. Findings – We find warming of the brow (near the corrugator muscle) and cheek (near the zygomatic major muscle) related most strongly to emotion valence and self-potency, with those whose brows and cheeks warmed the most feeling less positive emotion and less potent self-impressions. Warming in the eye area (near the orbicularis oculi) related most closely to undirected identity deflection and to positive self-sentiments. Positive self-views and strong identity disruptions both contributed to warming of the eyes. Implications – The rigor of contemporary sociological theories of emotion exceeds our current ability to empirically test these theories. Facial thermographic imaging may offer sociologists new assessments of affect and emotion that are ecologically valid, socially unreactive, temporally sensitive, and accurate. This could dramatically improve our ability to test and develop affect based theories of social interaction.
Article
Full-text available
Significance Emotions coordinate our behavior and physiological states during survival-salient events and pleasurable interactions. Even though we are often consciously aware of our current emotional state, such as anger or happiness, the mechanisms giving rise to these subjective sensations have remained unresolved. Here we used a topographical self-report tool to reveal that different emotional states are associated with topographically distinct and culturally universal bodily sensations; these sensations could underlie our conscious emotional experiences. Monitoring the topography of emotion-triggered bodily sensations brings forth a unique tool for emotion research and could even provide a biomarker for emotional disorders.
Conference Paper
Full-text available
Abstract—Interactions with animals have long been known to benefit people emotionally. However, most hospitals and nursing home do not accept animals because of infection, fear and allergy. Robot therapy which uses robots as substitutes has been developed as an alternative for animal assisted therapy. Therapeutic pet robots have been used as tools to improve patients and elderly people’s moods, making them more active and communicative with each other and their care givers. Stress levels have been reduced when elderly interact with therapeutic pets, which have been proven using urine analysis. Affective response of a stimulus from robotic therapeutic pets has been studied by statically correlating the Electroencephalogram (EEG) and Infrared thermo graphic Images (IRT) from subjects under study. The current study investigates human physiological response in terms of infrared thermography (IRT), Galvanic Skin Resistance (GSR) and Heart Rate (HR) with an emphasis on those that are critically relevant to the perception of robotic therapeutic pets. Analysis of the results from the subjects revealed significant correlations between IRT, HR and GSR. Results support the view that the proposed therapeutic pet robots bring physiological benefits for elderly.
Article
Full-text available
The aim of the present study was to establish a thermographic model of healthy subjects' hands and compare it with a model of the hands of patients with freezing fingers syndrome, a group usually regarded as a healthy population. A further aim was to establish the thermographic parameters that distinguish primary Raynaud's pheno-menon (RP) from secondary RP. The study was conducted on a group of 74 subjects, divided into 3 groups: patients with freezing hands symptoms (G1), those with primary RP (G2), and those with limited scleroderma (G3). In addition, 69 healthy volunteers served as a control group (G4). The most distinctive features of healthy subjects' hands are the thermal symmetry between left to right measurements (ΔT<0.5°C) and between mean temperatures of the metacarpus and digits (ΔT<0.5°C (1°C maximum)). A negative correlation was found between mean hands temperature and age of subjects in G4 (p < 0.0001). All the temperatures observed in G4 subjects were significantly higher than among patients in G1, G2 and G3 (p < 0.001). No significant differences were found between mean temperatures in G2 and G3. RP should be suspected when differences between mean temperatures of the metacarpus and digits are ≥ to 3ºC. Moreover, we suggest that a cut-off point > 1ºC is established for subjects with "freezing" symptoms.
Article
Full-text available
Infra red imaging can only produce reliable and valid results if the technique follows established standards. In medical applications these standards are based on the physics of heat radiation and the physiology of thermoregulation of the human body. This paper describes the requirements for the location, setting up the equipment and the preparation of the human subject to be investigated.A list of references is given to support each part of the recommended procedure. Despite the fact that thermal imaging has been available for many years, there are still some applications of this technique which require more research.
Article
Full-text available
This study investigates whether people high in emotional empathy are more facially reactive than are people low in emotional empathy when exposed to pictures of angry and happy facial expressions. Facial electromyographic activity was measured from the corrugator and the zygomatic muscle regions. In accordance with the predictions, the high empathic group reacted with larger corrugator activity to angry as compared to happy faces and with larger zygomatic activity to happy faces. However, the low empathic group did not differentiate between the angry and happy stimuli at all. The high empathic group, as compared to the low empathic group, also rated the angry faces as expressing more anger and the happy faces as being happier. It is concluded that high empathic people are particularly sensitive in reacting with facial reactions to facial expressions and that this ability is accompanied by a higher level of empathic accuracy. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Feelings and emotions influence our daily routine and affect our social behavior. Especially while driving a car we change our emotions very quickly depending on the situation. To understand the different mental situations, the recording of biosignals such as EEG, ECG, blood pressure, eye movements or respiration plays an essential role. Here, a measurement station is presented which can be used to detect a great variety of biosignals from a person sitting in a driving simulator. By developing suitable data processing algorithms, the recorded biosignals can be used to detect and identify the emotional status of a person.
Article
Full-text available
Earlier researchers were able to extract the transient facial thermal features from thermal infrared images (TIRIs) to make binary distinctions between the expressions of affective states. However, effective human-computer interaction would require machines to distinguish between the subtle facial expressions of affective states. This work, for the first time, attempts to use the transient facial thermal features for recognizing a much wider range of facial expressions. A database of 324 time-sequential, visible-spectrum, and thermal facial images was developed representing different facial expressions from 23 participants in different situations. A novel facial thermal feature extraction, selection, and classification approach was developed and invoked on various Gaussian mixture models constructed using: neutral and pretended happy and sad faces, faces with multiple positive and negative facial expressions, faces with neutral and six (pretended) basic facial expressions, and faces with evoked happiness, sadness, disgust, and anger. This work demonstrates that (1) infrared imaging can be used to observe the affective-state-specific facial thermal variations, (2) pixel-grey level analysis of TIRIs can help localise significant facial thermal feature points along the major facial muscles, and (3) cluster-analytic classification of transient thermal features can help distinguish between the facial expressions of affective states in an optimized eigenspace of input thermal feature vectors. The observed classification results exhibited influence of a Gaussian mixture model's structure on classifier-performance. The work also unveiled some pertinent aspects of future research on the use of facial thermal features in automated facial expression classification and affect recognition.
Article
Full-text available
The norms of the Spanish adaptation for shows 9­ 14 (second part) of the International Affective Picture System (lAPS) are presented. The results are highly consistent with those obtained in the first part of the Spanish adaptation and in the original USA version. The picture distribution in the bi-dimensional space, defined by the ratings of valence and arousal, displays the typical boomerang form. In addition, a more pronounced slope -and a smaller dispersion- is observed in the unpleasant extreme of the boomerang than in the pleasant one. The correlations between the Noth-American and the Spanish values are all highly significant. Nevertheless, the differences found in the first part of the study are confirmed: Spanish people perceive the affective pictures, as a whole, as more arousing and less dominant than North-Americans. Similarly, our results confirm the gender differences previously found: women rate the pictures as more arousing and less dominant than men. Although no significant differences are observed in valence ratings, as a whole, the pictures evaluated as more pleasant are clearly different for men and women. The implications of the results regarding the theoretical model underlying the lAPS are highlighted. Key words: emotion, affective valence, arousal, dominance, cross-cultural differences, gender differences.
Article
Full-text available
There is common neural activity in parietal and premotor cortex when executing and observing goal-directed movements: the "mirror" response. In addition, active and passive limb movements cause overlapping activity in premotor and somatosensory cortex. This association of motor and sensory activity cannot ascribe agency, the ability to discriminate between self- and non-self-generated events. This requires that some signals accompanying self-initiated limb movement dissociate from those evoked by observing the action of another or by movement imposed on oneself by external force. We demonstrated associated activity within the medial parietal operculum in response to feedforward visual or somatosensory information accompanying observed and imposed finger movements. In contrast, the response to motor and somatosensory information during self-initiated finger and observed movements resulted in activity localized to the lateral parietal operculum. This ascribes separate functions to medial and lateral second-order somatosensory cortex, anatomically dissociating the agent and the mirror response, demonstrating how executed and observed events are distinguished despite common activity in widespread sensorimotor cortices.
Article
Full-text available
We have developed a high-definition thermal-imaging technique that can detect attempted deceit by recording the thermal patterns from people's faces. This technique has an accuracy comparable to that of polygraph examination by experts and has potential for application in remote and rapid security screening, without the need for skilled staff or physical contact.
Article
Full-text available
Mammals and birds have evolved three primary, discrete, interrelated emotion-motivation systems in the brain for mating, reproduction, and parenting: lust, attraction, and male-female attachment. Each emotion-motivation system is associated with a specific constellation of neural correlates and a distinct behavioral repertoire. Lust evolved to initiate the mating process with any appropriate partner; attraction evolved to enable individuals to choose among and prefer specific mating partners, thereby conserving their mating time and energy; male-female attachment evolved to enable individuals to cooperate with a reproductive mate until species-specific parental duties have been completed. The evolution of these three emotion-motivation systems contribute to contemporary patterns of marriage, adultery, divorce, remarriage, stalking, homicide and other crimes of passion, and clinical depression due to romantic rejection. This article defines these three emotion-motivation systems. Then it discusses an ongoing project using functional magnetic resonance imaging of the brain to investigate the neural circuits associated with one of these emotion-motivation systems, romantic attraction.
Article
Full-text available
General population longitudinal cohort studies have demonstrated the prognostic validity of self-reported psychotic experiences, but data on reliability and cross-validation with interview-based measures of these experiences are sparse. This study tested the reliability and validity of the Community Assessment of Psychic Experiences (CAPE42). At baseline, the CAPE42 was used to measure the subclinical psychosis phenotype in a general population sample (n = 765). At follow-up (mean interval: 7.7 months), the Structured Interview for Schizotypy, Revised (SIS-R), the Brief Psychiatric Rating Scale (BPRS), and the CAPE42 were administered (n = 510). Baseline self-reported dimensions of psychosis were specifically and independently associated with their equivalent interview-based dimension at follow-up (standardized effect sizes of 0.4-0.5) and with their equivalent self-reported measure (standardized effect sizes of 0.6-0.8). The results indicate that self-reported dimensions of psychotic experiences in general population samples appear to be stable, reliable and valid.
Article
Full-text available
Whether observation of distress in others leads to empathic concern and altruistic motivation, or to personal distress and egoistic motivation, seems to depend upon the capacity for self-other differentiation and cognitive appraisal. In this experiment, behavioral measures and event-related functional magnetic resonance imaging were used to investigate the effects of perspective-taking and cognitive appraisal while participants observed the facial expression of pain resulting from medical treatment. Video clips showing the faces of patients were presented either with the instruction to imagine the feelings of the patient (“imagine other”) or to imagine oneself to be in the patient's situation (“imagine self”). Cognitive appraisal was manipulated by providing information that the medical treatment had or had not been successful. Behavioral measures demonstrated that perspective-taking and treatment effectiveness instructions affected participants' affective responses to the observed pain. Hemodynamic changes were detected in the insular cortices, anterior medial cingulate cortex (aMCC), amygdala, and in visual areas including the fusiform gyrus. Graded responses related to the perspective-taking instructions were observed in middle insula, aMCC, medial and lateral premotor areas, and selectively in left and right parietal cortices. Treatment effectiveness resulted in signal changes in the perigenual anterior cingulate cortex, in the ventromedial orbito-frontal cortex, in the right lateral middle frontal gyrus, and in the cerebellum. These findings support the view that humans' responses to the pain of others can be modulated by cognitive and motivational processes, which influence whether observing a conspecific in need of help will result in empathic concern, an important instigator for helping behavior.
Article
Alexithymia, the inability to describe one's own emotions, is linked to deficits in empathy, manifesting as a diminished capacity to recognize or understand the emotions and mental states of others. Several brain centers of autonomic control and interoception that are activated in empathy are thought to misfunction in alexithymia. We hypothesized that individual differences in autonomic changes under affective stimulation might be associated with differences in alexithymia and empathy. We studied 21 healthy volunteers, comparing their alexithymia and empathy scores with changes in their sympathetic autonomic arousal, indexed by the palmar skin potential level, during 3 tasks: playing a computer game, performing mental arithmetic, and watching a negative emotional valence video. Both autonomic and subjective sense of arousal increased at the beginning of each task and then gradually subsided over the course of the task. Higher autonomic arousal at the onset of the computer game was associated with higher empathy scores, and at the onset of the negative video with higher scores for both empathy and alexithymia. Alexithymia delayed the habituation of autonomic arousal during the computer game, while the empathy score was related to a faster decline in arousal during the negative video task. High alexithymia and high empathy scores were linked to increased autonomic arousal at the onset of emotional stimulation, but were distinguishable in the rates of habituation of the evoked arousal. Our data provide insight into the relationships among interacting psychological traits, physiologic regulation, and the arousal dimension of emotional experience.
Article
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
Article
Facial skin temperature enables automatic non-contact measurement, and is valid for objectively evaluating sensations like stress and fatigue because it varies with autonomic nerve activity triggered by the onset of sensations. The present report therefore proposes a method that uses facial skin temperature to infer stress, and outlines the results of evaluations conducted to confirm the validity of the inference formula. At the same time, this report also proposes a formula that infers fatigue through facial skin temperature based on the assumption that fatigue is the result of accumulated stress, and experimental results here suggest that the fatigue inference formula is also valid. Furthermore, color image evaluations using the stress inference formula suggest that the stress inference formula is valid for objectively evaluating color images as well. Relevance to industryThe present method can be used to create a system of evaluating sensations through facial skin temperature. The system is useful not only to estimate stress and fatigue of operators in nuclear power plant, but also to evaluate color images.
Article
We studied the method for objectively evaluating emotion through FST (Facial Skin Thermogram), which brings less stress and non-contact measurement. We tried to estimate unpleasant and pleasant states on FST with Neural Network. Subjective VAS was precisely estimated at any estimation accuracy.
Article
Synaesthesia is a condition in which one type of stimulation evokes the sensation of another, as when the hearing of a sound produces photisms, i.e. mental percepts of colours. R is a 20 year old colour blind subject who, in addition to the relatively common grapheme-colour synaesthesia, presents a rarely reported cross modal perception in which a variety of visual stimuli elicit aura-like percepts of colour. In R, photisms seem to be closely related to the affective valence of stimuli and typically bring out a consistent pattern of emotional responses. The present case study suggests that colours might be an intrinsic category of the human brain. We developed an empirical methodology that allowed us to study the subject's otherwise inaccessible phenomenological experience. First, we found that R shows a Stroop effect (delayed response due to interference) elicited by photisms despite the fact that he does not show a regular Stroop with real colours. Secondly, by manipulating the colour context we confirmed that colours can alter R's emotional evaluation of the stimuli. Furthermore, we demonstrated that R's auras may actually lead to a partially inverted emotional spectrum where certain stimuli bring out emotional reactions opposite to the normal ones. These findings can only be accounted for by considering R's subjective colour experience or qualia. Therefore the present paper defends the view that qualia are a useful scientific concept that can be approached and studied by experimental methods.
Conference Paper
Useful information about affects and affective states can be extracted form the physiological signals, even under difficult lighting and pose conditions. Little work has been done on using physiological signals in automated affect recognition systems. We employed measurements of facial skin temperature variations for developing a non-intrusive automated facial expression classification system. Variances in thermal intensity values recorded at thermally significant locations on human faces were used to discern between normal, pretended happy and pretended sad facial expression of affective states. A three-step algorithmic approach was used to construct the classifier. Employed approach involved derivation of principal components, stepwise selection of optimal and most discriminating features, and linear discriminant analysis within the reduced optimal eigenspace. The resulting classifier performed at an impressively low (16.2%) error rate
Article
Research on subjective well-being suggests that it is only partly a function of environmental circumstances. There may be a personality characteristic or a resilient disposition toward experiencing high levels of well-being even in unfavorable circumstances. Adult attachment may contribute to this resilient disposition. This study examined whether the association between attachment anxiety and subjective well-being was mediated by Neff's (2003a, 2003b) concept of self-compassion. It also examined empathy toward others as a mediator in the association between attachment avoidance and subjective well-being. In Study 1, 195 college students completed self-report surveys. In Study 2, 136 community adults provided a cross-validation of the results. As expected, across these 2 samples, findings suggested that self-compassion mediated the association between attachment anxiety and subjective well-being, and emotional empathy toward others mediated the association between attachment avoidance and subjective well-being.
Article
Using an infrared thermographic system, we have demonstrated, as previously reported, that temperatures in the nasal region of macaque monkeys decrease during negative emotional states, such as when facing a threatening person. In this study, we explored the usefulness of measuring nasal skin temperatures in studies of monkey emotions as manifested by conspecific emotional behaviors and expressions. We measured nasal skin temperatures of rhesus monkeys (Macaca mulatta) in response to video clips, all showing monkeys: a raging individual (Experiment 1), three distinct emotional expressions (Experiment 2), and only faces or voices representing a threat (Experiment 3). We found that nasal skin temperatures significantly decreased in response to a threatening stimulus, even when the stimulus was a 2D image with digitized sound, similar to those used in many psychological or neurophysiological studies on animal emotion. Moreover, species-specific aggressive threats invariably elicited a decrease in nasal skin temperatures and skin conductance responses; however, screams or coos did not elicit this response. Simultaneous perception of both facial expressions and vocalizations induced a more prominent decrease in nasal skin temperatures than did the perception of facial expressions or vocalizations alone. Taken together, these data suggest that decreased nasal skin temperatures should be added to the list of indicators of emotional states in animals.
Article
Tesis Univ. Granada. Departamento de Personalidad, Evaluación y Tratamiento Psicológico. Leída el 24 de marzo de 2006
Article
The theory that stress (or emotional) rise in central temperature (Tc) in rats is a fever with an upward shift of the set-point temperature was tested with three experiments: 1) Measurement of tail skin temperature and Tc during the emotional Tc rise; 2) Investigation of the effect of ambient temperature on the emotional Tc rise; and 3) The assessment of emotional Tc rise during daytime and nighttime. Skin vasomotor responses helped the increase of Tc toward a higher level and contributed to the regulation of central temperature at this new higher level. The cold environment did not diminish the emotional rise of central temperature as it would be expected in the case of a hyperthermia. However, at night emotional fever reached a higher level than during the daytime, suggesting that prostaglandin rise in Tc is distinct from emotional or stress-induced hyperthermia. In conclusion, the experiments reported here confirm the hypothesis that the rise of Tc induced by handling or disturbance of the rats is regulated, and is due to a shift of the set-point as occurs in fever.
Article
The construct of empathy may be located conceptually at several different points in the network of interpersonal cognition and emotion. We discuss one specific form of emotional empathy--other-focused feelings evoked by perceiving another person in need. First, evidence is reviewed suggesting that there are at least two distinct types of congruent emotional responses to perceiving another in need: feelings of personal distress (e.g., alarmed, upset, worried, disturbed, distressed, troubled, etc.) and feelings of empathy (e.g., sympathetic, moved, compassionate, tender, warm, softhearted, etc.). Next, evidence is reviewed suggesting that these two emotional responses have different motivational consequences. Personal distress seems to evoke egoistic motivation to reduce one's own aversive arousal, as a traditional Hullian tension-reduction model would propose. Empathy does not. The motivation evoked by empathy may instead be altruistic, for the ultimate goal seems to be reduction of the other's need, not reduction of one's own aversive arousal. Overall, the recent empirical evidence appears to support the more differentiated view of emotion and motivation proposed long ago by McDougall, not the unitary view proposed by Hull and his followers.
Article
Emotions are action dispositions--states of vigilant readiness that vary widely in reported affect, physiology, and behavior. They are driven, however, by only 2 opponent motivational systems, appetitive and aversive--subcortical circuits that mediate reactions to primary reinforcers. Using a large emotional picture library, reliable affective psychophysiologies are shown, defined by the judged valence (appetitiv