ArticlePDF Available

Is eye contact the key to the social brain?

Authors:
Atsushi Senju at Hamamatsu University School of Medicine
Atsushi Senju
Mark H. Johnson
Mark H. Johnson
Mark H. Johnson
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Download full-text PDF
Read full-text
Download citation

Abstract

Eye contact plays a critical role in many aspects of face processing, including the processing of smiles. We propose that this is achieved by a subcortical route, which is activated by eye contact and modulates the cortical areas involve in social cognition, including the processing of facial expression. This mechanism could be impaired in individuals with autism spectrum disorders.
Content uploaded by Atsushi Senju
Author content
All content in this area was uploaded by Atsushi Senju
Content may be subject to copyright.
1
Behavioral and Brain Sciences (2010) 33, 458-459.
Is eye contact the key to the social brain?
Atsushi Senju and Mark H Johnson
Centre for Brain and Cognitive Development, Birkbeck, University of London,
London WC1E 7HX, United Kingdom
a.senju@bbk.ac.uk
mark.johnson@bbk.ac.uk
Abstract: Eye contact plays a critical role in many aspects of face processing, including the
processing of smiles. We propose that this is achieved by a subcortical route, which is
activated by eye contact and modulates the cortical areas involve in social cognition,
including the processing of facial expression. This mechanism could be impaired in
individuals with autism spectrum disorders.
2
The Simulation of Smiles (SIMS) model proposed by Niedenthal et al. emphasizes the core
role of eye contact, which is hypothesized to trigger embodied simulation of the perceived
smile. The authors also speculated that the same mechanism may also mediate the processing
of other facial expressions. However, eye contact is known to modulate a far wider range of
cognitive processes, such as the encoding of gender, identity, and gaze (Senju & Johnson
2009b). We recently reviewed this phenomenon, which we have termed the “eye contact
effect,” and proposed the fast-track modulator (FTM) model to explain its neural and
developmental basis (Senju & Johnson 2009b). In this commentary, we present a brief
overview of the FTM model and discuss several areas in which the FTM model
complements the SIMS model, and thus would facilitate further exploration of the neural,
cognitive, and developmental mechanism underlying the effect of eye contact on face
processing.
The FTM model proposes that the eye contact effect is mediated by a subcortical face
detection pathway hypothesized to include the superior colliculus, pulvinar, and amygdala.
This route is fast, operates on low spatial frequency visual information, and modulates
cortical face processing (Figure 1).<F1> Evidence that the route is fast comes from
event-related potential and magnetoencephalographic studies showing that components
associated with this pathway can occur at shorter latencies than those usually associated with
the cortical processing of faces (Bailey et al. 2005). In addition, evidence that the subcortical
route modulates cortical processing comes from several functional imaging studies
indicating that the degree of activation of structures in the subcortical route (amygdala,
superior colliculus, and pulvinar) predicts or correlates with the activation of cortical face
processing areas (George et al. 2001, Kleinhans et al. 2008). It has also been proposed that
the subcortical route is also responsible for face preference in newborn infants (Johnson
2005) and even in adults (Tomalski et al. 2009). We hypothesized that the combination of
this subcortical pathway and contextual modulation given by the task demands and social
context directly or indirectly modulates key structures involved in the cortical social brain
network.
The FTM model shares several key features with the SIMS model. However, there are
several differences between these two models, by which the FTM model expands and
broadens the SIMS model. First, the FTM model proposes the neural mechanism linking eye
contact and facilitation of cortical face processing, including the embodied simulation. The
FTM model proposes that perceived eye contact directly activates a subcortical route, which
then modulates the cortical areas involved in different aspects of social cognitive processing.
Thus, it is possible to incorporate the SIMS model by arguing that the subcortical route also
modulates the motor cortex, which controls mimicry. The FTM model also provides new
predictions about the effect of eye contact on the processing of smiles: It should be fast and
operate on low spatial frequency visual information.
Second, the FTM model can also provide alternative hypotheses about the mechanism
by which eye contact facilitates the processing of smiles. The FTM model hypothesizes that
the subcortical route receiving input from perceived eye contact directly modulates the
cortical face processing areas. This contrasts with the SIMS model that assumes that eye
contact must elicit the embodied simulation first in order to facilitate the processing of
smiles. As we discussed earlier, the FTM does not rule out the possibility that the subcortical
route mediates the embodied simulation in response to eye contact. However, the FTM also
leads us to propose a more parsimonious hypothesis: The subcortical route directly
3
modulates visual cortical areas, which then facilitates the processing of facial expression,
including smiles. For example, the FTM model predicts that eye contact modulates the
processing of smiles even when the activation of the motor cortex is experimentally
suppressed. By contrast, the SIMS model would not predict that eye contact facilitates the
processing of smiles under this condition, because embodied simulation is suppressed.
Third, the FTM model presents a unique perspective on the development of the eye
contact effect. In the target article, Niedenthal et al. suggested an interesting hypothesis that
the preference for eye contact in infants reflects an evolutionary-based mechanism for
triggering embodied simulation, even though they did not discuss how such a mechanism
develops. By contrast, the FTM model assumes that infants are born with widespread
connections between the subcortical route and cortical structures. As a consequence, input
from perceived eye contact initially activates widespread cortical structures, which combines
with architectural bias to form specific connections between the subcortical “eye contact
detector” and relevant cortical and subcortical structures during the course of development.
Interestingly, recent studies on the early development of autism spectrum disorder (ASD),
showing manifest atypical patterns of eye contact behavior, are consistent with the
predictions based on the FTM model. Even though infants and young children with autism
show apparently typical eye contact behavior (Chawarska & Shic 2009), neuroimaging
studies demonstrate more widespread and nonspecific cortical activation in response to eye
contact (Elsabbagh et al. 2009), and behavioral studies demonstrated that eye contact does
not facilitate cognitive processing in children with ASD (Senju et al. 2003). These studies
suggests that infants and young children with ASD are sensitive to eye contact, but that it
fails to modulate cortical face processing in the same specialized way as typically developing
children (Senju & Johnson 2009a). Future studies will need to test whether eye contact
elicits facial mimicry and affects the processing of smiles in individuals with ASD,
especially because current evidence is inconsistent as to whether individuals with ASD show
spontaneous facial mimicry (Magnée et al. 2007) or not (McIntosh et al. 2006, Oberman et al.
2009).
Eye contact plays a critical role in face-to-face communication, and we propose it is
the key to adaptively modulate many aspects of social cognition, including the processing of
smiles. We hope the areas of overlap and contrast between the SIMS and the FTM models
will generate empirical studies, and help further understand the neural, cognitive, and
developmental mechanisms underlying human social behavior.
ACKNOWLEDGMENTS
Atsushi Senju was supported by an ESRC Research Fellowship (RES-063–27–0207), and
Mark H Johnson was supported by the UK Medical Research Council (G0701484).
4
References [Atsushi Senju] [AS]
Bailey, A. J., Braeutigam, S., Jousmaki, V. & Swithenby, S. J. (2005) Abnormal activation
of face processing systems at early and intermediate latency in individuals with
autism spectrum disorder: A magnetoencephalographic study. European Journal of
Neuroscience 21:2575–85. [AS]
Chawarska, K. & Shic, F. (2009) Looking but not seeing: Atypical visual scanning and
recognition of faces in 2 and 4-year old children with Autism Spectrum Disorder.
Journal of Autism and Developmental Disorders 39:1663–72. [AS]
Elsabbagh, M., Volein, A., Csibra, G., Holmboe, K., Garwood, H., Tucker, L., Krljes, S.,
Baron-Cohen, S., Bolton, P., Charman, T., Baird, G. & Johnson, M. H. (2009) Neural
correlates of eye gaze processing in the infant broader autism phenotype. Biological
Psychiatry 65:31–38. [AS]
George, N., Driver, J. & Dolan, R. J. (2001) Seen gaze-direction modulates fusiform activity
and its coupling with other brain areas during face processing. Neuroimage
13:1102–12. [AS]
Johnson, M. H. (2005) Subcortical face processing. Nature Reviews Neuroscience 6:766–74.
[AS]
Kleinhans, N. M., Richards, T., Sterling, L., Stegbauer, K. C., Mahurin, R., Johnson, L. C.,
Greenson, J., Dawson, G. & Aylward, E. (2008) Abnormal functional connectivity in
autism spectrum disorders during face processing. Brain 131:1000–12. [AS]
Magnée, M. J., Stekelenburg, J. J., Kemner, C. & de Gelder, B. (2007) Similar facial
electromyographic responses to faces, voices, and body expressions. Neuroreport
18:369–72. [AS]
McIntosh, D. N., Reichmann-Decker, A., Winkielman, P. & Wilbarger, J. L. (2006) When
the social mirror breaks: Deficits in automatic, but not voluntary, mimicry of
emotional facial expressions in autism. Developmental Science 9:295–302. [AS]
Oberman, L. M., Winkielman, P., & Ramachandran, V. S. (2009). Slow echo: facial EMG
evidence for the delay of spontaneous, but not voluntary, emotional mimicry in
children with autism spectrum disorders. Developmental Science, 12:510-20. [AS]
Senju, A., & Johnson, M. H. (2009a). Atypical eye contact in autism: Models, mechanisms
and development. Neuroscience & Biobehavioral Reviews, 33:1204-14. [AS]
Senju, A. & Johnson, M. A., (2009b) The eye contact effect: Mechanisms and development.
Trends in Cognitive Sciences 13:127–34. [AS]
Senju, A., Yaguchi, K., Tojo, Y., & Hasegawa, T. (2003). Eye contact does not facilitate
detection in children with autism. Cognition, 89:B43-51. [AS]
Tomalski, P., Johnson, M. H. & Csibra, G. (2009) Temporal-nasal asymmetry of rapid
orienting to face-like stimuli. Neuroreport 20:1309–12. [AS]
5
<CAPT>Figure 1. (Senju et al.) An illustration of the fast-track modulator model (FTM).
Perceived eye contact (upper left) is initially detected by a subcortical route that projects to
various regions of the social brain network (thick black lines). This signal from the
subcortical route then interacts with contextual modulation based on the task demands, as
well as the social context (thick gray lines) to modulate the response of these regions to the
following input from a cortical route (thin black lines). These pathways are based on
previous analyses of cortical and subcortical face processing, as well as on top-down
voluntary attention. OFC = orbitofrontal cortex, PFC = prefrontal cortex, STS = superior
temporal sulcus. Reproduced with permission from A. Senju & M. H. Johnson (2009b) The
eye contact effect: Mechanisms and development. Trends in Cognitive Sciences 13:127–34.
... Faces and eye gaze are particularly salient cues for humans (Driver et al., 1999;Frischen et al., 2007;Kizilcec et al., 2014), with a large portion of the cortex devoted to their processing (Kanwisher & Yovel, 2006;Kingstone et al., 2000). Attending to another person's direct gaze is an important signal in human social communication (Argyle et al., 1973;Mares et al., 2016;Senju & Johnson, 2010). In addition, attending to another person's eye gaze direction is important for gaze following in support of joint atten-tion (Cleveland et al., 2007;Driver et al., 1999). ...
... The instructor's facing direction and eye gaze may serve as important aspects of the instructional message conveyed in a video lesson (Ouwehand et al., 2015;van Gog & Jarodzka, 2013) by capturing or directing learners' attention at key points during the lesson. For example, the instructor's face is a strong visual cue that can attract a learner's attention (Böckler et al., 2014;Driver et al., 1999;Ouwehand et al., 2015;Zwickel & Vô, 2010), the instructor's direct gaze to the learner (i.e., a social cue) may be beneficial for promoting social engagement (Conty et al., 2016;Senju & Johnson, 2010;Wang et al., 2018), and the instructor's gaze guidance to the whiteboard (i.e., an attentional cue) may be beneficial for redirecting a learner's attention (Driver et al., 1999;Wang et al., 2018) to the presented instructional material. In a similar manner, facing direction and eye gaze may be thought of as deictic gestures (i.e., pointing) because they, like pointing gestures, direct a viewer's attention toward critical aspects of the shared environment (Cochet & Vauclair, 2014). ...
... For the embodied instructional message, we posit two possible pathways from instructional video to posttest performance-the social pathway and the attentional pathway, as summarized in Figure 1. In the social pathway, the instructor's eye gaze toward the learner (i.e., direct gaze) is a social cue that draws the learner to look at the instructor (Mares et al., 2016;Senju & Hasegawa, 2005;Senju & Johnson, 2010), which creates a feeling of social partnership (Boggia & Ristic, 2014) in the learner, thereby contributing to increased engagement (Hayward et al., 2017). We propose that these social processes contribute to greater motivation to engage with the lesson, leading to better posttest performance (Weidlich & Bastiaens, 2017). ...
View
... Faces and eye gaze are particularly salient cues for humans (Driver et al., 1999;Frischen et al., 2007;Kizilcec et al., 2014), with a large portion of the cortex devoted to their processing (Kanwisher & Yovel, 2006;Kingstone et al., 2000). Attending to another person's direct gaze is an important signal in human social communication (Argyle et al., 1973;Mares et al., 2016;Senju & Johnson, 2010). In addition, attending to another person's eye gaze direction is important for gaze following in support of joint atten-tion (Cleveland et al., 2007;Driver et al., 1999). ...
... The instructor's facing direction and eye gaze may serve as important aspects of the instructional message conveyed in a video lesson (Ouwehand et al., 2015;van Gog & Jarodzka, 2013) by capturing or directing learners' attention at key points during the lesson. For example, the instructor's face is a strong visual cue that can attract a learner's attention (Böckler et al., 2014;Driver et al., 1999;Ouwehand et al., 2015;Zwickel & Vô, 2010), the instructor's direct gaze to the learner (i.e., a social cue) may be beneficial for promoting social engagement (Conty et al., 2016;Senju & Johnson, 2010;Wang et al., 2018), and the instructor's gaze guidance to the whiteboard (i.e., an attentional cue) may be beneficial for redirecting a learner's attention (Driver et al., 1999;Wang et al., 2018) to the presented instructional material. In a similar manner, facing direction and eye gaze may be thought of as deictic gestures (i.e., pointing) because they, like pointing gestures, direct a viewer's attention toward critical aspects of the shared environment (Cochet & Vauclair, 2014). ...
... For the embodied instructional message, we posit two possible pathways from instructional video to posttest performance-the social pathway and the attentional pathway, as summarized in Figure 1. In the social pathway, the instructor's eye gaze toward the learner (i.e., direct gaze) is a social cue that draws the learner to look at the instructor (Mares et al., 2016;Senju & Hasegawa, 2005;Senju & Johnson, 2010), which creates a feeling of social partnership (Boggia & Ristic, 2014) in the learner, thereby contributing to increased engagement (Hayward et al., 2017). We propose that these social processes contribute to greater motivation to engage with the lesson, leading to better posttest performance (Weidlich & Bastiaens, 2017). ...
The Case for Embodied Instruction: The Instructor as a Source of Attentional and Social Cues in Video Lectures
Article
Full-text available
  • Dec 2020
There is a growing demand for online courses but there are few validated practices to guide instructors on how to design and deliver effective online lessons. This study investigates whether an instructor’s facing direction and eye gaze behavior can be used as cues to support student engagement, attention, and learning. Access to the instructor’s eye gaze varied by whether she looked at the student and whether her face was visible to the students. We found that the instructor’s face and eye gaze can direct students’ attention during a video lecture, but also it may interfere with learning when students should attend to information on the board rather than to the instructor. Evidence from this study suggests that instructors should choreograph how they use their behavioral cues to guide students through a video lesson.
View
... Recent research suggests that ASDs represent extreme manifestations of socioemotional and communication behaviours that exist on a continuum. Children with both parents exhibiting high levels of autistic behaviours are more likely to have ASD themselves, as certain autistic behaviours are moderate to highly heritable [7]. Genetics can also shed light on more nuanced aspects of cognitive and socioemotional disorders associated with autism. ...
Exploring Sparse Gaussian Processes for Bayesian Optimization in Convolutional Neural Networks for Autism Classification
Article
Full-text available
  • Jan 2024
Autism Spectrum Disorder (ASD) constitutes a prevalent childhood condition, impacting approximately 0.62% of the global population. Children grappling with ASD encounter challenges in acquiring language proficiency, comprehending spoken communication, and interpreting nonverbal cues, including eye contact, facial expressions, and hand gestures. This discourse delves into the application of eye tracking as a foundational instrument in ASD screening, leveraging the distinctive attributes of gaze behavior. Within the realm of Artificial Intelligence(AI), Deep Learning(DL) manifests substantial promise, exhibiting considerable strides across diverse domains and emerging as an indispensable technological paradigm. This treatise introduces an innovative Convolutional Neural Network (CNN) model predicated on Bayesian optimization for the discernment of eye-tracking images. The model comprises two primary components: the initial segment employs CNN to extract and amalgamate salient features, while the subsequent section incorporates a Bayesian optimizer tasked with fine-tuning the CNN’s hyperparameters contingent on an objective function. In the preliminary analysis, three distinct scenarios devoid of specific elements are juxtaposed with Bayesian optimization. These scenarios undergo evaluation through convergence charts and accuracy metrics, culminating in an impressive 97% accuracy rate. The proposed algorithm evinces superior performance, yielding a noteworthy accuracy level of 97% in the model evaluation. This outcome constitutes a commendable advancement in research, accentuating the efficacy and resilience of the developed approach. The article undertakes a comparative analysis of research methodologies and thematic analysis approaches, furnishing valuable insights for qualitative researchers and offering methodologically robust pathways to address research inquiries. In practical applications, the model attests to heightened reliability and precision, thereby underscored as efficacious in real-world scenarios.
View
... Communicative gaze is a strong social signal that has the potential to alter other sociocognitive processes (for a review, see Hamilton, 2016;Senju & Johnson, 2010). Prior work has shown that communicative gaze has the ability to modulate attention orienting (Kompatsiari et al., 2018(Kompatsiari et al., , 2022Belkaid, Kompatsiari, Tommaso, Zablith, & Wykowska, 2021;McKay et al., 2021;Dalmaso, Alessi, et al., 2020;Xu et al., 2018;Bristow et al., 2007). ...
Modulatory Effects of Communicative Gaze on Attentional Orienting Are Driven by Dorsomedial Prefrontal Cortex but Not Right Temporoparietal Junction
Article
Full-text available
Communicative gaze (e.g., mutual or averted) has been shown to affect attentional orienting. However, no study to date has clearly separated the neural basis of the pure social component that modulates attentional orienting in response to communicative gaze from other processes that might be a combination of attentional and social effects. We used TMS to isolate the purely social effects of communicative gaze on attentional orienting. Participants completed a gaze-cueing task with a humanoid robot who engaged either in mutual or in averted gaze before shifting its gaze. Before the task, participants received either sham stimulation (baseline), stimulation of right TPJ (rTPJ), or dorsomedial prefrontal cortex (dmPFC). Results showed, as expected, that communicative gaze affected attentional orienting in baseline condition. This effect was not evident for rTPJ stimulation. Interestingly, stimulation to rTPJ also canceled out attentional orienting altogether. On the other hand, dmPFC stimulation eliminated the socially driven difference in attention orienting between the two gaze conditions while maintaining the basic general attentional orienting effect. Thus, our results allowed for separation of the pure social effect of communicative gaze on attentional orienting from other processes that are a combination of social and generic attentional components.
View
... In the present study, we did not measure participants' intelligence, as (40), but we asked participants to self-report their comprehension/intellectual complexity level. The different results that we obtained compared to those of De Keersmaecker et al. (41) might be related to the various psychological mechanisms that might also determine the attitudes toward disability, including cultural influences (47), peer norms (48), parental and social influences (49), moral values (50), or other related variables. Also, our measurements were different; we did not use the same scales, though the concepts share many similarities (51). ...
Do We React Differently Toward Bionic Devices Versus Cochlear Implants and Wheelchairs? Possible Links with Personality Traits
Article
Full-text available
  • Jun 2023
The present study explored the attitudes toward individuals with bionic eyes and limbs, cochlear implants, and people with disabilities that imply using a wheelchair. Our sample consisted of 474 Romanian adults aged 18 to 61 (M = 27.56, SD = 11.80). Participants were randomly divided into five groups. They all filled scales related to personality characteristics, i.e., agreeableness, neuroticism, openness to experience, comprehension/ intellectual efficiency, and previous contact with disability. Then, each group was presented with a vignette describing a character (wheelchair / bionic eye / bionic leg / cochlear implant/control group). Finally, they answered questions about their emotions, cognitions, and behaviors related to that context. Overall, our results suggested that higher agreeability, extraversion, openness to experience, intellectual complexity, and lower neuroticism were generally associated with more positive attitudes toward disability. When examining the differences in participants' emotions, cognitions, and behaviors depending on the target's characteristics, our results generally suggested that the most negative reactions were toward the character with a bionic eye. We discuss these findings considering their importance for shaping positive attitudes related to disability, especially related to the future technological advances in bionic devices.
View
... Communicative gaze is a strong social signal that has the potential to alter other sociocognitive processes (for a review, see Hamilton, 2016;Senju & Johnson, 2010). Prior work has shown that communicative gaze has the ability to modulate attention orienting (Kompatsiari et al., 2018(Kompatsiari et al., , 2022Belkaid, Kompatsiari, Tommaso, Zablith, & Wykowska, 2021;McKay et al., 2021;Dalmaso, Alessi, et al., 2020;Xu et al., 2018;Bristow et al., 2007). ...
Modulatory effects of communicative gaze on attentional orienting are driven by dmPFC but not rTPJ
Preprint
  • Feb 2023
Communicative gaze (e.g., mutual or averted) has been shown to affect attentional orienting. However, no study to date has clearly separated the neural basis of the pure social component that modulates attentional orienting in response to communicative gaze from other processes that might be a combination of attentional and social effects. We used Transcranial Magnetic Stimulation (TMS) to isolate the purely social effects of communicative gaze on attentional orienting. Participants completed a gaze-cueing task with a humanoid robot who engaged either in mutual or in averted gaze prior to shifting its gaze. Before the task, participants received either Sham stimulation (baseline), stimulation of right temporoparietal junction (rTPJ) or dorso-medial prefrontal cortex (dmPFC). Results showed, as expected, that communicative gaze affected attentional orienting in baseline condition. This effect was not evident for rTPJ stimulation. Interestingly, stimulation to rTPJ also canceled out attentional orienting altogether. On the other hand, dmPFC stimulation eliminated the socially-driven difference in attention orienting between the two gaze conditions while maintaining the basic general attentional orienting effect. Thus, our results allowed for separation of the pure social effect of communicative gaze on attentional orienting from other processes that are a combination of social and generic attentional components.
View
... This board of directors has a responsibility to create strategic objectives for the company, lead them, oversee business management, and report on its confidence to shareholders (Azeez, 2015). In addition to the above responsibilities, the board of directors is considered an important indicator for CG (Senju & Johnson, 2010). From the evidence of previous researchers, the board size is positively correlated to CG and company success (Muchemwa, Padia, & Callaghan, 2016). ...
Corporate governance and firm integrated performance: A conceptual framework
Article
Full-text available
  • May 2022
Though the corporate governance has been studied from the viewpoint of first, accounting and financial performance (Khatib & Nour, 2021; Goel, 2018; Mohamed, Basuony, & Badawi, 2013), next, marketing performance (El Fawal & Mawlawi, 2018), and finally, logistic and supply chain performance (Hernawati & Surya, 2019) in isolation, moreover, literature on the first is comparatively higher than on the other two, it is further argued that it has not been studied from the viewpoint of firm integrated performance. The purpose of this study, therefore, is to conceptualize the relationship between corporate governance and firm integrated performance. The study adopted a rigorous literature review in forming critical arguments for the theme studied. Accordingly, the study embraced rigorous a priori knowledge in building the arguments for hypotheses development. The study proposes a conceptual framework for the relationship between corporate governance and firm integrated performance which has the potential of facilitating efficient decision-making on corporate governance and firm integrated performance. The study concludes with a foundation for the theoretical basis of the relationship between corporate governance and firm integrated performance.
View
... Relatedly, face signals expected early comprise those modulating sensory exposure (e.g., Nose Wrinkler and Upper Lid Raiser; [1, 30, 31]), which, by virtue of their evolutionary and biological origins [32, 33], probably evolved as rapid behaviors to enhance their sensory advantages (e.g., rapid muscle contractions protecting the eyes, nose and mouth would provide an effective strategy for rejecting noxious contaminants). Early signals also comprise information characteristic of detectability (e.g., sudden movement, high contrast typical of danger signals [34]), which could act as salient ''attention grabbers'' [35, 36]. Thus, our models adhere to biological signaling accounts, where features of signal design such as predictability, detectability , and speed confer an adaptive advantage to both the expresser and receiver (see also [32, 37]). ...
Space-by-time manifold representation of dynamic facial expressions for emotion categorization
Article
Full-text available
  • Jun 2016
  • J VISION
Visual categorization is the brain computation that reduces high-dimensional information in the visual environment into a smaller set of meaningful categories. An important problem in visual neuroscience is to identify the visual information that the brain must represent and then use to categorize visual inputs. Here we introduce a new mathematical formalism-termed space-by-time manifold decomposition-that describes this information as a low-dimensional manifold separable in space and time. We use this decomposition to characterize the representations used by observers to categorize the six classic facial expressions of emotion (happy, surprise, fear, disgust, anger, and sad). By means of a Generative Face Grammar, we presented random dynamic facial movements on each experimental trial and used subjective human perception to identify the facial movements that correlate with each emotion category. When the random movements projected onto the categorization manifold region corresponding to one of the emotion categories, observers categorized the stimulus accordingly; otherwise they selected "other." Using this information, we determined both the Action Unit and temporal components whose linear combinations lead to reliable categorization of each emotion. In a validation experiment, we confirmed the psychological validity of the resulting space-by-time manifold representation. Finally, we demonstrated the importance of temporal sequencing for accurate emotion categorization and identified the temporal dynamics of Action Unit components that cause typical confusions between specific emotions (e.g., fear and surprise) as well as those resolving these confusions.
View
Vision-Based Approach for Autism Diagnosis Using Eye Tracking based on Machine Learning and Deep Learning
Article
Full-text available
  • Aug 2022
Although the early diagnosis of autism spectrum disorder (ASD) is highly sought, it is still a challenging task that requires a battery of cognitive tests and hours of clinical exams to be properly accomplished. ASD may manifest itself in various ways, which makes diagnosing it even more challenging. However, although diagnostic tests are generally devised by specialists, human mistakes are still possible. Computer-assisted technologies can be quite useful in this respect for aiding in the selection procedure. The use of eye tracking as a crucial component of ASD screening evaluation based on the distinctive characteristics of the eye gaze is continued in this article. This research adds to the growing body of evidence that eye tracking technologies may help in ASD screening. Eye tracking, visualization, using machine learning and deep learning are part of the suggested solution. Eye-tracking scan paths were first converted into a visual representation in the form of a series of pictures. After that, an image classification job was taught to a convolutional neural network. The findings of the experiments showed that using a visual representation simplified the diagnostic process while also achieving excellent accuracy. The convolutional neural network model, in particular, can attain promising classification accuracy. This shows that visuals are capable of encoding information about gaze movements and its underlying dynamics. Based on the greatest information coefficient, we examined potential relationships between autism severity and eye movement dynamics. The results demonstrate that using the great potential in combining eye tracking, visualization, machine learning, and deep learning to create an impartial tool to help in ASD screening. Generally, the approach we propose may be used to identify various diseases, including neurodevelopmental problems.
View
Conscience de soi et contact interindividuel : études en électrophysiologie et magnétoencéphalographie
Thesis
Full-text available
  • Jun 2018
Les situations de contact interpersonnel participeraient à la construction d’un sens basique du soi durant l’enfance, et de nos représentations de soi tout au long de la vie. Bien que l’on retrouve cette hypothèse de manière répandue dans la littérature, elle n’a été que très peu investigué expérimentalement. Cette thèse teste cet effet potentiel chez l’adulte. Deux études combinant mesures électrophysiologiques et comportementales montrent une augmentation d’une forme minimale de conscience de soi –des informations afférentes provenant du corps– suite à un contact social. Cet effet est reproduit dans 3 modalités sensorielles (contact social visuel, auditif et tactile). Une 3ème étude en magnétoencéphalographie teste l’effet d’un contexte de contact interpersonnel multisensoriel accru (vs réduit), entre un expérimentateur et des participants, sur la connectivité fonctionnelle des réseaux cérébraux dits de repos, et sur le contenu des pensées des participants. Nos résultats mettent en évidence une augmentation des processus cérébraux et cognitifs orientés sur le soi sous une forme hautement intégrée, associée à une diminution des processus sensoriels orientés sur l’environnement extérieur, à la suite d’un contact social accru. Dans l’ensemble, nos résultats suggèrent que le contact social améliore plusieurs facettes de la représentation de soi, aussi bien des aspects corporels, que des aspects plus haut-niveau et intégrés du soi narratif. Nos interactions sociales tout au long de la vie pourraient ainsi induire un contexte cérébral et cognitif centré sur un soi multifacette qui favoriserait la conscience de soi, et la construction d’un sens de l’identité incarné et situé.
View
Temporal-nasal asymmetry of rapid orienting to face-like stimuli
Article
Full-text available
  • Sep 2009
Recent work suggests that a subcortical visual route may mediate rapid orienting towards faces in the visual periphery. We now demonstrate that this orienting bias towards faces shows a temporal-nasal visual field asymmetry of responses, supporting the view that it is mediated by extrageniculate pathways. Upright schematic face-like pattern elicited faster behavioural responses than inverted one in the temporal but not in the nasal hemifield of each eye. This effect occurred for saccades but not for manual responses. The presence of a similar asymmetry of the orienting bias in newborns supports the role of extrageniculate pathways in face detection in both neonates and adults.
View
Neural Correlates of Eye Gaze Processing in the Infant Broader Autism Phenotype
Article
Full-text available
  • Feb 2009
Studies of infant siblings of children diagnosed with autism have allowed for a prospective approach to study the emergence of autism in infancy and revealed early behavioral characteristics of the broader autism phenotype. In view of previous findings of atypical eye gaze processing in children and adults with autism, the aim of this study was to examine the early autism phenotype in infant siblings of children diagnosed with autism spectrum disorder (sib-ASD), focusing on the neural correlates of direct compared with averted gaze. A group of 19 sib-ASD was compared with 17 control infants with no family history of ASD (mean age=10 months) on their response to direct versus averted gaze in static stimuli. Relative to the control group, the sib-ASD group showed prolonged latency of the occipital P400 event-related potentials component in response to direct gaze, but they did not differ in earlier components. Similarly, time-frequency analysis of high-frequency oscillatory activity in the gamma band showed group differences in response to direct gaze, where induced gamma activity was late and less persistent over the right temporal region in the sib-ASD group. This study suggests that a broader autism phenotype, which includes an atypical response to direct gaze, is manifest early in infancy.
View
Subcortical face processing
Article
  • Jan 2005
  • NAT REV NEUROSCI
View
Slow echo: Facial EMG evidence for the delay of spontaneous, but not voluntary, emotional mimicry in children with autism spectrum disorders
Article
  • Aug 2009
  • DEVELOPMENTAL SCI
Spontaneous mimicry, including that of emotional facial expressions, is important for socio-emotional skills such as empathy and communication. Those skills are often impacted in autism spectrum disorders (ASD). Successful mimicry requires not only the activation of the response, but also its appropriate speed. Yet, previous studies examined ASD differences in only response magnitude. The current study investigated timing and magnitude of spontaneous and voluntary mimicry in ASD children and matched controls using facial electromyography (EMG). First, participants viewed and recognized happy, sad, fear, anger, disgust and neutral expressions presented at different durations. Later, participants voluntarily mimicked the expressions. There were no group differences on emotion recognition and amplitude of expression-appropriate EMG activity. However, ASD participants' spontaneous, but not voluntary, mimicry activity was delayed by about 160 ms. This delay occurred across different expressions and presentation durations. We relate these findings to the literature on mirroring and temporal dynamics of social interaction.
View
Looking But Not Seeing: Atypical Visual Scanning and Recognition of Faces in 2 and 4-Year-Old Children with Autism Spectrum Disorder
Article
  • Aug 2009
  • J AUTISM DEV DISORD
This study used eye-tracking to examine visual scanning and recognition of faces by 2- and 4-year-old children with autism spectrum disorder (ASD) (N = 44) and typically developing (TD) controls (N = 30). TD toddlers at both age levels scanned and recognized faces similarly. Toddlers with ASD looked increasingly away from faces with age, atypically attended to key features of faces, and were impaired in face recognition. Deficits in recognition were associated with imbalanced attention between key facial features. This study illustrates that face processing in ASD may be affected early and become further compromised with age. We propose that deficits in face processing likely impact the effectiveness of toddlers with ASD as social partners and thus should be targeted for intervention.
View
Atypical eye contact in autism: Models, mechanisms and development
Article
  • Jul 2009
An atypical pattern of eye contact behaviour is one of the most significant symptoms of Autism Spectrum Disorder (ASD). Recent empirical advances have revealed the developmental, cognitive and neural basis of atypical eye contact behaviour in ASD. We review different models and advance a new 'fast-track modulator model'. Specifically, we propose that atypical eye contact processing in ASD originates in the lack of influence from a subcortical face and eye contact detection route, which is hypothesized to modulate eye contact processing and guide its emergent specialization during development.
View
The eye contact effect: Mechanisms and development
Article
  • Mar 2009
  • TRENDS COGN SCI
The 'eye contact effect' is the phenomenon that perceived eye contact with another human face modulates certain aspects of the concurrent and/or immediately following cognitive processing. In addition, functional imaging studies in adults have revealed that eye contact can modulate activity in structures in the social brain network, and developmental studies show evidence for preferential orienting towards, and processing of, faces with direct gaze from early in life. We review different theories of the eye contact effect and advance a 'fast-track modulator' model. Specifically, we hypothesize that perceived eye contact is initially detected by a subcortical route, which then modulates the activation of the social brain as it processes the accompanying detailed sensory information.
View
Seen Gaze-Direction Modulates Fusiform Activity and Its Coupling with Other Brain Areas during Face Processing
Article
  • Jul 2001
Gaze-contact is often a preliminary to social interaction and so constitutes a signal for the allocation of processing resources to the gazing face. We investigated how gaze direction influences face processing in an fMRI study, where seen gaze and head direction could independently be direct or deviated. Direct relative to averted gaze elicited stronger activation for faces in ventral occipitotemporal cortices around the fusiform gyrus, regardless of head orientation. Moreover, direct gaze led to greater correlation between activity in the fusiform and the amygdala, a region associated with emotional responses and stimulus saliency. By contrast, faces with averted gaze (again, regardless of head orientation) yielded increased correlation between activity in the fusiform and the intraparietal sulcus, a region associated with shifting attention to the periphery.
View
Eye contact does not facilitate detection in children with autism
Article
  • Sep 2003
  • COGNITION
Eye contact is crucial in achieving social communication. Deviant patterns of eye contact behavior are found in individuals with autism, who suffer from severe social and communicative deficits. This study used a visual oddball paradigm to investigate whether children with high functioning autism have difficulty in detecting mutual gaze under experimental conditions. The results revealed that children with autism were no better at detecting direct gaze than at detecting averted gaze, which is unlike normal children. This suggests that whereas typically developing children have the ability to detect direct gaze, children with autism do not. This might result in altered eye-contact behavior, which hampers subsequent development of social and communicative skills.
View
Abnormal activation of face processing systems at early and intermediate latency in individuals with autism spectrum disorder: A magnetoencephalographic study
Article
  • Jun 2005
The neurological basis of developmental psychopathology in autism is a matter of intense debate. Magnetoencephalography (MEG) was used to study the neuronal responses associated with the processing of faces in 12 able adults with autism spectrum disorders (ASD), performing image categorization and image identification tasks. The neuromagnetic data were analysed using nonparametric time-series analysis and equivalent current dipole estimation. Comparison data were obtained from 22 normally developing adults. In individuals with ASD, the neural responses to images of faces, observed in right extrastriate cortices at approximately 145 ms after stimulus onset, were significantly weaker, less lateralized and less affected by stimulus repetition than in control subjects. Early latency (30-60 ms) responses to face images, over right anterior temporal regions, differed significantly between the two subject groups in the image identification task. No such difference was observed for images of mugs or meaningless geometrical patterns. These findings suggest that, during the course of development in individuals with ASD, the cortical activity associated with the processing of human faces assumes a different-from-normal localization in extrastriate brain regions. This abnormal localization may be associated with unusual, but nevertheless face-specific, fast processing pathways.
View