ArticlePDF Available

Harnessing fast periodic visual stimulation to study face cognition: Sub‐processes, brain‐behavior relationships, and objectivity

Authors:

Abstract

Rossion et al. (2020) review over a decade of work investigating the neural basis of unfamiliar face individuation (FI) ‐ the brain’s ability to distinguish unfamiliar face identity ‐ using fast periodic visual stimulation (FPVS). Though FPVS measures rapid, automatic processing, its value for studying vision and face cognition could be increased by addressing three important aspects.
This article has been accepted for publication and undergone full peer review but has not been
through the copyediting, typesetting, pagination and proofreading process, which may lead to
differences between this version and the Version of Record. Please cite this article as doi:
10.1111/EJN.15115
This article is protected by copyright. All rights reserved
DR. MEIKE RAMON (Orcid ID : 0000-0001-5753-5493)
Article type : Featured Paper Commentary
Corresponding author mail id: meike.ramon@gmail.com
Harnessing fast periodic visual stimulation to study face cognition:
sub-processes, brain-behavior relationships, and objectivity
Jeffrey D. Nador & Meike Ramon
Applied Face Cognition Lab, Department of Psychology, University of Fribourg, Fribourg, Switzerland
Key words: EEG; FPVS; Face cognition; Face individuation; Methodology
Corresponding author:
Meike Ramon
University of Fribourg
Applied Face Cognition Lab
Department of Psychology
Faucigny 2
1700 Fribourg
Switzerland
Accepted Article
This article is protected by copyright. All rights reserved
1Rossion et al. (2020) review over a decade of work investigating the neural basis of unfamiliar
2face individuation (FI) - the brain’s ability to distinguish unfamiliar face identity - using fast
3periodic visual stimulation (FPVS). Though FPVS measures rapid, automatic processing, its value
4for studying vision and face cognition could be increased by addressing three important aspects.
5Processing levels. Most FPVS studies of face cognition to date have focussed on face vs.
6object categorization or FI, and collectively support the ability of FPVS to assess face cognition
7sub-processes of primarily visual nature (i.e., detection, discrimination; Ramon & Gobbini, 2018).
8However, FI critically contributes to further sub-processes (Rossion et al., 2020), i.e. recognition
9and identification, which involve a mnemonic component investigated in only few published
10 studies (Campbell et al 2020; Verosky et al., 2020; Yan et al., 2020; Zimmerman et al., 2019).
11 Thus, systematic characterization of the relationships between FPVS signatures of different
12 processing levels (cf. Quek et al., 2020), and their modulations via task demands and stimulus
13 predictability (cf. Ramon, 2018; Ramon et al., 2019) require further scrutiny to better inform our
14 conceptualization of brain-behavior relationships in this domain. Such research will require
15 increased paradigmatic flexibility, in order to adapt the FPVS methodology to this end.
16 Brain-behavior relationships. Studies measuring the FI response have contrasted
17 neurotypical and atypical/impaired populations to characterize the mechanisms of face cognition
18 and development of FI. Among neurotypical observers, findings are limited to modest correlations
19 between FI response amplitude and behavioral test performance (Xu et al., 2017; Dzhelyova et al.,
20 2020; Rossion et al., 2020). However, it is questionable whether these behavioral tests possess
21 the virtues [...] for adequately measuring FI” (Rossion et al., 2020). The tests considered include
22 the Cambridge Face Memory Test (CFMT; Duchaine & Nakayama, 2006) and Benton Face
23 Recognition Test (BFRT; Benton & Van Allen, 1968). Since both were designed to distinguish
24 typical from impaired populations, their usefulness in investigating brain-behavior relationships
25 amongst neurotypical individuals is questionable. This aligns with the reasoning that previously
26 observed significant, weak correlations may be driven by low performing individuals (Rossion et
27 al., 2020). Moreover, by relying on simultaneous matching only (BFRT), or perception and
28 recognition (CFMT), performance on these tests relies, to a nontrivial extent, on
29 cognitive/perceptual abilities other than specifically FI.
30 Given the importance of describing the relationship between FI response and behavior, we
31 advance two methodological alternatives to abandoning this pursuit. Firstly, correlations should be
32 performed between the FI FPVS response and performance for behavioral tests that are
Accepted Article
This article is protected by copyright. All rights reserved
1conceptually closer to FI, avoiding the pitfalls introduced by differing sources of variability
2between tests. Secondly, the approach to investigating FI-behavior correspondence should be
3reconsidered by improving the flexibility of the FPVS paradigm. Given the FI response’s
4reliability (Dzhelyova et al., 2019, Stacchi et al., 2019) and fundamental role for recognition and
5identification, it could be used as a reference to probe the value of a given behavioral test (or
6performance measure), by control or systematic variation of task/stimulus factors during FPVS.
7Characterizing the relationships between the objective FI response and different tests of face
8cognition could provide an objective means to ascertain differential test validity, and behavioral
9specificity.
10 Objectivity. Notwithstanding the FI response’s advantages, there are two aspects related to
11 the FPVS applications advocated by Rossion et al. (2020) that limit its adaptability to research on
12 individual differences, and its objectivity in capturing these. First, the inflexibility of eliminating
13 the contributions of other cognitive processes to the FI response hampers the advancement of
14 theoretically based questions regarding the relationship between sub-processes involved in face
15 cognition, their neural basis, and the degree to which these vary inter-individually and are
16 differentially modulated. The authors’ suggested procedure for measuring individual differences
17 involves taking a unitary FI measure, and correlating it with a range of behaviors or participant
18 attributes. We suggest that it is time for an agnostic approach to investigating FI response
19 modulation, employing systematic variations of task- and stimulus-related features to provide
20 theoretically important insights into how the FI process unfolds across individuals.
21 The second aspect concerns agnostic electrode selection. Objectivity in psychological
22 research generally implies maximizing available degrees of freedom, while limiting controlled
23 task- and stimulus-related attributes to those necessary for eliminating potential confounds.
24 Rossion et al. (2020) suggest reducing the degrees of freedom in FI response analysis by
25 downsampling to occipital, temporal and parietal electrodes. Yet, this practice can inflate false-
26 positive statistical inferences, by homogenizing them based on potentially circular selection
27 criteria (Kriegeskorte et al., 2009; Simons, 2011). Conducting analyses as agnostically as possible,
28 with respect to the spatial distribution of the FI response, can maximize the available degrees of
29 freedom, for instance by retaining all channels in analyses or conducting whole-brain analyses
30 (Kriegeskorte et al., 2009; Simmons, et al., 2011; Veldkamp et al., 2017; Stacchi et al., 2019),
31 rather than focusing on a selection of occipito-temporal ROIs.
Accepted Article
This article is protected by copyright. All rights reserved
1We appreciate the reasoning that “fluctuations of amplitude outside of the ROIs associated
2with the maximal response may be essentially due to signal leakage, which is problematic for
3MVPA”. However, we contest the view that limited reliability observed for “outside-ROIs” should
4be interpreted as “no evidence that they could account for significant modulations of the FI
5response” (Rossion et al., 2020). In our opinion, combining systematically varied task- and
6stimulus-related attributes with electrode-agnostic analyses could unveil important individual
7differences in the FI response’s spatial distribution. This is particularly important if cognitive
8processes involved in or contributing to the FI response occur at different cortical loci.
9Expanding on the volume of previous work, we anticipate that with increasing,
10 independent usage of FPVS, individual differences in FI responses could provide crucial insights
11 into brain-behavior relationships. Systematically characterizing the relationship between behavior
12 and (task/stimulus related modulation) of FI with a more agnostic view of its spatial distribution,
13 we aim to examine the sub-processes of face cognition holistically - in the same way that FI is
14 conceptualized to proceed.
15 ACKNOWLEDGEMENT
16 MR is supported by a Swiss National Science Foundation PRIMA (Promoting Women in
17 Academia) grant (PR00P1_179872).
18 CONFLICT OF INTEREST
19 The authors declare no conflicts of interest.
20 ORCID
21 Jeffrey Daniel Nador: https://orcid.org/0000-0002-6502-7226
22 Meike Ramon: https://orcid.org/0000-0001-5753-5493
23 REFERENCES
24 Benton, A. L., & Van Allen, M. W. (1968). Impairment in facial recognition in patients with
25 cerebral disease. Transactions of the American Neurological Association, 93, 38–42.
26 Campbell A, Louw R, Michniak E, Tanaka JW (2020). Identity-specific neural responses to three
27 categories of face familiarity (own, friend, stranger) using fast periodic visual stimulation.
28 Neuropsychologia, 141, 107415.
29 Duchaine, B.C., & Nakayama, K. (2006). The Cambridge face memory test: Results for
30 neurologically intact individuals and an investigation of its validity using inverted face stimuli
31 and prosopagnosic participants. Neuropsychologia, 44, 576-585.
Accepted Article
This article is protected by copyright. All rights reserved
1Dzhelyova, M. Schiltz, C., & Rossion, B. (2020). The relationship between the benton face
2recognition test and electrophysiological unfamiliar face individuation response as revealed
3by fast periodic visual Stimulation. I-Perception.
4Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S., & Baker, C. I. (2009). Circular analysis in
5systems neuroscience: the dangers of double dipping. Nature neuroscience, 12(5), 535.
6Quek GL, Rossion B & Liu-Shuang J (2020). Critical information thresholds underlying
7concurrent face recognition functions. bioRxiv 2020.06.22.163584; doi:
8https://doi.org/10.1101/2020.06.22.163584
9Ramon M (2018). The power of how – lessons learned from neuropsychology and face processing.
10 Cognitive Neuropsychology, 35, 83-6.
11 Ramon M, Sokhn N & Caldara R (2019). Decisional space modulates visual categorization –
12 evidence from saccadic reaction times. Cognition, 186, 42-9.
13 Ramon, M., & Gobbini, M. I. (2018). Familiarity matters: A review on prioritized processing of
14 personally familiar faces. Visual Cognition, 26(3), 179–195.
15 Rossion B, Retter TL, Liu-Shuang J (2020). Understanding human individuation of unfamiliar
16 faces with oddball fast periodic visual stimulation and electroencephalography. Eur J
17 Neurosci, 10.1111/ejn.14865.
18 Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed
19 flexibility in data collection and analysis allows presenting anything as significant.
20 Psychological science, 22(11), 1359-1366.
21 Stacchi, L., Liu-Shuang, J., Ramon, M., & Caldara, R. (2019). Reliability of individual differences
22 in neural face identity discrimination. NeuroImage, 189, 468–475.
23 Veldkamp, C. L. S., Bakker, M., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H.,
24 Soderberg, C. K., & Wicherts, J. M. (2017). Restriction of opportunistic use of researcher
25 degrees of freedom in pre-registrations on the Open Science Framework. Preprint retrieved
26 from https://psyarxiv. com/g8cjq.
27 Verosky, S.C., Zoner, K.A., Marble, C.W., Sammon, M.M., & Babarinsa, C.O. (2020).
28 Familiarization increases face individuation measured with fast periodic visual stimulation.
29 Biol Psychol. 153:107883.
30 Yan X, Zimmermann FG, Rossion B (2020). An implicit neural familiar face identity recognition
31 response across widely variable natural views in the human brain. Cogn Neurosci, 11, 143-
32 156.
Accepted Article
This article is protected by copyright. All rights reserved
1Zimmermann FGS, Yan X, Rossion B (2019). An objective, sensitive and ecologically valid
2neural measure of rapid human individual face recognition. R Soc Open Sci, 6, 181904.
Accepted Article
... Fig. 3). Crucially, individuals' performances across independent tests should never be estimated by simple raw score summation (see Nador and Ramon, 2021). ...
... Going one step further, if the goal is to characterize differences between typical and high-performing Super-Recognizers (Ramon, 2021), the analytical focus has yet again to change. For instance, in some scenarios, inter-observer differences in ability may be reflected more accurately via measured differences in performance consistency (e.g., Nador et al., 2021). It is clear that such questions may require the development of novel analytical procedures (e.g., Nador et al., 2021b). ...
Article
In recent years, the number of face identity matching tests in circulation has grown considerably and these are being increasingly utilized to study individual differences in face cognition. Although many of these tests were designed for testing typical observers, recent studies have begun to utilize general-purpose tests for studying specific, atypical populations (e.g., super-recognizers and individuals with prosopagnosia). In this study, we examined the capacity of four tests requiring binary face-matching decisions to study individual differences between healthy observers. Uniquely, we used performance of the patient PS (Rossion, 2018), a well-documented case of acquired prosopagnosia (AP), as a benchmark. Two main findings emerged: (i) PS could exhibit typical rates of accuracy in all tests; (ii) compared to age-matched controls and when considering both accuracy and speed to account for potential trade-offs, only the KFMT — but not the EFCT, PICT or GFMT — was able to detect PS’s severe impairment. These findings reflect the importance of considering both accuracy and response times to measure individual differences in face matching, and the need for comparing tests in terms of their sensitivity, when used as a measure of human cognition and brain functioning.
Article
Full-text available
People are better at recognizing familiar versus unfamiliar faces. The current study investigated whether familiarity would lead to a larger face individuation response as measured via fast periodic visual stimulation (FPVS). While electroencephalography (EEG) was recorded, participants viewed oddball sequences of faces made up of more versus less familiarized faces. In each sequence, a single base face was repeated at a rate of 6 Hz and oddball faces with different identities were presented every fifth face (6 Hz/5 = 1.2 Hz). As in previous studies, significant face individuation responses were observed at 1.2 Hz and its harmonics, with the strongest responses located over right occipito-temporal electrode sites. Despite a relatively minimal learning manipulation, the face individuation response over right occipito-temporal sites was stronger for more versus less familiarized faces. These results suggest that the fast-periodic visual oddball paradigm offers a promising means for investigating face learning.
Article
Full-text available
Highly variable natural images of the same familiar face celebrity interleaved periodically in a rapid (6 images/second) train of unfamiliar faces automatically elicit an objective electroencephalographic (EEG) response over the occipito-temporal cortex of neurotypical human adults within a few minutes. However, the extent to which this frequency-tagged response goes beyond the association of common physical features of the periodically repeated face identity remains unknown. Here we compare participants who know or do not know the very same periodically repeated face celebrity and show that long-term familiarity accounts for about 80% of the neural face identity recognition response. This familiarity advantage disappears with upside-down images. Variability in response amplitude between face identities is preserved for inverted faces and in unfamiliar participants, suggesting a contribution of within-person physical face variability and distinctiveness to about 20% of the face identity response. These observations provide the strongest difference to date in human brain response between the same famous face identities perceived as familiar or unfamiliar in an implicit task. The frequency-tagged neural response largely reflects the strengthening effect of long-term memory in the human occipito-temporal cortex, and may serve to index automatic familiar face identity recognition in individual observers.
Article
Full-text available
Humans may be the only species able to rapidly and automatically recognize a familiar face identity in a crowd of unfamiliar faces, an important social skill. Here, by combining electroencephalography (EEG) and fast periodic visual stimulation (FPVS), we introduce an ecologically valid, objective and sensitive neural measure of this human individual face recognition function. Natural images of various unfamiliar faces are presented at a fast rate of 6 Hz, allowing one fixation per face, with variable natural images of a highly familiar face identity, a celebrity, appearing every seven images (0.86 Hz). Following a few minutes of stimulation, a high signal-to-noise ratio neural response reflecting the generalized discrimination of the familiar face identity from unfamiliar faces is observed over the occipito-temporal cortex at 0.86 Hz and harmonics. When face images are presented upside-down, the individual familiar face recognition response is negligible, being reduced by a factor of 5 over occipito-temporal regions. Differences in the magnitude of the individual face recognition response across different familiar face identities suggest that factors such as exposure, within-person variability and distinctiveness mediate this response. Our findings of a biological marker for fast and automatic recognition of individual familiar faces with ecological stimuli open an avenue for understanding this function, its development and neural basis in neurotypical individual brains along with its pathology. This should also have implications for the use of facial recognition measures in forensic science.
Article
To investigate face individuation (FI), a critical brain function in the human species, an oddball fast periodic visual stimulation (FPVS) approach was recently introduced (Liu‐Shuang et al., 2014). In this paradigm, an image of an unfamiliar “base” facial identity is repeated at a rapid rate F (e.g., 6 Hz) and different unfamiliar “oddball” facial identities are inserted every nth item, at a F/n rate (e.g., every 5th item, 1.2 Hz). This stimulation elicits FI responses at F/n and its harmonics (2F/n , 3F/n, etc.), reflecting neural discrimination between oddball vs. base facial identities, which is quantified in the frequency‐domain of the electroencephalogram (EEG). This paradigm, used in 20 published studies, demonstrates substantial advantages for measuring FI in terms of validity, objectivity, reliability, and sensitivity. Human intracerebral recordings suggest that this FI response originates from neural populations in the lateral inferior occipital and fusiform gyri, with a right hemispheric dominance consistent with the localization of brain lesions specifically affecting facial identity recognition (prosopagnosia). Here we summarize the contributions of the oddball FPVS framework towards understanding FI, including its (a)typical development, with early studies supporting the application of this technique to clinical testing (e.g., Autism Spectrum Disorder). This review also includes an in‐depth analysis of the paradigm’s methodology, with guidelines for designing future studies. A large‐scale group analysis compiling data across 130 observers provides insights into the oddball FPVS FI response properties. Overall, we recommend the oddball FPVS paradigm as an alternative approach to behavioral or traditional event‐related‐potential EEG measures of face individuation.
Article
Previous studies have focused on the modulatory effects of face familiarity on different components of an event-related potential (ERP), but there is controversy in the literature regarding the precise component that reflects the process of identity recognition. This may be partly explained by limits to this waveform analysis approach, as waveforms elicited by the presentation of a face are likely to reflect a variety of different cognitive processes that overlap in time. Using fast periodic visual stimulation and EEG (FPVS-EEG), we directly measured the electrophysiological response reflecting identity-specific recognition after isolating it from responses attributable to low-level visual processing and face-selective processes that are not identity-specific. The observed response therefore provides a robust and objective measure of the recognition of a personally familiar face generated bilaterally in the occipito-temporal region. We tested the magnitude of this identity-specific response to three categories of familiarity: the own-face (high familiarity), a friend's face (moderate familiarity), and a stranger's face (no familiarity). We found the largest response to the participant's own-face, followed by an intermediate response to a highly personally familiar face, and the smallest response to an unfamiliar face. An additional response was observed over the posterior cortical midline for familiar faces only, consistent with theories that familiar identity recognition also triggers post-perceptual semantic processing.
Article
A recent approach to implicitly study face recognition skills has been the fast periodic visual stimulation (FPVS) coupled with electroencephalography (EEG). Its relationship with explicit behavioral measures of face individuation remains largely undocumented. We evaluated the relationship of the FPVS–EEG measure of individuation and performance at a computer version of the Benton Face Recognition Test. High-density EEG was recorded in 32 participants presented with an unfamiliar face at a rate of 6 Hz (F) for 60 s. Every five faces, new identities were inserted. The resulting 1.2 Hz (F/5) EEG response and its harmonics objectively indexed rapid individuation of unfamiliar faces. The robust individuation response, observed over occipitotemporal sites, was significantly correlated with speed, but not accuracy rate of the computer version of the Benton Face Recognition Test. This effect was driven by a few individuals who were particularly slow at the behavioral test and also showed the lowest face individuation response. These results highlight the importance of considering the time taken to recognize a face, as a complementary to accuracy rate variable, providing valuable information about one’s recognition skills. Overall, these observations strengthen the diagnostic value of FPVS–EEG as an objective and rapid flag for specific difficulties at individual face recognition in the human population.
Article
Manual and saccadic reaction times (SRTs) have been used to determine the minimum time required for different types of visual categorizations. Such studies have demonstrated extremely rapid detection of faces within natural scenes, whereas increasingly complex decisions (i.e. levels of processing) require longer processing times. We reasoned that visual categorization speed is not only dependent on the processing level, but is further affected by decisional space constraints. In the context of two different tasks, observers performed choice saccades towards female (gender categorization) or personally familiar (familiarity categorization) faces. Additionally, familiarity categorizations were completed with stimulus sets that differed in the number of individuals presented (3 vs. 7 identities) to investigate the effect of decisional space constraints. We observe an inverse relationship between visual categorization proficiency and decisional space. Observers were most accurate for categorization of gender, which could be achieved in as little as 140ms. Categorization of highly predictable targets was more error-prone and required an additional ~100ms processing time. Our findings add to increasing evidence that pre-activation of identity-information can modulate early visual processing in a top-down manner. They also emphasize the importance of considering procedural aspects as well as terminology when aiming to characterize cognitive processes.
Article
Over the past years, much interest has been devoted to understanding how individuals differ in their ability to process face identity. Fast periodic visual stimulation (FPVS) is a promising technique to obtain objective and highly sensitive neural correlates of face processing across various populations, from infants to neuropsychological patients. Here we use FPVS to investigate how neural face identity discrimination varies in amplitude and topography across observers. To ascertain more detailed inter-individual differences, we parametrically manipulated the visual input fixated by observers across ten viewing positions (VPs). Specifically, we determined the inter-session reliability of VP-dependent neural face discrimination responses both across and within observers (6-month inter-session interval). All observers exhibited idiosyncratic VP-dependent neural response patterns, with reliable individual differences in terms of response amplitude for the majority of VPs. Importantly, the topographical reliability varied across VPs and observers, the majority of which exhibited reliable responses only for specific VPs. Crucially, this topographical reliability was positively correlated with the response magnitude over occipito-temporal regions: observers with stronger responses also displayed more reliable response topographies. Our data extend previous findings of idiosyncrasies in visuo-perceptual processing. They highlight the need to consider intra-individual neural response reliability, in order to better understand the functional role(s) and underlying basis of such inter-individual differences. LINK TO PAPER: https://www.sciencedirect.com/science/article/pii/S1053811919300230?via%3Dihub
Article
In their review, Geskin and Behrmann address the longstanding question of whether congenital prosopagnosia (CP) is a face-specific or domain-general disorder. The authors review CP cases reported over the past 40 years, categorizing them according to capacities assessed (face, object processing), and measures considered (accuracy, reaction times). The authors succeed in integrating an impressive volume of data in this succinct and important review. Based on the behavioural profiles reported for several hundred CP cases, they conclude that dissociations of face and object processing deficits are the exception, rather than the rule. Their review emphasizes the lack of consistency in cognitive assessment in the CP field and the expressed need for a consensus regarding formal diagnostic criteria. Aiming to advance our understanding of the behavioural and neural mechanisms governing face cognition, an evaluation of diagnostic criteria should also carefully consider (at least) three additional important aspects discussed below: processes, paradigms, and procedures.
Article
In this review, we synthesize the existing literature investigating personally familiar face processing and highlight the remarkable, enhanced processing efficiency resulting from real-life experience. Highly learned identity-specific visual and semantic information associated with personally familiar face representations facilitates detection, recognition of identity and social cues, and activation of person knowledge. These optimizations afford qualitatively different processing of personally familiar as compared to unfamiliar faces, which manifests on both the behavioural and neural level. (Access the pdf of this paper here: https://www.tandfonline.com/eprint/tV4Q77TzdikFavPThjHr/full)