Content uploaded by Xiaoming Jiang
Author content
All content in this area was uploaded by Xiaoming Jiang on May 03, 2022
Content may be subject to copyright.
Available via license: CC BY 3.0
Content may be subject to copyright.
1
Chapter
The Role of Multilingual Script
Systems in Face Processing
QiYang, XiaohuaCao and XiaomingJiang
Abstract
Becoming multilingual has a broad impact on cognitive abilities, especially
visual processing. An important theoretical issue is whether the acquisition of
distinct script systems affects face processing in an identical way, or, if not, how
this acquisition may exert differential impacts on face processing. By reviewing the
existing literature, we propose that Asian participants with the logographic script
system differ from Western counterparts with the alphabetic script system in view-
ing faces. The contribution of the chapter is to identify the possible role of types
of script systems in face processing mechanisms and to put forward the research
direction in the future with several new methodological efforts.
Keywords: bilingualism, script system, face processing, neural recycling hypothesis,
literacy acquisition
1. Introduction
As globalization progresses worldwide, more and more individuals have become
bilinguals or even multilingual. Multilingual differ from monolinguals in at least
two aspects. First, multilingual usually have a larger vocabulary size compared with
monolingual because these multilingual need to use words from different languages
to express the same concept. Second, multilingual may have to deal with differences
in many linguistic aspects of different languages, such as word-to-sound mapping,
phonemes and the number of letters/letters, and orthography/word forms [1]. Such
differences affect cognitive abilities (visual working memory [2], attentional control
[3]), and efficient communication [4, 5] of multilingual. However, little attention
was paid to the question of how acquiring different script systems, for example, with
different phonological transparency of orthography and different visual configura-
tions, impacts the visual perception of words, and even non-words (such as faces).
Before discussing the relationship of the acquisition of multiple script
systems with face processing, we first reviewed the link of the script system to
face processing in a single language. One important aspect of literacy acquisi-
tion is to use script systems to write and read in daily life. An interesting and
fundamental issue underlying literacy acquisition is how our brain deals with
faces. Why does literacy acquisition (in particular, what script system is learned
to read) affect face processing? One dominant view, i.e., the neural recycling
hypothesis [6], has been proposed. Since the script system is not fully formed
until 5000~6000years ago due to a recent cultural invention, unlike faces, it is
possible that our brain does not evolve in time to develop a specific cortical terri-
tory dedicated to processing words, relative to faces. This invention of the script
Multilingualism
2
system inevitably invades the pre-existing brain cortex, originally acted as other
functions (such as recognizing faces and other objects), and re-organizes these
brain structures to adapt themselves to word processing. That is, the process of
acquiring a new script system may share some neural resources with recognizing
faces. This opinion is called the neural recycling hypothesis [6], which assumes
that: first, the anatomy of connection constrained strongly by the evolutionary
pressure determines our brain organization, which, in turn, guides our subse-
quent learning. Second, learning to read must find suitable neural substrates,
which are a set of circuits close enough in their function and revealing enough
plasticity, in order to recycle a large part of the circuits for this new function.
Third, although these other-serving cortical territories are (partially) occupied
by literacy, their prior organization is never completely erased. Thus, prior neu-
ral constraints have a powerful impact on the acquisition of cultural invention
and individual brain organization. Based on this recycling process, it is reasoned
that literacy acquisition most likely has much to do with face processing.
This chapter is organized as follows. We first briefly introduced empirical
evidence with respect to the impact of literacy acquisition on linguistic and non-lin-
guistic (especially face) processing. Then, experimental evidence on cross-linguistic
and cross-cultural comparisons provided insights into the role of script systems in
processing faces, which echoes controversies about two kinds of theoretical hypoth-
eses, i.e., language specificity and cross-language universality. Moreover, multilin-
gual differences in face processing could be accounted for by different possibilities,
such as the perceptual expertise hypothesis and attention-reshaped-by-language
hypothesis. Finally, further studies are encouraged to increase the weight of the
script system in explaining face processing (or reduce the weight of social and
cultural interpretation) and to distinguish the visual form of the script system and
the role of speech in face processing.
2. Literacy effect on the processing of linguistic and non-linguistic
materials
According to the neural recycling hypothesis, literacy acquisition reshapes the
processing of linguistic and non-linguistic materials (e.g., faces). Without a doubt
literacy acquisition impacts the linguistic materials owing to that the brain struc-
tures of faces and objects are engaged in the representation of words.
2.1 Literacy effect on the processing of linguistic materials
Literacy acquisition is a milestone for human civilization. It is well-documented
that literacy acquisition modulates our ability to deal with linguistic information
[7], such as word repetition, speech segmentation, and character identification. For
example, using an auditory-verbal repetition paradigm, Petersson et al. revealed
that literates performed better than their illiterate counterparts in the pseudo-word
repetition task [8]. Also, Morais et al. showed that ex-illiterates, who attended
classes of elementary instruction during adulthood, were superior to the illiterate
in multiple speech segmentation tasks [9]. Besides, Duñabeitia et al. found that, in
two perceptual matching tasks, literates showed great sensitivity to changes in the
letter position (i.e., transposed-characters) and identity (i.e., replaced-characters)
of a character, whereas illiterates were less sensitive to these changes [10]. These
findings suggested that literacy acquisition has a profound influence on processing
different linguistic aspects, and the supportive evidence mostly is obtained in the
monolingual context.
3
The Role of Multilingual Script Systems in Face Processing
DOI: http://dx.doi.org/10.5772/intechopen.101539
2.2 The effect of literacy on the processing of non-linguistic face materials
Many studies have demonstrated that literacy acquisition modulates the ability
to process non-linguistic objects [11], especially faces. We here review a tremendous
amount of work on children, adults, and special populations.
One way of understanding the associations between literacy acquisition and face
processing is to track the development of face processing ability with the size of
literacy or to the relations between lexical and face processing in children. Evidence
from the functional magnetic resonance imaging (fMRI), a high-spatial-resolution
technique, has suggested that, during literacy acquisition in children, there may be
coordination between word and face processing in the left fusiform gyrus (FFA),
also termed visual word form area (VWFA) [12, 13]. For instance, neural responses
to faces in VWFA declined gradually with the increase in children’s letter knowledge
[14]. Consistently, event-related potentials (ERPs) studies have identified a stable
electrophysiological hallmark, N170 response, of viewing words, which was elicited
at electrodes over the left occipitotemporal areas (roughly corresponding to the
VWFA). The N170 peaks about 150–200ms after word onset, and may index the
visual processing of words at an expert level. Using the color-matching task, Li
etal. found that both the left-lateralization indexed by the N170 of words and the
vocabulary was associated positively with the right-lateralization of faces in Chinese
preschool children [15]. Similarly, employing the “half-field” paradigm, Dundas et
al. showed that the emergence of face lateralization was related positively to reading
competence with the control of age, reasoning scores, and face discrimination
accuracy [16]. Furthermore, they also reported that the N170 evoked by faces in the
right hemisphere was positively related to that by words in the left hemisphere in
American children [17]. In a word, there is a tight link between visual face process-
ing and word lateralization or between this processing and the size of literacy.
Also, empirical work from normal adults’ study provided stronger evidence of
the relationships between word and face processing. For example, using the adapta-
tion paradigm, in which the first adaptor face (or word) was followed subsequently
by a target face (or word) of either the same or different identity, Cao et al. found
that the adaptor face led to the reduced N170 response to the target word, while the
adaptor word did not result in the decreased N170 response to the target face [18].
Neural adaptation occurred because the neural response to the test stimulus was
reduced when the stimulus was preceded by a physically identical or categorically
identical adaptor stimulus. Therefore, the results of this study may indicate that the
facial N170 function partially encompasses the N170 function of word processing,
which is consistent with the neural recycling hypothesis. In another study, subjects
were asked to view artificial objects (i.e., face-like Greebles) centrally presented
with the concurrent lateral presentation of faces and then to judge which side each
face was presented. Results showed that the N170 response to faces tremendously
decreased after subjects were trained to recognize Greebles compared to before
those Greeble novice [19]. Analogously, the bilateral N170 response to faces
decreased in identifiable Chinese characters and faces as compared to unidentifi-
able conditions [20]. In Robinson et al.’s study with an attentional blink paradigm,
they found that word (target 2) recognition performance was inferior at short
inter-target lags when the word stimulus was preceded by faces compared to glasses
and words condition. No effects were observed when words were followed by other
objects. Furthermore, ERP results indicated that N170 responses to face (target 1)
were associated with the reduction of N170 to words within the face–word condi-
tion in the left hemisphere, but not for other object-word conditions [21]. These
findings indicated that face and word shared some overlapping neural resources,
possibly associated with the same specialized processing module.
Multilingualism
4
In contrast with these studies mentioned above, the special population, such as
dyslexia, alexia, prosopagnosia, illiterates, and so on, can provide stronger evidence
for the relationship between words and face processing. For example, dyslexic
readers performed poorer on recognition of both word and face and even decreased
level of hemispheric lateralization to words and faces compared with normal readers
[22]. Face recognition was impaired severely with bilateral compared to unilateral
temporo-occipital cortex lesion [23], and a left occipital impairment gave rise to both
pure alexia and prosopagnosia [24]. Also, prosopagnosic patients showed mild but
reliable words recognition deficits, and pure alexic patients showed face recognition
deficits [25]. Another important avenue is to examine the differences in face process-
ing between literates and illiterates. The ideal illiterate and literate groups differ only
in whether the script system could be used to read and write by them, so they are the
ideal group for researchers to understand the relationship between face and word
processing. For example, Deheane et al. found that reduced responses to faces in
VWFA were observed for literates compared to illiterate adults [26]. And ERP results
showed that the literacy effects were observed not only in the letter strings but also in
faces, revealing the impact of literacy on common early visual processing [27]. These
findings have led to a proposal that face and word processing engage some overlap-
ping neural substrates, and there are interactions between the development of visual
representations for faces and words [28]. However, these studies did not weigh the
potential contribution of the script system to face processing because they used the
script system of one language in a particular social culture.
3. The differences in face processing between Western and eastern
participants
Although the neural recycling hypothesis points out the possible relationships
between word and face processing in VWFA, it does not postulate whether differ-
ent script systems affect face processing in the same way, and if not, how the script
system influences face processing. There are two possible theoretical hypotheses,
the language specificity hypothesis, and cross-language universality hypothesis, to
explain the impact of literacy on face processing.
The first hypothesis holds that neural computations are functionally independent
underlying languages of different script systems. Evidence for the language specific-
ity hypothesis comes from Siok et al.’s study [29], showing that the left medial frontal
gyrus is crucial and unique to normal Chinese reading, and its dysfunction is only
linked with reading difficulty in Chinese, but not other languages. Similarly, prior
studies have consistently found declined activation in left temporoparietal regions,
which is a biological signature of English reading difficulty in a homophone judgment
task [30, 31]. Moreover, Xu et al., using a passive reading task, found that distinct
activity patterns in the middle occipital cortices, fusiform gyri, and lateral temporal,
temporoparietal, and prefrontal cortices were observed between Chinese and English
[32]. In contrast, the second hypothesis deems that VWFA consistently and equally
responds to words of different script systems. For instance, Feng et al. measured the
fMRI responses to words, faces, and houses among Chinese and French 10-year-old
children, half of them with reading difficulties. The results did not reveal any effects
of language on the peak locations and activations in the bilateral FFA [33].
These findings mainly focused on whether there are differences in neural
substrates underlying different script systems. It remains unclear whether and
how these different script systems affect face processing. However, behavioral,
eye-tracking, and neural evidence of face processing from cross-cultural studies
provided insights into this issue.
5
The Role of Multilingual Script Systems in Face Processing
DOI: http://dx.doi.org/10.5772/intechopen.101539
3.1 Behavioral evidence
Holistic face processing, a typical hallmark of perceptual expertise for faces,
refers to that participants tend to deal with face parts as a whole, rather than
as separate features [34, 35]. Numerous studies have demonstrated that Asian
adults outperformed Western adults in holistic face processing [36–39]. For
instance, Miyamoto et al. found that the Japanese performed more holistic,
rather than featural, strategies in comparison with Americans in choosing to
match the prototype faces [37]. Furthermore, Rhodes et al. found that, with the
face inversion paradigm in which there is an impaired recognition in inverted
compared to inverted faces [40], Chinese subjects exhibited a larger face
inversion effect than European counterparts. Some studies used the complete
composite face paradigm to tap the holistic face processing, in which the top and
bottom parts of two faces are constructed to form a new composite face. In this
paradigm, two factors were manipulated. The first manipulation is whether the
study face is aligned, meaning that the position of the bottom part of the face
is shifted right or left from the top part (misaligned) or not (aligned). And the
second is consistency: the consistent trials refer to which the top and bottom
parts of the study face are the same as the test face or changed simultaneously;
In inconsistent trials, the study face is different from the test face in either the
top part or the bottom part. And observers are asked to attend to the target part
(such as the top part of the face) and meanwhile to ignore other parts (such as
the bottom part of the face). Results showed that the recognition accuracy was
better in consistent than inconsistent conditions when faces were aligned, while
the consistency effect became weak or disappeared when faces were misaligned
[41]. Using the composite face paradigm, Michel et al. showed that Asians had
stronger holistic processing (indexed by the composite face effect) as compared
to Caucasians [42]. Employing the part-whole paradigm, Tanaka et al. asked
Caucasian and Asian observers to recognize facial features of Caucasian and
Asians in isolation or in the whole face, showing that Caucasians processed
own-race faces holistically compared to Asian faces, while the pattern of holistic
processing was observed for both Caucasian and Asian faces in Asians [43]. In
summary, converging evidence from different experimental paradigms reaches
a consistent agreement that Asians were superior to Westerners in the holistic
face processing.
3.2 Eye-tracking evidence
Evidence from eye-tracking studies has suggested that participants employed
distinct processing strategies to fetch visual information from faces in cross-
cultural studies. For instance, participants are instructed to learn, recognize, and
categorize faces of Western Caucasians and East Asians according to race, and
their eye movements were monitored. Results revealed that Western participants
tended to fixate on a triangular region (eyes and mouth) of faces, not affected
by facial races and tasks; While East Asian observers paid more attention to
the central region (nose) of faces [44]. Moreover, Kelly et al. asked children
aged 7–12 from the UK and China to complete an old/new face recognition task
while simultaneously recording their eye movements. The patterns of fixations
observed in children are consistent with those of adults from their respective
cultural groups reported in previous studies [44, 45], that is, children from the
UK fixated more on the eyes and mouth regions whereas children from China
fixated more centrally on the nose region. These findings distinguished different
fixation patterns for western and eastern subjects during face recognition.
Multilingualism
6
3.3 Neural evidence
Many studies have also examined differences in neural responses to faces in
observers from different countries. For example, Wang et al. investigated whether
distinct attended areas between two cultures tunes the time course of face process-
ing towards configural and featural information respectively. In this experiment,
participants were asked to judge the two concurrent faces identical, the two faces
either different in the distance between the face features (configural processing), or
in the face features (featural processing). Results showed that a configural processing
bias is associated with P1 amplitude in their own-race faces and other-race faces and a
featural processing bias is associated with P2 amplitude for own-race faces in Chinese
participants. In contrast, both a featural processing bias for their own-race faces and a
configural processing bias for other-race faces are correlated with P1 amplitude, and a
configural processing bias for both own- and other-race faces is related to P2 ampli-
tude in Western participants [46]. A recent study conducted by Ma et al. revealed that
relative to German children in the second grade, the N170 response to face is remark-
ably higher in Chinese children [47]. During an fMRI experiment conducted by Goh
et al. [48], East Asians and Westerners were asked to passively view Singaporean and
American faces and the corresponding scrambled pictures. They found that more
neural responses to faces in the bilateral FFA, especially in the left FFA, were found
in Western participants, while more neural responses to faces in the right FFA were
observed in East Asian participants. In a nutshell, there are stable and reliable neural
mechanisms of different participants underlying face processing.
The existing studies showed that differences in processing faces can be attrib-
uted to distinctions among disparate social cultures. Indeed, prior work has been
made with respect to scene perception [49, 50], description [51], and categorization
[52] in support of cultural differences. For example, Western participants paid
attention to objects with more salience in an analytic fashion and based on catego-
rization, while East Asians (e.g., Chinese, Japanese, and Korean) paid more atten-
tion to relationships and similarities among objects in a holistic fashion when they
organized their environment [44]. However, most of these studies did not try to
control the participants’ second language experience. Therefore, the cultural differ-
ences in face processing between Western and Eastern subjects may be confounded
by linguistic experiences as well.
4. Multilingual experience and face processing
A great deal of evidence has suggested that the multilingual vs. monolingual
experience has a differential impact on the processing of linguistic and non-linguistic
stimuli. For instance, the cerebral lateralization of the word [1] and face processing
[53] has been reduced in bilinguals. Regarding how multilingualism affects linguistic
and non-linguistic stimuli processing, three hypotheses were proposed to explain the
effect of multilingualism on face processing. One hypothesis is the perceptual exper-
tise hypothesis, that is, the amount of exposure to face modulated face processing [54,
55]. For example, in Canada, there are many immigrants whose children have to learn
not only English, but also their mother tongue, and even other languages. At the same
time, they are also exposed to different faces, which may make them process faces
more efficiently when viewing different faces. This chapter does not intend to spend
much time in discussing the relationship between the amount of exposure to face and
face processing given that much work with respect to this hypothesis has been done.
The second hypothesis is that attention is reshaped by visual features of the
script. According to the second hypothesis, during the process of learning words,
7
The Role of Multilingual Script Systems in Face Processing
DOI: http://dx.doi.org/10.5772/intechopen.101539
language shapes how its language user deploys the attentional resource to the visual
processing of words. For example, Awadh et al. found that French and Spanish
individuals possess lower visual attentional span than Arabic individuals [56]. This
advantage concerning the visual attention span may have been transferred into the
processing of faces. Face processing can be attentionally-driven, so it is possible to
change the outcomes and mechanisms of face processing by changing individuals’
attentional allocation [57, 58]. This hypothesis is partly supported by two recent
studies using different script systems, with individuals exposed to different systems
behaving differently in the visual processing of face tasks. For example, in the
Portuguese script system, Ventura et al. showed that illiterate participants processed
faces and houses consistently more holistically compared to literate participants
with the composite face paradigm [57]. However, in the Chinese script system,
Cao et al. found that literates had a great sensitivity to the spatial configuration of
upright, rather than inverted, faces, compared to illiterates in a second-order con-
figuration task [58]. Since social culture was kept constant in the two studies, the
inconsistent findings are difficult to be accounted for by the cultural differences,
but instead, can be explained by differences in orthographical and visual features
in the respective script systems (and possibly the experimental paradigms). On one
hand, the face composite paradigm is reflected as the failure to selectively attend
to and compare some parts of the face [41]; while the spatial configural distance
paradigm typically emphasizes that the viewer attends to the spatial relationship
between different parts of the face [59]. Distinct experimental paradigms may ease
the level of face processing in different groups of language users. Since the majority
of the previous studies demonstrated a generally consistent pattern that East Asian
participants show stronger holistic processing compared to Western counterparts,
regardless of experimental demands in a short-term task setting, it is more likely
that the long-term exposure to the different visual form of a script system (e.g.
alphabetical vs. logographic) can be the reason why the attentional allocation is
reshaped. On another hand, in terms of visual characteristics, one Chinese charac-
ter comprises strokes and sub-character components, which is packed into a square
configuration with similar size. And Chinese characters are possessed in a highly
nonlinear visual complex shape [60, 61]. Portuguese words are comprised of some
basic letters, and line-shaped. Extensive exposure to these differences in visual
features between both script systems could give rise to distinct reading demands for
Chinese in comparison with Portuguese words, which are further transferred into
face processing. Additionally, some experts have proposed that the research on the
impact of literacy acquisition on cognitive ability should always take differences
among the distinct script systems into consideration [62].
Last but not the least, the impact of multilingualism on face processing can be
traced to the effect of phonemic differences and their impact on the attentional
allocation to a speaker’s face (attention-reshaped-by-speech hypothesis). Robust
and reliable evidence comes from the comparison between monolingual and bilin-
gual infants. Infants in a bilingual environment fixated more at the mouth region
of talking and non-talking faces compared to those in a monolingual environment
[63]. Given the absence of the script exposure, these findings cannot be attributed
to the differences in visual features between writing systems; In contrast, it is more
possible that the language environment to which infants are exposed constrain
their visual processing. Using the Cambridge Face Memory Tests, Burns found that
bilingual Singaporean Chinese participants showed a decreased other race effect
with the increase of reported cross-language proficiency [64]. This relationship was
driven by Chinese, rather than English, listening ability. These findings suggested
that multilingual exhibit different processing of faces compared with monolinguals
is attributed to how words are realized in the face in communication.
Multilingualism
8
Given these remarkable differences between the logographic and alphabetic
scripts outlined above, extensive exposure to distinct script systems may result in
differences in processing faces. Combined with the existing findings and theories,
we proposed that Asian participants with logographic script system process face
different from Western counterparts with alphabetic script systems. Future studies
are encouraged to clarify these confounding explanations.
5. Future studies
Despite much evidence from the cross-cultural and cross-linguistic studies that
indicate differences in face processing, more efforts should be devoted to effectively
dissociate possibly separate contributions of social culture and script system to the
explanation of differences in face processing and to clarify how multilingualism
affects face processing. Moreover, the relationship between the script system and
face processing provides an important window into the understanding of brain
plasticity. To answer the issue, we proposed to test multilingual speakers to unveil
the impact of a distinct script systems on face processing. Several possible research
directions are put forward, such as 1) training illiterates to acquire distinct script
systems respectively in an identical culture; 2) comparing monolingual Chinese
and Korean subjects sharing the East-Asian culture; 3) comparing monolinguals
and bilinguals (or multilingual) speakers; 4) using one artificial language to train
participants to either learn the visual form or the speech in communication.
5.1 Training illiterates to learn distinct script systems respectively
One way to dissociate the contributions of social culture and script systems to
holistic processing is to train the distinct types of script systems on illiterates from an
identical social culture. Illiterates are a special population who share the social culture
(including spoken language) with literates. One important distinction between literates
and illiterates is that literates acquired the script system. Therefore, it helps to under-
stand how a script system shapes the processing of faces in a particular social culture.
Two previous studies, Ventura et al. [57] and Cao et al. [58], examined the relationships
between the acquisition of a script system and face processing in the Western and
the Eastern culture respectively. However, these studies cannot reveal the potential
contributions of the distinct script systems to face processing because of the absence
of a direct comparison. Based on the previous studies, we proposed one possible way
of segregating the role of social culture from script system in face processing, in other
words, we propose ways to heighten the weights of the script system in the explanation
of face processing, thus merely representing the influence of different script systems on
face processing after controlling social culture. More specifically, this can be achieved
by teaching Western illiterates in Chinese and English respectively or teaching Chinese
illiterates in Chinese and English respectively. Combined with a variety of experimen-
tal paradigms which tap distinct dimensions of holistic face processing, the relation-
ships between script system and face processing could be systematically investigated.
5.2 Comparing face processing mechanisms between monolinguals and
multilingual
The first suggestion is proposed against the monolingual context. With the
development of globalization, more and more individuals have become bilingual
or multilingual. It is possibly easier for bilinguals to acquire the script system of
a second language than master the social culture. Therefore, one viable way of
9
The Role of Multilingual Script Systems in Face Processing
DOI: http://dx.doi.org/10.5772/intechopen.101539
examining the relationships between distinct types of script systems and face pro-
cessing is to recruit the monolinguals and bilinguals (or multilingual) and compare
the differences between these groups. In this case, Chinese-English bilinguals (or
multilingual) and Chinese monolinguals, whose mother tongue is Chinese, or
English-Chinese bilinguals (or multilingual) and English monolinguals, whose
mother tongue is English, are recruited to decrease the possible role of social culture
in the explanation of face processing.
5.3 Comparing face processing mechanisms between Chinese and Korean
Here, we focused on the role of script or spoken form of one language in
face processing. Previous studies have shown that too many differences in face
processing existed between East Asian and Western cultures that prevent attribu-
tion of sole factor to the cultural difference in face processing. Therefore, one
way is to investigate the role of two disparate script systems in face processing in
East Asian culture. As we know, respecting the East Asian culture are people from
China, Japan, and Korea, and so on. Notably, one possibility is to explore the rela-
tionship between script systems and face processing by recruiting subjects with
different script systems in East Asia, which lies in an identical culture. A typical
way to achieve this is to compare individuals from China and South Korea. One
unique feature of this comparison is that the pronunciation of Korean resembles
that of an alphabetic language, however, the visual characteristics are closer to
Chinese [65]. Therefore, employing the identical experimental task, one could
morph the Chinese and Korean faces. By doing so, one could eliminate the impact
of the amount of exposure to face processing, and explain face processing by the
weights of speech.
5.4 Training the participants to learn words via visual or speech forms in an
artificial language
Compared with the natural language, an artificial language has some advan-
tages in studying how the script system affects face processing. For example,
the artificial language can allow researchers to train participants to learn only
the visual features of characters or only the pronunciation of characters, which
facilitates understanding of through what mechanisms the script system could
affect the face processing.
6. Conclusion
Massive evidence revealed a tight link between the acquisition of script systems
with face processing, and by reviewing prior studies, we proposed that distinct
script systems impact face processing in a different way. To clarify how the script
system affects face processing, we proposed the attention-reshaped-by-language
hypothesis. Finally, further research directions were proposed.
Acknowledgments
This study was supported by the grants from Natural Science Foundation of
China (31971037), Shanghai International Studies University Key Research Project
“An interdisciplinary study on the effect of bilingual cognitive advantage”, and
National Social Science Foundation of China (21FYYB051).
Multilingualism
10
Conflict of interest
The authors declare no conflict of interest.
Author details
QiYang1, XiaohuaCao2* and XiaomingJiang3*
1 Department of philosophy, Tongji University, Shanghai, China
2 Department of Psychology, Zhejiang Normal University, Jinhua, China
3 Institute of Linguistics, Shanghai International Studies University, Shanghai,
China
*Address all correspondence to: caoxh@zjnu.cn and xiaoming.jiang@shisu.edu.cn
© 2021 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms
of the Creative Commons Attribution License (http://creativecommons.org/licenses/
by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly cited.
11
The Role of Multilingual Script Systems in Face Processing
DOI: http://dx.doi.org/10.5772/intechopen.101539
References
[1] Lam SM, Hsiao JH. Bilingual
experience modulates hemispheric
lateralization in visual word processing.
Bilingualism: Language and cognition.
2014;17:589-609. DOI: 10.1017/
S1366728913000734
[2] Cockcroft K, Wigdorowitz M,
Liversage L. A multilingual advantage in
the components of working memory.
Bilingualism. 2017;22:15-29.
DOI:10.1017/S1366728917000475
[3] Martin-Rhee MM, Bialystok E. The
development of two types of inhibitory
control in monolingual and bilingual
children. Bilingualism. 2008;11:81-93.
DOI: 10.1017/S1366728907003227
[4] Fan SP, Liberman Z, Keysar B,
Kinzler KD. The exposure advantage:
Early exposure to a multilingual
environment promotes effective
communication. Psychological Science.
2015;26:1090-1097. DOI: 10.1177/
0956797615574699
[5] Bialystok E, Craik FI, Luk G.
Bilingualism: Consequences for mind
and brain. Trends in cognitive sciences.
2012;16:240-250. DOI: 10.1016/j.tics.
2012.03.001
[6] Dehaene S, Cohen L. Cultural
recycling of cortical maps. Neuron.
2007;56:384-398. DOI: 10.1016/j.neuron.
2007.10.004
[7] Ardila A, Bertolucci PH, Braga LW,
Castro-Caldas A, Judd T, Kosmidis MH,
et al. Illiteracy: The neuropsychology of
cognition without reading. Archives of
Clinical Neuropsychology. 2010;25:
689-712. DOI: 10.1093/arclin/acq079
[8] Petersson KM, Reis A, Askelöf S,
Castro-Caldas A, Ingvar M. Language
processing modulated by literacy: A
network analysis of verbal repetition in
literate and illiterate subjects. Journal of
Cognitive Neuroscience. 2000;12:364-
382. DOI: 10.1162/089892900562147
[9] Morais J, Bertelson P, Cary L,
Alegria J. Literacy training and speech
segmentation. Cognition. 1986;24:45-64.
DOI: 10.1016/0010-0277(86)90004-1
[10] Dunabeitia JA, Orihuela K,
Carreiras M. Orthographic coding in
illiterates and literates. Psychological
Science. 2014;25:1275-1280.
DOI:10.1177/0956797614531026
[11] Rosselli M, Ardila A. The impact of
culture and education on non-verbal
neuropsychological measurements: A
critical review. Brain and cognition.
2003;52:326-333. DOI: 10.1016/S0278-
2626(03)00170-2
[12] Dehaene S, Cohen L, Morais J,
Kolinsky R. Illiterate to literate:
Behavioural and cerebral changes
induced by reading acquisition. Nature
Reviews Neuroscience. 2015;16:234-244.
DOI: 10.1038/nrn3924
[13] Dehaene-Lambertz G, Monzalvo K,
Dehaene S. The emergence of the visual
word form: Longitudinal evolution of
category-specific ventral visual areas
during reading acquisition. PLoS biology.
2018;16:e2004103. DOI: 10.1371/journal.
pbio.2004103
[14] Cantlon JF, Pinel P, Dehaene S,
Pelphrey KA. Cortical representations of
symbols, objects, and faces are pruned
back during early childhood. Cerebral
cortex. 2011;21:191-199. DOI: 10.1093/
cercor/bhq078
[15] Li S, Lee K, Zhao J, Yang Z, He S,
Weng X. Neural competition as a
developmental process: Early hemispheric
specialization for word processing delays
specialization for face processing.
Neuropsychologia. 2013;51:950-959. DOI:
10.1016/j.neuropsychologia.2013.02.006
[16] Dundas EM, Plaut DC, Behrmann M.
The joint development of hemispheric
lateralization for words and faces.
Multilingualism
12
Journal of Experimental Psychology:
General. Journal of Experimental
Psychology General. 2013;142:348-358.
DOI: 10.1037/a0029503
[17] Dundas EM, Plaut DC,
Behrmann M. An ERP investigation of
the co-development of hemispheric
lateralization of face and word
recognition. Neuropsychologia.
2014;61:315-323. DOI: 10.1016/j.
neuropsychologia.2014.05.006
[18] Cao X, Jiang B, Gaspar C, Li C. The
overlap of neural selectivity between
faces and words: Evidences from the
N170 adaptation effect. Experimental
Brain Research. 2014;232:3015-3021.
DOI: 10.1007/s00221-014-3986-x
[19] Rossion B, Kung CC, Tarr MJ. Visual
expertise with nonface objects leads to
competition with the early perceptual
processing of faces in the human
occipitotemporal cortex. Proceedings of
the National Academy of Sciences.
2004;101:14521-14526. DOI: 10.1073/
pnas.0405613101
[20] Fan C, Chen S, Zhang L, Qi Z, Jin Y,
Wang Q, et al. N170 changes reflect
competition between faces and
identifiable characters during early visual
processing. NeuroImage. 2015;110:32-38.
DOI: 10.1016/j.neuroimage.2015.01.047
[21] Robinson AK, Plaut DC,
Behrmann M. Word and face processing
engage overlapping distributed
networks: Evidence from RSVP and EEG
investigations. Journal of Experimental
Psychology: General. 2017;146:943-961.
DOI: 10.1037/xge0000302
[22] Gabay Y, Dundas E, Plaut D,
Behrmann M. Atypical perceptual
processing of faces in developmental
dyslexia. Brain and language. 2017;173:41-
51. DOI: 10.1016/j.bandl.2017.06.004
[23] Gainotti G, Marra C. Differential
contribution of right and left temporo-
occipital and anterior temporal lesions
to face recognition disorders. Frontiers
in Human Neuroscience. 2011;5:1-11.
DOI: 10.3389/fnhum.2011.00055
[24] Liu YC, Wang AG, Yen MY. "Seeing
but not identifying": pure alexia
coincident with prosopagnosia in
occipital arteriovenous malformation.
Graefes archive for clinical &
experimental ophthalmology.
2011;249:1087-1089. DOI: 10.1007/
s00417-010-1586-4
[25] Behrmann M, Plaut DC. Bilateral
hemispheric processing of words and
faces: Evidence from word impairments
in prosopagnosia and face impairments in
pure alexia. Cerebral Cortex. 2014;24:
1102-1118. DOI: 10.1093/cercor/bhs390
[26] Dehaene S, Pegado F, Braga LW,
Ventura P, Nunes Filho G, Jobert A, et
al. How learning to read changes the
cortical networks for vision and
language. Science. 2010;330:1359-1364.
DOI: 10.1126/science.1194140
[27] Pegado F, Comerlato E, Ventura F,
Jobert A, Nakamura K, Buiatti M, et al.
Timing the impact of literacy on visual
processing. Proceedings of the National
Academy of Sciences. 2014;111:
E5233-E5242. DOI: 10.1073/
pnas.1417347111
[28] Plaut DC, Behrmann M.
Complementary neural representations
for faces and words: A computational
exploration. Cognitive Neuropsychology.
2011;28:251-275. DOI: 10.1080/026
43294.2011.609812
[29] Siok WT, Perfetti CA, Jin Z, Tan LH.
Biological abnormality of impaired
reading is constrained by culture.
Nature. 2004;431:71-76. DOI: 10.1038/
nature02865
[30] Horwitz B, Rumsey JM,
Donohue BC. Functional connectivity
of the angular gyrus in normal reading
and dyslexia. Proceedings of the
National Academy of Sciences.
13
The Role of Multilingual Script Systems in Face Processing
DOI: http://dx.doi.org/10.5772/intechopen.101539
2018;95:8939-8944. DOI: 10.1073/
pnas.95.15.8939
[31] Aylward EH, Richards TL,
Berninger VW, Nagy WE, Field KM,
Grimme AC, et al. Instructional
treatment associated with changes in
brain activation in children with dyslexia.
Neurology. 2003;61:212-219. DOI:
10.1212/01.WNL.0000068363.05974.64
[32] Xu M, Baldauf D, Chang CQ,
Desimone R, Tan LH. Distinct distributed
patterns of neural activity are associated
with two languages in the bilingual brain.
Science Advances. 2017;3:e1603309.
DOI: 10.1126/sciadv.1603309
[33] Feng X, Altarelli I, Monzalvo K,
Ding G, Ramus F, Shu H, et al. A
universal reading network and its
modulation by writing system and
reading ability in French and Chinese
children. eLife. 2020;9:e54591.
DOI:10.7554/elife.54591
[34] Richler J, Palmeri TJ, Gauthier I.
Meanings, mechanisms, and measures
of holistic processing. Frontiers in
Psychology. 2012;3:553. DOI: 10.3389/
fpsyg.2012.00553
[35] Rossion B. The composite face
illusion: A whole window into our
understanding of holistic face perception.
Visual Cognition. 2013;21:139-253.
DOI:10.1080/13506285.2013.772929
[36] Lewis RS, Goto SG, Kong LL.
Culture and context: East Asian
American and European American
differences in P3 event-related
potentials and self-construal.
Personality and Social Psychology
Bulletin. 2008;34:623-634.
DOI:10.1177/0146167207313731
[37] Miyamoto Y, Yoshikawa S,
Kitayama S. Feature and configuration
in face processing: Japanese are more
configural than Americans. Cognitive
Science. 2011;35:563-574. DOI: 10.1111/
j.1551-6709.2010.01163.x
[38] Michel C, Caldara R, Rossion B.
Same-race faces are perceived more
holistically than other-race faces. Visual
Cognition. 2006;14:55-73. DOI: 10.1080/
13506280500158761
[39] Mondloch CJ, Elms N, Maurer D,
Rhodes G, Hayward WG, Tanaka JW,
et al. Processes underlying the cross-
race effect: An investigation of holistic,
featural, and relational processing of
own-race versus other-race faces.
Perception. 2010;39:1065-1085.
DOI:10.1068/p6608
[40] Rhodes G, Brake S, Taylor K, Tan S.
Expertise and configural coding in face
recognition. British Journal of Psychology.
1989;80:313-331. DOI:10.1111/.
2044-8295.1989.tb02323
[41] Richler JJ, Tanaka JW, Brown DD,
Gauthier I. Why does selective attention
to parts fail in face processing? Journal
of Experimental Psychology: Learning,
Memory, and Cognition. 2008;34:1356-
1368. DOI: 10.1037/a0013080
[42] Michel C, Rossion B, Han J,
Chung CS, Caldara R. Holistic processing
is finely tuned for faces of one's own race.
Psychological Science. 2010;17:608-615.
DOI: 10.1111/j.1467-9280.2006.01752.x
[43] Tanaka JW, Kiefer M, Bukach CM.
A holistic account of the own-race
effect in face recognition: Evidence from
a cross-cultural study. Cognition.
2004;93:B1-B9. DOI: 10.1016/j.
cognition.2003.09.011
[44] Blais C, Jack RE, Scheepers C,
Fiset D, Caldara R. Culture shapes how
we look at faces. PLoS One. 2008;3:e3022.
DOI: 10.1371/journal.pone.0003022
[45] Kelly DJ, Liu S, Rodger H, Miellet S,
Ge L, Caldara R. Developing cultural
differences in face processing.
Developmental science. 2011;14:1176-
1184. DOI: 10.1111/j.1467-7687.2011.
0 10 6 7. x
Multilingualism
14
[46] Wang H, Qiu RLW, Li S, Fu S.
Cultural differences in the time course
of configural and featural processing for
own-race faces. Neuroscience.
2020;446:157-170. DOI: 10.1016/j.
neuroscience.2020.08.003
[47] Ma X, Kang J, Li X, Cao X,
Sommer W. The Effects of Script System
on Face Processing: A Comparison of
German and Chinese Children in
Second Grade. In Perception. September
2019. 1 OLIVERS YARD, 55 City ROAD,
London EC1Y 1SP. England: SAGE
publications LTD; 2019. pp. 211-212
[48] Goh JO, Leshikar ED, Sutton BP,
Tan JC, Sim SK, Hebrank AC, et al.
Culture differences in neural processing
of faces and houses in the ventral visual
cortex. Social Cognitive and Affective
Neuroscience. 2010;5:227-235.
DOI: 10.1093/scan/nsq060
[49] Miyamoto Y, Nisbett RE, Masuda T.
Culture and the physical environment:
Holistic versus analytic perceptual
affordances. Psychological Science.
2010;17:113-119. DOI: 10.1111/j.1467-
9280.2006.01673.x
[50] Masuda T, Nisbet RE. Attending
holistically versus analytically:
Comparing the context sensitivity of
Japanese and Americans. Journal of
personality and social psychology.
2001;81:922-934. DOI: 10.1037/0022-
3514.81.5.922
[51] Norenzayan A, Smith EE, Kim BJ,
Nisbett RE. Cultural preferences for
formal versus intuitive reasoning.
Cognitive science. 2010;26:653-684.
DOI: 10.1207/s15516709cog2605_4
[52] Norenzayan A, Smith EE, Kim BJ,
Nisbett RE. Cultural preferences for
formal versus intuitive reasoning.
Cognitive science. 2010;26(5):653-684.
DOI: 10.1207/s15516709cog2605_4
[53] Hausmann M, Durmusoglu G,
Yazgan Y, Güntürkün O. Evidence for
reduced hemispheric asymmetries in
non-verbal functions in bilinguals.
Journal of Neurolinguistics. 2004;17:
285-299. DOI: 10.1016/S0911-6044(03)
00049-6
[54] De Heering A, De Liedekerke C,
Deboni M, Rossion B. The role of
experience during childhood in shaping
the other-race effect. Developmental
science. 2010;13:181-187. DOI: 10.1111/
j.1467-7687.2009.00876.x
[55] Gauthier I, Skudlarski P, Gore JC,
Anderson AW. Expertise for cars and
birds recruits brain areas involved in
face recognition. Nature neuroscience.
2000;3:191-197. DOI: 10.1038/72140
[56] Awadh FH, Phénix T, Antzaka A,
Lallier M, Carreiras M, Valdois S. Cross-
language modulation of visual attention
span: An Arabic-French-Spanish
comparison in skilled adult readers.
Frontiers in Psychology. 2016;7: 30 7.
DOI: 10.3389/fpsyg.2016.00307
[57] Ventura P, Fernandes T, Cohen L,
Morais J, Kolinsky R, Dehaene S. Literacy
acquisition reduces the influence of
automatic holistic processing of faces and
houses. Neuroscience Letters. 2013;554:
105-109. DOI: 10.1016/j.neulet.
2013.08.068
[58] Cao X, Yang Q, Zhong P, Chen C.
The characteristics of face configural
effect in illiterates and literates. Acta
psychologica. 2019;201:102951.
DOI:10.1016/j.actpsy.2019.102951
[59] Maurer D, Le Grand R, Mondloch CJ.
The many faces of configural processing.
Trends in cognitive sciences. 2002;6:255-
260. DOI: 10.1016/S1364-6613(02)
01903-4
[60] Huang K, Itoh K, Suwazono S,
Nakada T. Electrophysiological correlates
of grapheme-phoneme conversion.
Neuroscience Letters. 2004;366:254-258.
DOI: 10.1016/j.neulet.2004.05.099
15
The Role of Multilingual Script Systems in Face Processing
DOI: http://dx.doi.org/10.5772/intechopen.101539
[61] Tan LH, Spinks JA, Eden GF,
Perfetti CA, Siok WT. Reading depends
on writing, in Chinese. Proceedings of
the National Academy of Sciences.
2005;102:8781-8785. DOI: 10.1073/
pnas.0503523102
[62] Huettig F, Mishra RK. How literacy
acquisition affects the illiterate mind–a
critical examination of theories and
evidence. Language and Linguistics
Compass. 2014;8:401-427. DOI: 10.1111/
lnc3.12092
[63] Kandel S, Burfin S, Méary D,
Ruiz-Tada E, Costa A, Pascalis O. The
impact of early bilingualism on face
recognition processes. Frontiers in
Psychology. 2016;7:1080. DOI: 10.3389/
fpsyg.2016.01080
[64] Burns EJ, Tree J, Chan AH, Xu H.
Bilingualism shapes the other race
effect. Vision Research. 2019;157:192-
201. DOI: 10.1016/j.visres.2018.07.004
[65] Wang A, Yeon J, Zhou W, Shu H,
Yan M. Cross-language parafoveal
semantic processing: Evidence from
Korean–Chinese bilinguals. Psychonomic
Bulletin & Review. 2016;23(1):285-290.
DOI: 10.3758/s13423-015-0876-6
