Available via license: CC BY 4.0
Content may be subject to copyright.
ORIGINAL RESEARCH
published: 16 November 2020
doi: 10.3389/fneur.2020.597790
Frontiers in Neurology | www.frontiersin.org 1November 2020 | Volume 11 | Article 597790
Edited by:
Maria Pia Bucci,
Centre National de la Recherche
Scientifique (CNRS), France
Reviewed by:
Lori-Ann Rosalind Sacrey,
University of Alberta, Canada
Carmela Bravaccio,
University of Naples Federico II, Italy
*Correspondence:
Josephine Barbaro
j.barbaro@latrobe.edu.au
Specialty section:
This article was submitted to
Pediatric Neurology,
a section of the journal
Frontiers in Neurology
Received: 22 August 2020
Accepted: 05 October 2020
Published: 16 November 2020
Citation:
Barbaro J, Wang C, Wang J, Liu G,
Liang Y, Wang J, Abdullahi I and
Dissanayake C (2020) A Pilot
Investigation of the Social Attention
and Communication Surveillance
(SACS) Tool for the Early Identification
of Autism in Tianjin, China (SACS-C).
Front. Neurol. 11:597790.
doi: 10.3389/fneur.2020.597790
A Pilot Investigation of the Social
Attention and Communication
Surveillance (SACS) Tool for the Early
Identification of Autism in Tianjin,
China (SACS-C)
Josephine Barbaro 1
*, Chongying Wang 2, Jing Wang 3, Gongshu Liu 3, Ying Liang 3,
Ji Wang 1,4,5 , Ifrah Abdullahi 1and Cheryl Dissanayake 1
1Olga Tennison Autism Research Centre, School of Psychology and Public Health, La Trobe University, Melbourne, VIC,
Australia, 2Department of Social Psychology, Zhou Enlai School of Government, Nankai University, Tianjin, China, 3Tianjin
Women and Children’s Health Centre, Tianjin, China, 4Yangzhou Maternal and Child Health Hospital, Tianjin, China, 5Harbin
Children’s Hospital, Harbin, China
Introduction: Autism spectrum disorder (ASD) comprises difficulties in social
communication and restrictive and repetitive behaviors. Despite an increased global
prevalence, little remains known about early detection and diagnosis of autism in
Mainland China. Our aim was to conduct a pilot investigation of the implementation of
an Australian tool, Social Attention and Communication Surveillance (SACS), in Tianjin,
China (SACS-C) by trained professionals to identify autism early compared to the
Checklist for Autism in Toddlers-23 (CHAT-23) completed by parents and professionals.
Materials and Methods: A total of 10,514 children were monitored across 61
Community Health Service Centres in six Tianjin districts on the SACS-C at 12, 18,
and 24 months of age following a half-day training of 225 child health practitioners.
Children deemed at “high likelihood” for autism on either the SACS, CHAT-23, or both,
were referred for developmental assessments at the Tianjin Women and Children’s Health
Centre (TWCHC).
Results: A total of 87 children (0.8%) were identified at “high likelihood” on the SACS-C,
of whom 57 (66%) were assessed for autism; 24 children were subsequently diagnosed
with autism (42.1%), and the remaining 33 (57.9%) were diagnosed with developmental
and/or language delays. The SACS-C had a positive predictive value (PPV) of 42.1%,
a negative predictive value (NPV) of 99.8%, and sensitivity and specificity of 53.3 and
99.7%, respectively. Only 21 children were identified at “high risk” for autism on the
CHAT-23 (0.2%), over four times fewer children than the SACS-C, with 14 children
assessed for autism (66%); nine were diagnosed with autism (64.3%) and the remaining
five children were diagnosed with developmental and/or language delays. The CHAT-23
had an overall PPV of 64.3%, NPV of 99.6%, sensitivity of 27.3%, and specificity
of 99.9%.
Conclusion: This was the first large-scale study identifying autism in 12–24-month-old
children in China. We ascertained the feasibility of training community health practitioners
Barbaro et al. SACS Tool in China
to monitor infants and toddlers for the early signs of autism, and determined the
effectiveness of their use of SACS-C which had a better balance between accuracy and
sensitivity in detecting autism in contrast to the CHAT-23 which missed the majority of
children with autism (72.7%) vs. the SACS-C (46.7%). Given the emphasis on identifying
as many children with autism as possible in Mainland China, SACS-C was identified
as the tool of choice by the TWCHC. However, more work is needed to improve the
psychometric properties in using the SACS-C in Mainland China so that it is comparable
to its use in Australia.
Keywords: early detection, screening, autism spectrum disorder, developmental surveillance tool, China
INTRODUCTION
Autism spectrum disorder (ASD) comprises significant
difficulties in social attention, communication and the presence
of sensory and restrictive and repetitive behaviors (1). Early
developmental surveillance and screening plays a vital role
in the early identification, detection and diagnosis of autism,
which allows access to early intervention, leading to better child
outcomes and improved quality of life (2–5). In Mainland China,
early clinical manifestations and symptoms of autism are not
widely recognized, often being described as the “lonely disease”
(6). In 1982, Dr. Tao Guotai from Nanjing Brain Hospital
reported the first four cases of autism in Mainland China (6).
Increasing numbers of children are now diagnosed with autism
in China, particularly following the improved knowledge and
awareness about this condition (7).
The prevalence of autism in the US was recently reported to
be 1 in 54 children aged 8 years (8), whilst in the UK it is 1
in 64 (9), Australia 1 in 70 (10), and 1 in 38 in South Korea
(11). There remains limited knowledge about the prevalence of
autism in Mainland China. A meta-analysis of 18 studies found
a wide range in prevalence rates from 2.8 to 30.4 per 10,000,
with the pooled prevalence of autism being 12.8 per 10,000 (95%
CI: 9.4–17.5) (12), much less than that reported above. More
recently a meta-analysis in 2018 found a pooled ASD prevalence
of 39.2 per 10,000 (95% CIL 28.4–50.0) and specific prevalence of
autism as 10.2 per 10,000 (95% CI: 8.5–11.9) (13). Furthermore, a
2019 study used the Childhood Autism Spectrum Test (CAST)
screening tool to ascertain autism prevalence in the Chinese
cities of Jilian City, Shenzhen City, and Jiamusi City, finding that
autism prevalence estimates were similar to Western prevalence
rates in Jilian City (1.08%; 108 per 10,000) but lower in Shenzhen
City and Jiamusi City with rates of 0.42% (42 per 10,000) and
0.19% (19 per 10,000), respectively (14).
Sun et al. (15) found the strongest determinant of the
variability in prevalence estimates was the screening tool used,
and found that studies using the Autism Behavior Checklist
(ABC) (16), and the Clancy Autism Behavior Scale (CABS) (17),
produced lower prevalence estimates, whilst studies that used the
Checklist for Autism in Toddlers (CHAT) (18) reported higher
prevalence estimates for autism (15). The authors also noted age
at screening as another strong determinant in the prevalence
estimates. Fifteen of the 18 studies focused on children screened
between the ages of 2–6 years and a further seven focused on
children aged 6–14 years and older. Whilst most of the studies
included in the systematic review were young children, mean age
at diagnosis for children in Mainland China was not reported
(15). However, a recent study did report the mean age at diagnosis
in Mainland China, with an average age at diagnosis being 3.3
years for Chinese children aged 6–14 years of age (19).
The CHAT (18) and its modified versions (M-CHAT, CHAT-
23) are frequently used screening tools in Chinese populations in
Mainland China (15,20). The CHAT is more rigid with a specific
applicable age of 18-months; it is a nine-item questionnaire
for parents and contains five child observations by professional
(18). It has since been further validated, evaluated and modified
into the M-CHAT (21), and CHAT-23 versions, with the latter
designed for Chinese children (22). CHAT-23 has a validity and
reliability scores of 94 and 88%, respectively (23), as well as
a sensitivity and specificity of 93 and 85% (22). Despite these
seemingly high sensitivities and specificities reported, the age
groups screened were wide (or unclear), and none of these
studies were exclusively conducted in low-risk, community-
based populations. There is, therefore, a gap in the literature
on developmental surveillance and community-level screening
procedures for infants and toddlers in the general population in
China. There is also a lack of professional education available
to Chinese primary-care professionals on the early signs of
autism (24).
A recent systematic review jointly undertaken by Australian
and Chinese scholars (24) reported that whilst screening tools
currently used in China have reasonable psychometric properties
for identifying autism in clinical populations, there appear to
be no studies undertaken with community-based samples. They
stressed the need to align the screening and diagnostic systems
in Mainland China with best practice in autism screening and
diagnosis (25–29). Prioritizing the need for community-based
screening in the general population with psychometrically and
culturally validated tools is needed together with follow-up of
children deemed at high-likelihood of autism at the community
level so that they are assessed and diagnosed by a specialized
multidisciplinary clinical team (24).
A developmental surveillance tool designed for use in
low-risk populations within community-based settings is the
Social Attention and Communication Surveillance (SACS) tool.
Designed and implemented in Australia, this tool has an excellent
Positive Predictive Value (PPV; 81–83%), Negative Predictive
Value (99%), Sensitivity (82–84%), and Specificity (99–99.5%) for
identifying children with autism between 11 and 30 months of
age (26,30). Moreover, following diagnosis at age 24 months
Frontiers in Neurology | www.frontiersin.org 2November 2020 | Volume 11 | Article 597790
Barbaro et al. SACS Tool in China
using gold standard tools, diagnostic stability is high at 88.3%
between 24 and 48 months of age (25). On the strength of
these findings, the SACS tool has been implemented state-wide
throughout the universal Victorian and Tasmanian Maternal and
Child Health (MCH) Services in Australia, where children are
routinely monitored 10 times from birth to 3.5 years. The SACS
is administered at 12, 18, and 24 months of age by trained MCH
Nurses to identify the early signs of autism during children’s
routine check-up (30). Importantly, the availability of universal
developmental surveillance of babies by medical professionals in
China, undertaken within Women and Child Health Centres,
provides an ideal platform for monitoring the early signs of
autism to promote early identification and diagnosis of autism.
Tianjin is the fourth largest city in China, consisting of 16
county-level administrative areas. Over 100,000 babies are born
in Tianjin every year, which, based on the estimated prevalence
of autism was 27.5 per 10,000 (31), equates to 1,000–2,000 babies
potentially born with autism each year. A thoroughly developed
women and child health care system in this city, the Tianjin
Women and Children’s Health Centre (TWCHC), has made
it an ideal test site for the implementation of screening for
various child conditions such as congenital heart disease (32),
developmental dysplasia of the hip (33), and congenital cataract
(34). However, developmental surveillance and screening for
autism had not yet been implemented.
Our study objective was to conduct a pilot investigation of
a Mandarin translation of the SACS, –SACS-China (SACS-C)
– in Tianjin, by comparing the outcomes of implementing the
SACS-C with the CHAT-23, as described above, and which has
been widely used with Chinese children (22,35,36). The study
comprised two aims: firstly, to ascertain the feasibility of training
early child health experts to monitor babies and toddlers for
early signs of autism in Tianjin; and, secondly, to determine
the performance of two tools (SACS-C and CHAT-23) to enable
selection for use in early identification for autism in the TWCHC.
METHOD
Study Setting
Tianjin has a three-level maternal and child health care system,
consisting of a city level administrative centre (the TWCHC),
Women and Children’s Health Centres at a district level, and
the community level health centres (CHC). In Tianjin, children’s
health status and development are monitored in the community
health centers by qualified medical health practitioners. The
CHC services are offered to all families with children younger
than 7 years, with an emphasis on child health surveillance and
screening (37). As part of this service, routine health checks
for children in the community are scheduled from birth to 7
years of age. It is expected that children under 12 months are
examined every 3 months, children between 12 and 36 months
are examined every 6 months, and children over 36 months are
examined once a year (37). Every year, over 90% of babies in
Tianjin access the CHC service soon after birth, with attendance
remaining relatively high within the first 2 years; this service
has enormous potential to identify infants at high-likelihood
of autism.
Participants
From May 2013 to July 2014, a total of 10,514 children were
monitored through 61 CHCs in six selected districts in the urban
areas in Tianjin (see star in Figure 1). In 2010, 4.3 million out
of 13 million residents lived in the six central urban districts. The
districts in this study were chosen based on proximity to facilitate
ease of referral to the diagnostic center at the TWCHC, which is
in the city center.
While all 10,514 children were monitored with the SACS-C,
only a subset (n=6,744; 64%) were also screened with the CHAT-
23. Many children in the original SACS studies (26,30) were seen
at each of the 12, 18, and 24 months checks. However, in this
pilot study, children were only monitored twice on the SACS-C
(at 12, 18 and/or 24 months) due to funding restrictions. Initially
children were monitored on the SACS-C at 12-months (n=
3,178), 18-months (n=3,757), and 24-months (n=3,579) of age.
As the SACS-C is a developmental surveillance tool administered
at different time points, the majority of the cohort (66%) initially
monitored at 12, 18, or 24 months were also monitored again by
the health practitioners 6 months later; 79% of 12-month-olds
(n=2,497), 78% of 18-month-olds (n=2,911), and 42% of 24-
month-olds (n=1,494). Children within the age limits of the
CHAT-23 at 18-months (n=3,683) and 24-months (n=3,061)
were only monitored once, given its use as a “once-off” screening
tool. The average age of all children monitored in the study was
18.70 months (SD 4.99), with the sample comprising 52% boys
and 48% girls. Detailed age, gender and assessment characteristics
are shown in Table 1.
Measures
Translation of SACS Checklists
For effective implementation in Tianjin, the SACS was translated
from English into Chinese by one of the authors (CW) and
further validated by a practitioner from the TWCHC. The
SACS-C was then back translated to English, this English
version checked by the first author (JB) and this process
was repeated twice between CW, Chinese colleagues and JB,
with modifications made to be in line with the “meaning” of
the original instrument. Authors CW and JB then evaluated
both the English and Chinese versions to ensure these were
comparable in meaning. A summary of the behaviors monitored
with the SACS-C, highlighting the “key items,” are presented in
Supplementary Table 1.
Training of Community Practitioners on SACS-C
In March 2013, 225 child health practitioners from 61
communities within the six districts in Tianjin received a 3-hour-
training workshop by the authors of the SACS (JB & CD). The
workshops focused on typical and atypical social-communicative
development, the early signs of autism, and the administration
of the SACS items. Simultaneous translation from English to
Mandarin was undertaken during the workshops (by CW), with
all written content also translated and then back translated by the
CW and JB.
The SACS authors (JB and CD) also observed administration
of the SACS-C with two children at each of the ages of
12, 18, and 24-months, undertaken by a number of the
trained health practitioners at TWCHC, and provided in-person
Frontiers in Neurology | www.frontiersin.org 3November 2020 | Volume 11 | Article 597790
Barbaro et al. SACS Tool in China
FIGURE 1 | The map of Tianjin, highlighting the six urban districts involved in the current study.
TABLE 1 | Demographics characteristics of children administered the SACS-C and CHAT 23.
SACS-C CHAT-23
Age (months) 12 18 24 Overall 18 24 Overall
n3,178 3,757 3,579 10,514 3,683 3,061 6,744
Age M (SD) 12.36 (0.60) 18.44 (0.70) 24.6 (1.24) 18.70 (4.99) 18.45 (0.69) 24.23 (0.33) 21.08 (2.93)
Gender
Male (%) 1,653 (30.2) 1,954 (35.7) 1,861 (34.1) 5,468 (100) (52.0) 1,915 (54.6) 1,592 (45.4) 3,507 (100) (52.0)
Female (%) 1,525 (30.2) 1,803 (35.7) 1,718 (34.1) 5,046 (100) (48.0) 1,768 (54.6) 1,469 (45.4) 3,237 (100) (48.0)
Total (%) 3,178 (30.2) 3,757 (35.7) 3, 579 (34.1) 10,514 (100) 3,683 (54.6) 3,061 (45.4) 6,744 (100) (100)
CHAT-23, Checklist for Autism in Toddlers-23; SACS-C, Social Attention and Communication Surveillance-China Tool; n, number of participants; M (SD), mean (standard deviation).
feedback on these administrations. These health practitioners
then assisted CHC practitioners in any queries relating to SACS-
C administration and scoring.
SACS-C Implementation
Following training, the SACS-C was implemented as part of
the routine health checks in the CHCs. Community health
practitioners initially undertook a physical examination of the
child, and the child was monitored on the SACS-C in the presence
of a parent/caregiver. The practitioners, who had been trained on
how each item was to be administrated at each developmental
age, recorded whether the child’s behaviors were typical or
atypical (as opposed to present or absent) on a form provided
for each child. Children were considered at “high-likelihood” for
autism if they did not engage in three of the five “key” items at
12, 18, and/or 24-months-of-age. Practitioners were instructed to
Frontiers in Neurology | www.frontiersin.org 4November 2020 | Volume 11 | Article 597790
Barbaro et al. SACS Tool in China
administer items up to a maximum of three times (e.g., calling
a child’s name). In the minority of cases where practitioners were
unable to elicit a behavior because of the child being ill, distressed,
or asleep, detailed parental/caregiver report was used to mark
the item as typical or atypical. Children who were identified as
“high-likelihood” for autism were referred to the TWCHC for a
follow-up developmental and diagnostic assessment for autism
by two autism specialist pediatricians.
CHAT-23 Training and Implementation
The health practitioners were also trained on the use of the
CHAT-23 by one of the authors (CW), who is a native Chinese
speaker. CHAT-23 is popular in Chinese-speaking areas, and the
Chinese version of the test is available. The training focused
on how to identify the passing or failing in each item, and the
referral standards. The CHAT-23 comprises two parts: Part A
is a parent questionnaire with 23 questions regarding children’s
behaviors, and Part B comprises seven-key item scored based on
observations of the child, conducted by the health practitioner. If
a child fails six or more items of the total of 23 parent completed
items in Part A, they are administered Part B. If the child had
two or more failed items in Part B, the child was identified
as “high-likelihood” for autism on the CHAT-23 and referred
to the TWCHC for a further assessment. Notably, unlike the
SACS-C, which is a developmental surveillance tool administered
across the second year of life, the CHAT 23 is administered
only once between 18 and 24 months of age and is appropriate
after 18 months of age, as it is a screening tool designed to be
administered at one point in development.
Procedure
Study recruitment was conducted through advertisements on
clinic noticeboards, as well as brochures on the early signs of
autism displayed and issued to parents in the visiting room.
The SACS-C and CHAT-23 were introduced during children’s
routine health checks. Firstly, parents filled out Part A of the
CHAT-23 in the waiting room (if the child was aged between 18-
and 24-months). Children then underwent their routine health
checks with the health practitioner, including measurement of
height and weight. The health practitioners reviewed parent
responses on Part A of the CHAT 23 and followed this with
Part B of CHAT-23 (if the child had failed Part A); the SACS-
C was then administered by the practitioner for all children. If
a child was deemed “high-likelihood” for an autism on either
the SACS-C, CHAT-23, or both, the health practitioners advised
parents about their concerns regarding the child’s development in
social attention and communication. Parents were told that the
monitored behaviors were important developmental milestones
that need to be assessed further, and they were then referred to the
TWCHC for a further developmental and diagnostic assessment
for autism.
This study was approved by the Tianjin Women and
Children’s Health Centre (TWCHC) Human Ethics Committee.
Data Collection and Quality Control
Quality Control was undertaken during the entire data collection
process. During the early stages of data collection, nominated
staff from TWCHC and students from Nankai University (NU)
were sent to the six districts, with one person allocated per
district. They assisted the community health practitioners to
correctly administer, score, and use of the SACS-C and CHAT-23
with the children. Furthermore, one staff member from TWCHC
(JW) and Nankai University (CW) visited approximately 35% of
the communities, thus ensuring correct administration of the two
tools in all six districts, and accurate completion of the checklists.
They also provided feedback to the health practitioners on the
use of the tools and the referral procedure. Additionally, mid-
way through data collection, a TWCHC staff member (JW) and
one student from Nankai University re-visited ∼30% of the
communities to check project implementation.
The SACS-C and CHAT-23 data sheets were initially stored in
secure cabinets in the local CHCs and transferred to TWCHC at
the end of the data collection phase, where they were stored in a
secure cabinet. Each child was assigned a unique identification
number, used to link child and assessment details. Health
practitioners also entered the data from the record forms into
a database at each CHC. Both the hard copy form and the
database from each different district was then collected, and data
entered for a second time at the TWCHC. Students from NU
were involved in the second data entry process. Epidata 3.1 was
used for the double data entry, and all the statistical analysis was
undertaken using SPSS 21.0; the final database was stored in an
encrypted computer at TWCHC and Nankai University.
Assessment Protocols for Children at
“High-Likelihood” for Autism
The diagnostic procedure for autism in China involves a
four-step process: (1) Collecting the medical history, including
the clinical symptoms related to autism, the child’s growth
and developmental information, and family history; (2)
Conducting cognitive assessments, including observing the
child’s behavioral symptoms and communication. Based on
their clinical experience, each physician sets up an environment
and activities to observe the child’s behaviors (no one specific
standard applied); (3) physical and neurological examination,
including laboratory tests and administration of psychological
assessments to assist the diagnosis if needed; (4) Before
diagnosing as autism, other conditions such as language
developmental disorders, intellectual disability, childhood
schizophrenia and mental illnesses and other developmental
disorders are excluded (differential diagnosis).
Two pediatricians from the TWCHC [Dr. Liang, Associate
Chief Physician has 14-years of experience in diagnosing
children’s psychological and behavior disorders, and was trained
on the Autism Diagnostic Observation Scale (ADOS); Dr.
Yao, Chief Physician, has more than 10 years of experience
in diagnosing children’s psychological and behavior disorders]
undertook the assessments and diagnosis of the referred children.
The assessment tools commonly used with the referred children
included the ASD Behavior Checklist (ABC) (17), Gesell
Development Scale (GDS) (38), and Infants-Junior Middle
School Students Social-Life Abilities Scale (S-M scale) (39). These
tests were not used on all children but selected at the discretion of
the clinicians based on signs the children were displaying. A final
diagnosis of autism or Non-autism was then made on the basis
of the above tests and clinical judgment. The above assessment
Frontiers in Neurology | www.frontiersin.org 5November 2020 | Volume 11 | Article 597790
Barbaro et al. SACS Tool in China
scales conducted for the children identified “high-likelihood” on
SACS-C and CHAT-23 are listed in Table 2.
Follow-Up
Approximately 80% of the children monitored by the health
practitioners in this study were followed-up in kindergartens
from the six districts in Tianjin when they were aged between
3 and 4 years of age, to identify any “false negatives” from the
surveillance and screening procedure undertaken between 12-
and 24-months of age. JW from TWCHC and at least two trained
practitioners from every district-level Women and Children’s
Health Centers visited the municipal and district kindergartens,
respectively. Firstly, observation sheets were issued to the
teachers in advance, which listed eight atypical behaviors and/or
developmental concerns (see Supplementary Table 2), and they
were asked to nominate children demonstrating those behaviors.
Secondly, interviews were conducted with teachers regarding
children who were identified as showing atypical behaviors to
obtain more information about their behavior and development.
The interviewers then observed the nominated children in the
class, focusing on their social-communication skills and overall
development. If the children were indeed showing atypical
behaviors/development, they were referred to TWCHC for a
further assessment and diagnosis by the two pediatricians.
Early Intervention
Children diagnosed with autism were referred to an autism
intervention organization. Children who were diagnosed with
other delays and disorders were referred to one of the child
development intervention institutes, and their parents were
taught some simple interventions by the clinicians, such as
increasing social activities with other peers, encouraging more
eye contact, and applying effective reinforcers to decrease
behavioral problems.
RESULTS
Children Tested on Both the SACS-C and
CHAT-23
Of the children assessed on both the SACS-C and CHAT-23 (n
=6,744), 21 were flagged as “high-likelihood” on the CHAT-23,
and 52 were flagged as “high-likelihood” on SACS-C, with 17
children identified as being at “high-likelihood” on both tools (see
Table 4). The Positive Predictive Value (PPV) of children with
“high-likelihood” on both SACS-C and CHAT-23 was 81.0%,
whilst the Negative Predictive Value (NPV) was 99.2%.
Psychometric Properties of SACS-C
Of the 10,514 children monitored with the SACS-C, 87 children
were identified as “high-likelihood” (0.83% of the sample). Of
these children at high-likelihood, 27.6% were identified at 12-
months of age, 34.5% at 18-months of age, and 37.9% at 24-
months of age. Only 57 (65.5%) of the 87 high-likelihood children
were assessed for autism, as 30 families declined the invitation
for a developmental assessment (Table 3). Of the 57 children
assessed, 24 were diagnosed with autism (42.1%), and 25 (43.9%)
children were diagnosed with developmental and/or language
delays/disorders (DD/LD); a further 8 (14.0%) children were
determined to be typically developing (TD).
The positive predictive value (PPV) for the SACS-C was 42.1%
for autism and 86.0% for all developmental delays/disorders
when used between 12 and 24-months of age. At the 2-year post-
assessment follow-up, an additional 21 children were identified
and diagnosed with autism; these children had previously been
identified as “not high-likelihood” on the SACS-C when seen
between 12- and 24 months, resulting in a Negative Predictive
Value (NPV) of 99.8% for autism. Sensitivity and specificity
for autism on the SACS-C was 53.3 and 99.7%, respectively.
The estimated prevalence of autism among the study population
monitored by the SACS-C (including follow-up) was 0.55%.
Psychometric Properties of CHAT-23
Of the 6,744 children also monitored with the CHAT-23, 21
children were identified as “high-likelihood” (0.31% of the
sample), with 57% identified at 18-months and 43% at 24-
months. However, as seven families declined an offer for a
developmental assessment, only 14 children at “high-likelihood”
for autism was assessed at the TWCHC. Of these, nine were
diagnosed with autism (64.3%), four were diagnosed with
developmental and/or language delays/disorders (DD/LD), with
one child identified as typically developing (TD) (see Table 3).
The CHAT-23 had an overall PPV of 64.3% for autism and
92.9% for all developmental delays/disorders. At the 2-year
post-assessment follow-up, similar to SACS-C, an additional
24 children were identified and diagnosed with autism among
children originally defined as “low-likelihood” on the CHAT-23,
thus resulting in an NPV of 99.6%. Sensitivity and specificity
for autism on the CHAT-23 was 27.3 and 99.9%, respectively.
The estimated prevalence of autism among the study population
using the CHAT-23 (including follow-up) was 0.56%.
DISCUSSION
This is the first large-scale study on developmental surveillance
for autism in infants and toddlers among children in China.
The findings demonstrated the feasibility of implementing
developmental surveillance for autism within the Tianjin,
Mainland China. They also indicated that the SACS-C tool
was effective in identifying autism in a community-based
sample at an early age. The SACS-C was found to have
higher sensitivity compared with CHAT-23 (53.33 vs. 27.27%,
respectively), but a lower PPV (42.11 vs. 64.29%). For
both measures, the PPV increased with increasing age of
screening, from 12 to 24 months of age, and at the age
of 24 months, the PPV of SACS-C and CHAT-23 were the
same (both PPV =66.7%). A possible explanation is that
for older children, the atypical behaviors are more prevalent
and detectable by both parents and health practitioners.
The specificity and NPV of the two tools were also very
similar (SACS 99.7, 99.8%; CHAT-23 99.9, 99.6%, respectively).
However, the results showed that the SACS-C identified many
more children with autism than the CHAT-23 (0.83 vs.
0.31%), with the latter missing more children during these
early years.
Frontiers in Neurology | www.frontiersin.org 6November 2020 | Volume 11 | Article 597790
Barbaro et al. SACS Tool in China
TABLE 2 | Sample characteristics of assessed children grouped according to age and diagnosis at each health check 12, 18, and 24 months.
Group
Autism DD/LD TD
SACS-C at 12-month-check
SACS-C (n=16) 3 9 4
Mean age of identification (SD) 12.09 (0.11) 12.27 (0.24) 12.11 (0.19)
Mean age of assessment (SD) 15.05 (2.86) 13.07 (1.09) 15.70 (4.53)
Gender (male: female) 3:0 6:3 2:2
Tests
ABC n=1 78.00 n=3 19.33 (8.51) n=1 18.00
Development scale n=2 68.65 (10.96) n=8 78.13 (8.56) n=4 89.63 (7.36)
S-M n =1 9.0 n=4 9.75 (0.50) n=1 10.00
SACS-C at 18-month-check
SACS-C (n=20) 7 (5 “high-likelihood” on CHAT) 10 (2 “high-likelihood” on CHAT) 3 (1 “high-likelihood” on CHAT)
Mean age of identification (SD) 18.50 (0.56) 18.06 (0.43) 18.23 (0.26)
Mean age of assessment (SD) 21.38 (7.097) 22.03 (10.54) 22.18 (1.18)
Gender (male: female) 7:0 8:2 1:2
Tests
ABC n=2 47.50 (7.78) n=6 31.83 (18.23) n=2 18.50 (7.78)
Development scale – – n=6 78.10 (8.68) n=2 88.60 (3.68)
S-M n=5 9.20 (0.48) n=8 8.75 (3.66) n=2 10.00 (0.00)
CHAT-23 at 18-month-check
CHAT-23 (n=8) 5 (5 “high-likelihood” on SACS-C) 2 (2 “high-likelihood” on SACS-C) 1 (1 “high-likelihood” on SACS-C)
Mean age of identification (SD) 18.48 (0.68) 18.12 (0.12) 18.53
Mean age of assessment (SD) 22.39 (8.43) 34.89 (23.60) 23.03
Gender (male: female) 5:0 2:0 1:0
Tests
ABC n=2 47.50 (7.78) n=1 58.00 n=1 24.00
Development scale – – n=1 77.90 n=1 91.20
S-M n=3 9.33 (0.58) n=1 12.00 n=1 10.00
SACS-C at 24-month-check
SACS-C (n=21) 14 (4 “high-likelihood” on CHAT) 6 (0 “high-likelihood” on CHAT) 1 (0 “high-likelihood” on CHAT)
Mean age of identification (SD) 24.98 (1.92) 27.09 (2.78) 26.74
Mean age of assessment (SD) 25.82 (2.13) 27.92 (2.16) 27.04
Gender (male: female) 13:1 3:3 1:0
Tests
ABC n=12 44.08 (17.93) n=1 54.00
Development scale n=4 65.65 (18.03) n=1 91.10
S-M n=9 7.22 (0.44) n=2 8.00 (1.41)
CHAT-23 at 24-month-check
CHAT-23 (n=6) 4 (4 “high-likelihood” on SACS-C) 2 (0 “high-likelihood” on SACS-C) 0
Mean age of identification (SD) 24.22 (0.11) 24.38 (0.33) –
Mean age of assessment (SD) 25.06 (1.65) 34.27 ±14.22 –
Gender (male: female) 3:1 0:2 –
Tests
ABC n=4 56.25 (11.76) n=1 51.00 –
S-M n=1 7.00 – – –
ABC, ASD Behavior Checklist; CHAT-23, Checklist for Autism in Toddlers-23; Development scale, Gesell Development Scale; S-M, Infants-Junior Middle School Students Social-Life
Abilities Scale; SACS-C, Social Attention and Communication Surveillance-China tool; n, number of participants; SD, standard deviation. **Tests, These tests were not used on all
children but selected at the discretion of the clinicians based on signs the children were displaying. –, not applicable/administered.
Previous studies and meta-analyses have reported
considerable variability in prevalence estimates, ranging from
1.8 to 426.4 per 10,000 (12,15,40). These studies indicated that
compared with estimates of around 1% in developed countries,
the reported prevalence of autism in Mainland China is much
lower (12,15,40). Sun et al. reported an estimated prevalence of
Frontiers in Neurology | www.frontiersin.org 7November 2020 | Volume 11 | Article 597790
Barbaro et al. SACS Tool in China
TABLE 3 | Assessment characteristics of children administered the SACS-C and CHAT 23.
SACS-C CHAT-23
Age (months) 12 18 24 Overall 18 24 Overall
n3,178 3,757 3,579 10,514 3,683 3,061 6,744
Assessed (%) 16 (28.1) 20 (35.1) 21 (36.8) 57 (100) 8 (57.1) 6 (42.9) 14 (100)
Autism (%) 3 (12.5) 7 (29.2) 14 (58.3) 24 (100) 5 (55.6) 4 (44.4) 9 (100)
DD/LD (%) 9 (36.0) 10 (40.0) 6 (24.0) 25 (100) 2 (50.0) 2 (50.0) 4 (100)
TD (%) 4 (50.0) 3 (37.5) 1 (12.5) 8 (100) 1 (100) 0 1 (100)
Declined assessment (%) 8 (26.7) 10 (33.3) 12 (40.0) 30 (100) 4 (57.1) 3 (42.9) 7 (100)
Total “high-likelihood” (%) 24 (27.6) 30 (34.5) 33 (37.9) 87 (100) 12 (57.1) 9 (42.9) 21 (100)
PPV Autism % 18.75 35.0 66.7 42.1 62.5 66.7 64.3
PPV all disorders % 75.0 85.0 95.2 86.0 87.5 100.0 92.9
DD/LD, Developmental Delay or Language Delay; TD, Typically Developing (TD); CHAT-23, Checklist for Autism in Toddlers-23; SACS-C, Social Attention and Communication
Surveillance-China tool; PPV, Positive Predictive Value; n, number of participants. **Total “high-likelihood” equals to total children deemed high-likelihood for autism.
TABLE 4 | The number of children deemed at “high” (positive) and “low” (negative)
likelihood for autism following screening on SACS-C and CHAT-23.
CHAT positive CHAT negative Total
SACS-C positive 17 35 52
SACS-C negative 4 6,688 6,692
Total 21 6,723 6,744
CHAT-23, Checklist for Autism in Toddlers-23; SACS-C, Social Attention and
Communication Surveillance-China tool.
119 per 10,000 among 737 school-age (6–10 years) children (7).
In our study population based in Tianjin City, the rate of autism
was estimated to be 0.43% (1 in 233) based on the SACS-C and
0.49% on the CHAT-23 (1 in 204). This estimate is similar to
the prevalence in Shenzhen City, with an estimate of 0.42% (42
per 10,000 95% CI 20–89) (14). Our lower estimated prevalence
rates could possibly be explained by the lack of knowledge of
and experience with the early signs of autism, leading to a lower
detection rate (24).
When the two screening tools were compared in this study,
the SACS-C demonstrated a better balance between accuracy
(PPV) and sensitivity in identifying autism in infants and
toddlers compared to the CHAT-23. There are also a number of
advantages of using SACS-C; firstly, the SACS-C is potentially
more objective because the community health practitioners
directly observed and rated the SACS-C items, whereby their
administration of the CHAT-23 is based on parents responses
in the first instance, who are likely to be less knowledgeable
about autism. (15) Secondly, the SACS-C had a higher sensitivity,
detecting more autism cases in the community-based population,
which is essential as it is the ultimate aim of screening (26).
Although SACS-C had a lower PPV than the CHAT-23, the
higher PPV of the CHAT-23 came at the cost of fewer referrals,
and lower sensitivity. Also, when looking at the 24-month data,
the SACS-C and CHAT-23 had identical PPVs. Finally, the
SACS-C is a developmental surveillance tool, so that repeated
monitoring is conducted across the second year of life, ensuring
the tool is able to identify children with autism at subsequent
checks if they are not initially identified, rather than being a single
screen at a given period.
A significant strength of this study was the successful training
of community health professionals that enabled the community-
based surveillance of infants and toddlers in Tianjin, Mainland
China for autism and related conditions. However, there are a
few study limitations that should be noted. The lower sensitivity
and PPV of SACS-C, compared to the original SACS (30), could
be due to a few factors, such as possible cultural differences in
administration of the SACS-C, limited knowledge and experience
of community health professionals in early autism symptoms
presentation and detection prior to this pilot study, differences
between the two community health systems, and differences in
the diagnostic procedures.
The diagnostic procedures for autism in China differed to
those undertaken in Australia and varied according to the
pediatricians preference. For example, the diagnostic assessments
were not conducted using gold standard diagnostic tools such as
ADOS (41), and Autism Diagnostic Interview-Revised (ADI-R)
(42). Given that the percentage of children identified as “high-
likelihood” on SACS-C (0.83%) was similar to the rate of children
at “high-likelihood” for autism in the original SACS (1.04%)
(30), it is possible that the diagnostic assessments conducted in
Tianjin were not identifying as many children with autism that
did indeed have autism, and instead diagnosed children with
other conditions instead.
This pilot study implemented the SACS-C in Tianjin, China,
and effectively compared its performance with that of CHAT-23
in a large community-based sample. In so doing, the feasibility of
successfully training community health practitioners to monitor
infants and toddlers for the early signs of autism using SACS-
C was established. The SACS-C was found to be efficacious
and culturally valid for use with Tianjin infants and toddlers
aged 12- to−24-months. The SACS-C revealed a good balance
between accuracy and sensitivity in detecting autism compared
to the CHAT-23, which missed the majority of children on the
spectrum (72.7 vs. 46.7%). Given these findings, it was found
that newly-trained community health practitioners can identify
Frontiers in Neurology | www.frontiersin.org 8November 2020 | Volume 11 | Article 597790
Barbaro et al. SACS Tool in China
and refer more infants and toddlers with the early signs of
autism on SACS-C than CHAT-23, indicating that the SACS-C
is a viable alternative to be implemented in the CHC system in
Tianjin. Based on the findings from this study, the team at the
TWCHC selected SACS-C as the preferred autism developmental
surveillance tool, such that it was incorporated into the 7-year
Tianjin Women and Child Health Plan (2013–2020). Infants
and toddlers in Tianjin have since been monitored for autism
using the SACS-C following the training of all early child health
professionals in Tianjin. However, future research is needed to
improve the psychometric properties of the SACS-C in Mainland
China so that it is comparable to its use in Australia.
DATA AVAILABILITY STATEMENT
All datasets generated for this study are included in the
article/Supplementary Material.
ETHICS STATEMENT
The studies involving human participants were reviewed and
approved by Tianjin Women and Children’s Health Centre
(TWCHC) Human Ethics Committees. Written informed
consent to participate in this study was provided by the
participants’ legal guardian/next of kin.
AUTHOR CONTRIBUTIONS
JiW conducted the analyses, contributed to the interpretation of
the results, and drafted the initial manuscript. JB, CW, GL, and
CD developed the study design, contributed to data analysis and
interpretation, and reviewed drafts, with JB coordinating these
tasks. YL conducted the developmental assessment for children
referred for assessment. JinW and IA contributed to the literature
review and review of the manuscript. All authors contributed to
the article and approved the submitted version.
FUNDING
This research received funding from the projects of the
National Key Research and Development Program of China
(2016YFC0900600/2016YFC0900602), Tianjin Women’s and
Children’s Health Centre, and Tianjin Public Health Bureau of
Science and Technology Fund (Grant no. 12KG130). JB and CD
was supported by funding from the Australian Government’s
Department of Innovation, Industry, Science, and Research,
through its Australia-China Science and Research Fund Group
Mission, and JB was supported by a La Trobe Asia Grant through
La Trobe University.
ACKNOWLEDGMENTS
We would like to acknowledge the dedication of the community
health practitioners in the CHCs that conducted all the
developmental checks for this study, and the immense
contribution of the staff at TWCHC that made this project
possible. We also sincerely thank Dr. Yao for their expertise
in conducting the developmental assessments along with
YL, and all the parents and children that participated in
the study.
SUPPLEMENTARY MATERIAL
The Supplementary Material for this article can be found
online at: https://www.frontiersin.org/articles/10.3389/fneur.
2020.597790/full#supplementary-material
REFERENCES
1. American Psychiatric Association. Diagnostic and Statistical Manualof Mental
Disorders. 5th ed. Washington, DC: American Psychiatric Association (2013).
2. Clark M, Barbaro J, Dissanayake C. Continuity and change in cognition and
autism severity from toddlerhood to school age. J Autism Dev Disord. (2017)
47:328–39. doi: 10.1007/s10803-016-2954-7
3. Clark M, Vinen Z, Barbaro J, Dissanayake C. School age outcomes of children
diagnosed early and later with autism spectrum disorder. J Autism Dev Disord.
(2018) 48:92–102. doi: 10.1007/s10803-017-3279-x
4. Dawson G. Early behavioral inter vention, brain plasticity, and the
prevention of autism spectrum disorder. Dev Psychopathol. (2008) 20:775–
803. doi: 10.1017/S0954579408000370
5. Flanagan H, Perry A, Freemand N. Effectiveness of large-scale community-
based intensive behavioral intervention: a waitlist comparison study
exploring outcomes and predictors. Res Autism Spect Disord. (2012) 6:673–
82. doi: 10.1016/j.rasd.2011.09.011
6. Feinstein A. A History of Autism: Conversations with the Pioneers. Chichester;
Malden, MA: Wiley-Blackwell. (2010). 381 p.
7. Sun X, Allison C, Matthews FE, Zhang Z, Auyeung B, Baron-Cohen S, et al.
Exploring the underdiagnosis and prevalence of autism spectrum conditions
in Beijing. Autism Res. (2015) 8:250–60. doi: 10.1002/aur.1441
8. Maenner MJ, Shaw KA, Baio J, Washington A, Patrick M, DiRienzo
M, et al. Prevalence of autism spectrum disorder among children
aged 8 years — autism and developmental disabilities monitoring
network, 11 sites, United States, 2016. Surveill Summ. (2020)
69:1–12. doi: 10.15585/mmwr.ss6904a1
9. Baron-Cohen S1, Scott FJ, Allison C, Williams J, Bolton P, Matthews FE,
et al. Prevalence of autism-spectrum conditions: UK school-based population
study. Br J Psychiatry. (2009) 194:500–9. doi: 10.1192/bjp.bp.108.059345
10. Autism Spectrum Australia. Autism Prevalence Rate up by an Estimated 40%
to 1 in 70 People. Sydney, NSW: Autism Spectrum Australia (2018).
11. Autism Speaks. New Study Reveals Autism Prevalence in South Korea
Estimated to be 2.6% or 1 in 38 Children. New York, NY: Autism Speaks (2011).
12. Wan Y, Hu Q, Li T, Du Y, Feng L, Wong JCM, et al. Prevalence of autism
spectrum disorders among children in China: a systematic review. Shanghai
Arch Psychiatry. (2013) 25:70–80. doi: 10.3969/j.issn.1002-0829.2013.02.003
13. Wang F, Lu L, Wang SB, Zhang L, Ng CH, Ungvari GS, et al. The prevalence
of autism spectrum disorders in China: a comprehensive meta-analysis. Int J
Biol Sci. (2018) 14:717–25. doi: 10.7150/ijbs.24063
14. Sun X, Allison C, Wei L, Matthews F, Auyeung B, Wu YY, et al. Autism
prevalence in China is comparable to Western prevalence. Mol Autism. (2019)
10:1–19. doi: 10.1186/s13229-018-0246-0
15. Sun X, Allison C, Matthews FE, Sharp SJ, Auyeung B, Baron-Cohen S, et al.
Prevalence of autism in Mainland China, Hong Kong and Taiwan:a systematic
review and meta-analysis. Mol Autism. (2013) 4:7. doi: 10.1186/2040-2392-4-7
16. Clancy H, Dugdalei A, Rendle-Shortt J. The diagnosis of infantile autism. Dev
Med Child Neurol. (1969) 11:432–42. doi: 10.1111/j.1469-8749.1969.tb01461.x
17. Krug DA, Arick J, Almond P. Behaviour checklist for identifying severely
handicapped individuals with high levels of autistic behaviour. J Child
Frontiers in Neurology | www.frontiersin.org 9November 2020 | Volume 11 | Article 597790
Barbaro et al. SACS Tool in China
Psychol Psychiatry. (1980) 21:221–9. doi: 10.1111/j.1469-7610.1980.tb0
1797.x
18. Baron-Cohen S, Allen J, Gillberg C. Can autism be detected at 18 months?
The needle, the haystack, the CHAT. Br J Psychiatry. (1992) 161:839–
43. doi: 10.1192/bjp.161.6.839
19. Wang Ke, Wang C, Guo D, van Wihingaarden M, Begeer S. Children
with autism spectrum disorder from China and the Netherlands: age of
diagnosis, gender and comorbidities. Res Autism Spect Disord. (2018) 54:76–
82. doi: 10.1016/j.rasd.2018.07.004
20. Sun X, Allison C, Auyeung B, Matthews FE, Baron-Cohen S, Brayne C. What
is available for case identification in autism research in mainland China? Res
Autism Spect Disord. (2013) 7:579–90. doi: 10.1016/j.rasd.2012.11.003
21. Robins DL, Fein D, Barton M. The Modified Checklist for Autism in Toddlers
(M-CHAT). Storrs: Self-published (1999).
22. Wong V, Hui LH, Le WC, Leung LS, Ho PK, Lau WL. A modified screening
tool for autism (Checklist for Autism in Toddlers [CHAT-23]) for Chinese
children. Pediatrics. (2004) 114:e166–76. doi: 10.1542/peds.114.2.e166
23. Wu X, Lu Y, Wamg Y, Zheng Q, Wang T, Lin J. Investigation of childhood
autism status in Lianyungang city. J Mod Med Hyg. (2010) 26:3724−6.
24. Wang J, Hedley D, Bury S, Barbaro J. A systematic review of
screening tools for the detection of autism spectrum disorder
in mainland China and surrounding regions. Autism. (2020)
24:285–96. doi: 10.1177/1362361319871174
25. Barbaro J, Dissanayake C. Diagnostic stability of autism spectrum disorder in
toddlers prospectively identified in a community-based setting: behavioural
characteristics and predictors of change over time. Autism. (2017) 21:830–
40. doi: 10.1177/1362361316654084
26. Mozolic-Staunton B, Donelly M, J Y, Barbaro J. Early detection for better
outcomes: universal developmental surveillance for autism across health
and early childhood education settings. Res Autism Spect Disord. (2020)
71:101496. doi: 10.1016/j.rasd.2019.101496
27. Barbaro J, Dissanayake C. Developmental profiles of infants and
toddlers with autism spectrum disorders identified prospectively
in a community-based setting. J Autism Dev Disord. (2010)
42:1939–48. doi: 10.1007/s10803-012-1441-z
28. Carbone PS, Norlin C, Young PC. Improving early identification and
ongoing care of children with autism spectrum disorder. Pediatrics. (2016)
137:e20151850. doi: 10.1542/peds.2015-1850
29. Li C, Zhu G, Feng J, ZXu Q, Zhou Z, Bm Z, et al. Improving the early screening
procedure for autism spectrum disorder in young children: experience
from a community based model in Shanghai. Autism Res. (2018) 11:1206–
17. doi: 10.1002/aur.1984
30. Barbaro J, Dissanayake C. Prospective identification of autism spectrum
disorders in infancy and toddlerhood using developmental surveillance: The
Social Attention and Communication Study. J Dev Behav Pediatr. (2010)
31:376–85. doi: 10.1097/DBP.0b013e3181df7f3c
31. Huang JP, Cui SS, Han Y, Irva HP, Qi LH, Zhang X. Prevalence and
early signs of autism spectrum disorder (ASD) among 18-36 month-
old children of Tianjin in China. Biomed Environ Sci. (2014) 27:453–61.
doi: 10.3967/bes2014.008
32. Liu X, Liu G, Wang P, Huang Y, Liu E, Li D, et al. Prevalence of
congenital heart disease and its related risk indicators among 90,796 Chinese
infants aged less than 6 months in Tianjin. Int J Epidemiol. (2015) 44:884–
93. doi: 10.1093/ije/dyv107
33. Yang GY, Li YY, Luo DZ, Hui C, Xiao K, Zhang H. Differences of
anteroposterior pelvic radiographs between supine position and standing
position in patients with developmental dysplasia of the hip. Orthop Surg.
(2019) 11:1142–8. doi: 10.1111/os.12574
34. Huo LA, Yang J, Zhang C. Regional difference of genetic factors for
congenitals cataract. The results of congenital cataract screening under
normal pupil conditions for infants in Tianjin city. Eur Rev Med Pharmacol
Sci. (2014) 18:426–30.
35. Ren S, Ma HW, Hu M, Wang LB, Wang L, Li F, et al. Clinical application of
M-CHAT and CHAT-23 for autism screening. Zhongguo Dang Dai Er Ke Za
Zhi. (2012) 14:946–50.
36. Guo W, Zhu G, Zhou Z, Chen H, Xu X, Lu S. Study on the application of
Chat-23 scale in early screening of childhood autism in communities. Matern
Child Health Care China. (2013) 28:28–32.
37. Wang J, Liu E, Wang Y, Qiao Y, Zhang T, Li B, et al. Association
of early pregnancy body mass index and children’s birth weight with
risk of being overweight in childhood. Am J Hum Biology. (2018)
30:e23174. doi: 10.1002/ajhb.23174
38. Gesell A, Amatrude CS. Developmental Diagnosis; Normal and Abnormal
Child Development. Ann Arbor, MI: P. B. Hoeber (1941).
39. Li B, Han K, Yang L, Huang M, Huang Z, Li Y,et al. The characteristics of social
maturity in infants and children with cochlear implants in China. Int J Pediatr
Otorhinolaryngol. (2020) 131:109887. doi: 10.1016/j.ijporl.2020.109887
40. Cubells JF. Prevalence of autism spectrum disorders in China. Shanghai
Arch Psychiatry. (2013) 25:176–7. doi: 10.3969/j.issn.1002-0829.2013.
03.008
41. Molloy C, Murray D, Akers R, Mitchell T, Manning-Courtney P.
Use of the Autism Diagnostic Observation Schedule (ADOS) in a
clinical setting. Autism. (2011) 15:143–6. doi: 10.1177/13623613103
79241
42. Cicchetti DV, Lord C, Koenig K, Klin A, Volkmar FR. Reliability
of the ADI-R: multiple examiners evaluate a single case. J
Autism Dev Disord. (2008) 38:764–70. doi: 10.1007/s10803-007-0
448-3
Conflict of Interest: The authors declare that the research was conducted in the
absence of any commercial or financial relationships that could be construed as a
potential conflict of interest.
Copyright © 2020 Barbaro, Wang, Wang, Liu, Liang, Wang, Abdullahi and
Dissanayake. This is an open-access article distributed under the terms of
the Creative Commons Attribution License (CC BY). The use, distribution or
reproduction in other forums is permitted, provided the original author(s) and the
copyright owner(s) are credited and that the original publication in this journal
is cited, in accordance with accepted academic practice. No use, distribution or
reproduction is permitted which does not comply with these terms.
Frontiers in Neurology | www.frontiersin.org 10 November 2020 | Volume 11 | Article 597790
