ArticlePDF Available

Abstract and Figures

Although social validation of the goals, methods, and outcomes of evidence-based practices (EBPs) in autism treatment is a significant factor in their selection and effective use, EBPs are typically identified on the basis of the technical soundness of research without consideration of social validity. The authors investigated EBPs and emerging treatments identified by the National Autism Center (NAC) and National Professional Development Center on Autism Spectrum Disorders (NPDC) to determine which interventions have evidence of social validity, and the types of social validation addressed. A review of 828 articles cited by the NAC and NPDC determined that only 221 articles (26.7%) demonstrated direct evidence of the measurement of social validation. Of seven social validity categories analyzed, only consumer satisfaction, clinically significant behavioral change, and socially important dependent variables were consistently reported. A list of EBPs with varying levels of social and empirical validation is presented, and implications for future research are discussed.
Content may be subject to copyright.
Focus on Autism and Other
Developmental Disabilities
2017, Vol. 32(3) 188 –197
© Hammill Institute on Disabilities 2016
Reprints and permissions:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/1088357616632446
focus.sagepub.com
Article
Although the field of autism intervention and treatment is
still emerging, a lack of knowledge about what to do should
no longer be considered a significant challenge facing ser-
vice providers. Recent large-scale, systematic, and robust
research reviews have identified interventions with strong
empirical evidence of effectiveness in some outcome areas.
The results of these reviews provide support for the use of
specific treatments with individuals who have autism spec-
trum disorder (ASD), especially within certain subpopula-
tions. Two initiatives, in particular, the National Autism
Center’s (NAC; 2009) National Standards Project (NSP)
and similar efforts by the National Professional Development
Center on Autism Spectrum Disorders (NPDC; Odom,
Collet-Klingenberg, Rogers, & Hatton, 2010; Wong et al.,
2014), have resulted in the dissemination of lists of empiri-
cally validated treatments that can be implemented by
teachers, parents, and therapists. Although differing some-
what in methodology, scope, and definitions (NPDC, 2013),
the NSP and NPDC studies independently and reliably
identified autism interventions and treatments considered to
be “established” and “confirmed,” respectively, as evi-
dence-based practices (EBPs) in autism. The NPDC list
originally specified 24 confirmed EBPs, 18 of which were
also identified as established treatments by the NSP.
Additional EBPs have recently been added (and one was
removed) by the NPDC (Wong et al., 2015).
Both the NSP and NPDC reviews developed criteria to
systematically analyze the content (i.e., the “quality, quan-
tity, and consistency,” NAC, 2009, p. 31) of research find-
ings and their corresponding strength of evidence. Both
research teams sought to ensure adequate empirical evi-
dence was available to reliably determine the efficacy of
treatments for individuals with ASD before labeling any
intervention as an established, confirmed, or emerging EBP.
Despite this encouraging development, research in spe-
cial education, applied behavior analysis (ABA), and other
therapeutic fields suggests empirical evidence of effective-
ness is often not sufficient to ensure the widespread use of
potentially effective interventions, treatments, and programs
(e.g., Bodfish, 2004; Carter, 2010). Although using EBPs in
the public education of children with autism and other dis-
abilities is mandated by federal regulations (e.g., Individuals
With Disabilities Education Improvement Act [IDEIA],
2004; No Child Left Behind [NCLB], 2002) and state and
632446FOAXXX10.1177/1088357616632446Focus on Autism and Other Developmental DisabilitiesCallahan et al.
research-article2016
1University of North Texas, Denton, USA
2Anadolu University, Eskisehir, Turkey
3National Taiwan Normal University, Taipei, Taiwan
Corresponding Author:
Kevin Callahan, Kristin Farmer Autism Center, University of North
Texas, 490 S. I-35 East, Denton, TX 76205-7768, USA.
Email: Callahan@unt.edu
Social Validity of Evidence-Based Practices
and Emerging Interventions in Autism
Kevin Callahan, PhD1, Heather L. Hughes, PhD1,
Smita Mehta, PhD1, Karen A. Toussaint, PhD1,
Susan M. Nichols, PhD1, Phoenix S. Ma, EdD1,
Metehan Kutlu, MEd2, and Hui-Ting Wang, PhD3
Abstract
Although social validation of the goals, methods, and outcomes of evidence-based practices (EBPs) in autism treatment is a
significant factor in their selection and effective use, EBPs are typically identified on the basis of the technical soundness of
research without consideration of social validity. The authors investigated EBPs and emerging treatments identified by the
National Autism Center (NAC) and National Professional Development Center on Autism Spectrum Disorders (NPDC)
to determine which interventions have evidence of social validity, and the types of social validation addressed. A review
of 828 articles cited by the NAC and NPDC determined that only 221 articles (26.7%) demonstrated direct evidence of
the measurement of social validation. Of seven social validity categories analyzed, only consumer satisfaction, clinically
significant behavioral change, and socially important dependent variables were consistently reported. A list of EBPs with
varying levels of social and empirical validation is presented, and implications for future research are discussed.
Keywords
autism, autism spectrum disorder, evidence-based practices, social validation, applied behavior analysis, special education
Callahan et al. 189
local education policies, many service providers resist their
use (Bodfish, 2004) or continue to implement practices dem-
onstrated by research to be ineffective (Cook & Odom,
2013; Lilienfeld, Marshall, Todd, & Shane, 2015). And,
even though the delivery of effective programming based on
scientific evidence is emphasized by professional standards
and codes of ethics in special education (Council for
Exceptional Children, 2013), ABA (Behavior Analyst
Certification Board, 2010), and most other therapeutic pro-
fessions providing autism services (e.g., American
Occupational Therapy Association, 2010), significant chal-
lenges remain in the widespread effective implementation of
research-demonstrated EBPs in public schools and other set-
tings, including homes, clinics, and centers (Cook & Cook,
2011; Mesibov & Shea, 2011). A variety of factors related to
the users of EBPs, the practices themselves, and their insti-
tutional context (Cook & Odom, 2013; Rapp et al., 2010)
have been reported to negatively affect the effective use of
EBPs. These factors include a lack of knowledge of EBPs,
limited availability of training, low competence of service
providers, high costs, lack of time for planning and prepara-
tion, and difficulties using some interventions, among many
others. Limited social validation is another possible factor
affecting the routine application of EBPs.
Social validity can generally be defined as consumer sat-
isfaction with the goals, procedures, and outcomes of pro-
grams and interventions (Alberto & Troutman, 2008; Wolf,
1978). The extent to which a particular intervention or treat-
ment receives social validation plays a significant role in
whether the treatment is adopted and implemented (Kazdin,
1977; Kern & Manz, 2004). According to Carter (2010),
“. . . just because a program is considered effective does not
mean that it will be considered appropriate by those closely
involved in implementation of the program, or that it should
even be implemented in the first place” (p. 7). Thus, without
social validation, there is perhaps less chance autism EBPs
will be used effectively in homes, schools, and clinics
(Bodfish, 2004; Carter, 2010; Kazdin, 1977; Schwartz &
Baer, 1991; Wolf, 1978). It can be argued that social valid-
ity is a necessary additional consideration in the selection
and effective use of EBPs in autism.
Extending previous seminal work investigating the qual-
ity of intervention research for individuals with disabilities
(Horner et al., 2005; Gersten et al., 2005), Reichow and col-
leagues (Reichow, Doehring, Cicchetti, & Volkmar, 2011;
Reichow, Volkmar, & Cicchetti, 2008) identified quality
indicators of robust autism research, including seven spe-
cific components of social validation. These social validity
indicators include (a) socially important dependent vari-
ables (i.e., society would value the changes in outcome of
the study), (b) time- and cost-effective intervention (i.e., the
ends justify the means), (c) comparisons between individu-
als with and without disabilities, (d) a behavioral change
that is large enough for practical value (i.e., it is clinically
significant), (e) consumers who are satisfied with the
results, (f) independent variable manipulation by people
who typically come into contact with the participant, and
(g) a natural context. Further research investigating key
aspects of social validation can provide important addi-
tional evidence to support the identification and use of spe-
cific interventions for individuals with ASD across the
autism spectrum and life span.
Fortunately, increased attention has been given to the
social validation of autism interventions and treatments.
Similar to the NSP and NPDC results, researchers have
identified autism interventions and treatment components,
as well as specific target skills, which have been socially
validated by parents, special educators, and public school
administrators (Callahan, Henson, & Cowan, 2008) and by
behavior analysts and other service providers (Dickson
et al., 2014). Callahan et al. (2008) asked parents, special
education teachers, and administrators to socially validate
essential components of autism programs based on their
perceived importance as part of comprehensive school-
based programming. These authors reported a rank-ordered
list of intervention components with broad-based support
for their use organized into a comprehensive treatment
model referred to as IDEAL, an acronym of the model’s
five functional areas, including (a) individualized program-
ming, (b) data-based decision making, (c) use of empiri-
cally demonstrated interventions, (d) active collaboration,
and (e) a focus on long-term outcomes. The IDEAL Model
components (see Table 1) were further reliably analyzed to
determine whether they aligned with practices typically
associated with ABA, the Treatment and Education of
Autistic and Related Communication-Handicapped
Children Model (TEACCH), both ABA and TEACCH, or
neither approach (Callahan, Shukla-Mehta, Magee, & Wie,
2010). A total of 24 subject matter experts with certification
and/or extensive training and experience (eight board certi-
fied behavior analysts, eight TEACCH-trained practitio-
ners, eight experts with training in both models) rated which
IDEAL Model components represented a “defining feature”
of ABA or TEACCH, or both models, as they are commonly
recognized and practiced with students with ASD. Overall
agreement with the authors’ assigned categories was 70.2%
(Callahan et al., 2010, p. 76).
More recently, Dickson et al. (2014) investigated core
skills important to individuals with autism across the life
span using an assessment instrument consisting of direct
and indirect measures of skills. Two hundred forty-five par-
ents, teachers, and therapeutic staff responsible for imple-
menting effective treatment programs were asked to indicate
which items they considered to be foundational. The results
provided a measure of social validation for specific treat-
ment targets in the domains of discrimination, communica-
tion, social skills, self-help, health and safety, and recreation/
physical education. The authors concluded these socially
190
Table 1. Summary of Evidence of SV.
IDEAL Model intervention
components/SV rating (SD)/model
affiliation: ABA, TEACCH, or BOTH
EBP/emerging practice/
articles cited (n)
Evidence of SV within NSP
and NPDC articles
Articles reporting social validity across Reichow, Doehring,
Cicchetti, and Volkmar (2011) SV categories (n)
NSP NPDC
Articles
revieweda (n)
Articles
reporting
social validity
(n/%)
Socially
important
dependent
variables
Time- and
cost-effective
Individual with
and without
disabilities
Clinically
significant
behavioral
change
Consumer
satisfied with
results
Independent
variable ×
People in typical
context
Natural
context
Social skills strategies/6.69 (0.56)/
BOTH
Social skills packageb/16 Social skills
training/15
28 13/46.4% 3 4 5 5 11 1 4
Structure and routines/6.62 (0.73)/
TEACCH
Schedules/12 NA/Structured
work systemsc
12 6/50.0% 3 3 1 5 5 1 1
Communication skills strategies/6.57
(0.70)/BOTH
Behavioral package/228 Functional
communication
training/12
234 35/15.0% 14 7 1 23 26 10 10
FBA/functional analysis/6.54 (0.69)/
ABA
Behavioral package/228 FBA/10 236 35/14.8% 14 9 1 23 26 8 8
Motivational proceduresd/6.53 (0.64)/
BOTH
Antecedent package/99 Antecedent-based
intervention/32
121 22/18.2% 10 4 3 5 16 3 3
Effective reinf./6.42 (0.80)/BOTH Behavioral package/ Reinf./43 253 35/13.8% 15 8 1 24 26 9 9
228
Task analysis/6.40 (0.79)/BOTH Behavioral package/ Task analysis/7 235 33/14.0% 15 6 2 23 24 8 9
228
Prompting, shaping, chaining, and
fading/6.39 (0.79)/ABA
Antecedent package/99 Prompting/33 128 23/18.0% 16 3 4 7 19 2 5
Incidental teaching/natural.
teach./6.32 (0.78)/BOTH
Natural. teaching
strategies/32
Natural.
intervention/10
37 17/45.9% 3 6 4 14 13 3 3
Peer-mediated intervention/6.32
(0.82)/BOTH
Peer training
package/33
Peer-mediated
instruction and
intervention/15
42 15/35.7% 8 5 6 13 3 5 4
Incentive programs based on positive
reinf./6.18 (0.94)/ABA
Behavioral package/228 Reinf./43 253 35/13.8% 15 8 1 24 26 9 9
Intensive 1:1 instruction trials/6.17
(1.00)/ABA
Behavioral package/228 Discrete trial
training/13
240 32/13.3% 14 6 1 22 23 8 8
Preferred reinforcers/6.16 (1.06)/
ABA
Antecedent package/99 Antecedent-based
intervention/32
121 22/18.2% 10 4 3 5 16 3 3
Visual activity schedules/visual
supports/6.16 (1.09)/TEACCH
Schedules/12 Visual supports/18 26 17/65.4% 13 7 4 7 16 1 2
PRT/6.15 (0.90)/BOTH PRT/14 PRT/7 17 6/35.3% 0 1 0 4 5 2 0
Individual. desks and furniture, other
classroom materials/6.10 (1.03)/
BOTH
Antecedent package/99 Antecedent-based
intervention/32
121 22/18.2% 10 4 3 5 16 3 3
Formal reinf. assessments/6.08 (0.95)/
ABA
Behavioral package/228 Reinf./43 253 35/13.8% 15 8 1 24 26 9 9
(continued)
191
IDEAL Model intervention
components/SV rating (SD)/model
affiliation: ABA, TEACCH, or BOTH
EBP/emerging practice/
articles cited (n)
Evidence of SV within NSP
and NPDC articles
Articles reporting social validity across Reichow, Doehring,
Cicchetti, and Volkmar (2011) SV categories (n)
NSP NPDC
Articles
revieweda (n)
Articles
reporting
social validity
(n/%)
Socially
important
dependent
variables
Time- and
cost-effective
Individual with
and without
disabilities
Clinically
significant
behavioral
change
Consumer
satisfied with
results
Independent
variable ×
People in typical
context
Natural
context
Differential reinf. strategies/6.06
(1.00)/ABA
Behavioral package/228 Differential reinf.
of alternative
incompat.
or other
behavioral/26
241 32/13.3% 14 6 1 22 23 8 8
Stimulus control/6.06 (1.10)/ABA Antecedent package/99 Antecedent-based
intervention/32
121 22/18.2% 10 4 3 5 16 3 3
Progress. and constant time
delay/6.05 (0.99)/ABA
Antecedent package/99 Time delay/12 107 18/16.8% 11 2 3 3 14 2 4
Adult-directed strategies/6.02 (0.97)/
ABA
Modeling/50 Prompting/33 83 17/20.5% 5 2 2 14 8 1 3
Student preferences and/or
interests/6.02 (1.03)/BOTH
Antecedent package/99 Antecedent-based
intervention/32
121 22/18.2% 10 4 3 5 16 3 3
Systematic prompts and prompt
hierarchies/5.99 (1.10)/ABA
Antecedent package/99 Prompting/33 128 23/18.0% 16 3 4 7 19 2 5
Modifying physical environment/5.95
(1.06)/TEACCH
Antecedent package/99 Antecedent-based
intervention/32
121 22/18.2% 10 4 3 5 16 3 3
Modeling/video modeling/5.93 (1.09)/
ABA
Modeling/50 Video modeling/32 66 26/39.4% 10 5 2 17 11 2 1
Extinction/5.91 (1.24)/ABA Reductive packageb/33 Extinction/11 44 1/2.3% 1 1 0 1 1 0 0
Physical classroom organizers/5.75
(1.28)/TEACCH
Schedules/12 NA/Structured
work systemsc
12 6/50.0% 3 3 1 5 5 1 1
Technology and other augmentative
communication devices/5.74 (1.30)/
BOTH
Technology-based
treatmentb/19
Technology-aided
instruction and
intervention/20
37 7/18.9% 6 2 0 3 6 2 1
Pictorial storyboard simulations
and/or social stories/5.72 (1.15)/
TEACCH
Story-based
intervention
package/21
Social
narratives/17
31 23/74.2% 13 5 2 14 13 3 7
PECS/5.71 (1.31)/TEACCH PECSb/13 PECS/6 17 6/35.3% 3 3 0 3 5 1 2
Self-management strategies/5.70
(1.10)/BOTH
Self-management/21 Self-
management/10
27 9/33.3% 5 4 1 6 6 2 4
Note. ABA = applied behavior analysis; TEACCH = Treatment and Education of Autistic and related Communication-handicapped Children; EBP = evidence-based practices; SV = social validation; NSP = National Standards
Project (National Autism Center); NPDC = National Professional Development Center on Autism Spectrum Disorders; FBA = functional behavior assessment; PRT = pivotal response training; PECS = Picture Exchange
Communication System; NAC = National Autism Center.
aThe number of articles reviewed equals the total number of articles cited by the NSP and NPDC minus the number of articles appearing on both reviews. bConsidered an “emerging practice” by the NAC. cNot applicable (NA)
because “more stringent evidence reduced previous evidence” and the intervention is no longer considered an EBP by the NPDC. dPreferred items and natural reinforcers.
Table1. (continued)
192 Focus on Autism and Other Developmental Disabilities 32(3)
validated skills, such as making requests for assistance or
desired items, responding to one’s name, and following
one-step directions, can be used as a starting point for
instructional programming for individuals with ASD
(Dickson et al., 2014).
Now that researchers have reliably identified potentially
effective interventions from which parents, teachers/thera-
pists, and administrators can choose, research attention can
begin to address specific factors affecting how these con-
sumers select and implement EBPs (Cook & Odom, 2013).
A critical first step in this process is to help service provid-
ers identify from among the relatively large list of available
interventions and treatments those that, ostensibly, have the
greatest likelihood of making a positive impact on specific
target deficits of their children, students, and clients (Odom,
2009; Odom, Cox, Brock, & the NPDC, 2013). The results
of additional social validity research can help inform this
process. For example, identifying which specific social val-
idation components are most critical for the successful
implementation of EBPs, and which aspects of social valid-
ity are explored most and least frequently in the literature,
can provide new insights for practitioners and researchers.
The purpose of this study is to identify the extent to
which autism interventions and treatment components are
reported to have both social and empirical validation and to
investigate the types of social validity commonly addressed
by autism researchers and practitioners. The resulting list of
EBPs could serve as a starting point for service providers
designing and implementing autism programming.
Identifying socially validated EBPs could help address (and
possibly eliminate) some of the barriers to effective imple-
mentation of EBPs, and address the pervasive research-to-
practice gaps in autism programming (Dingfelder &
Mandell, 2011). Most importantly, a prioritized list of vali-
dated interventions and training resources could ultimately
result in improved outcomes for individuals with ASD and
their families.
Method
Participants and Procedures
Preliminary analysis. Two analyses were conducted to iden-
tify the extent to which empirically demonstrated EBPs also
have evidence of social validation. For the first analysis, a
team of three researchers representing doctoral-level exper-
tise in special education, ABA, autism assessment, and
early childhood education compared the correspondence of
empirically validated treatments and interventions using the
NSP and NPDC lists of established, confirmed, and emerg-
ing EBPs with a list of 60 evidence-based intervention com-
ponents that were socially validated using a mail survey by
187 parents, special education teachers, and public school
administrators (Callahan et al., 2008). The authors included
treatments categorized as “emerging” by the NSP. These
treatments are supported by a minimum of one group design
or two single-subject design studies with a minimum of six
participants, with adequate scientific merit, and no conflict-
ing results reported. In addition, beneficial treatment effects
have been reported for these interventions for at least one
dependent variable/specific target behavior (NAC, 2009).
Previously, survey respondents rated each of the 60 inter-
vention components on a 7-point Likert-type scale based on
their perceived importance of the intervention as part of a
high-quality autism program, where a rating of 7 indicated
the component was “Absolutely Important (the component
is an indispensable part of a high-quality autism program)”
and a rating of 1 meant the component was “not at all
important (the component is totally irrelevant and/or unre-
lated to a high-quality autism program).”
Inter-rater agreement. For this preliminary study, the
research team independently analyzed each of the socially
validated intervention components of the IDEAL Model to
identify corresponding treatments empirically validated by
the NSP and/or NPDC reports. Each member of the research
team was instructed to thoroughly read designated NSP and
NPDC materials to become knowledgeable about the
research and analysis procedures and the treatment defini-
tions used. The researchers were directed to independently
review each of the 60 IDEAL Model components and deter-
mine whether the components directly aligned with any of
the 24 treatments identified as “established” or “emerging”
by the NSP and/or those “confirmed” as an EBP by the
NPDC. Descriptions of the identified EBPs and IDEAL
Model components were used by the study authors during
the review and alignment phase of this analysis. The NSP
information included the name of the intervention package,
if applicable, as well as descriptive information about the
various interventions making up the package. For example,
the NSP’s “Antecedent Package” was defined as including
behavior chain interruption, behavioral momentum, choice,
and cueing/prompting, among other components. The
results of the independent ratings were analyzed for agree-
ment. Twenty-three of the 24 NSP/NPDC EBPs aligned
directly with one or more of the socially validated IDEAL
Model intervention components. Inter-rater agreement
across all intervention components was 89.4%. The research
team met to address all items for which there was less than
100% agreement among the three raters. This process clari-
fied definitions and resulted in 100% agreement among all
raters for all items.
Outcomes. A total of 31 IDEAL Model components were
determined to align directly with an NSP/NPDC EBP.
Although typically a single IDEAL Model component
aligned with just one EBP, in some cases, more than one
IDEAL Model component corresponded with a particular
Callahan et al. 193
EBP. For example, two IDEAL Model components, “The
use of visual activity schedules . . .,” and “The use of physi-
cal classroom organizers . . .,” were determined to align
with the EBP “Visual Supports” (NPDC)/”Schedules”
(NSP).
Expert validation process. The corresponding IDEAL Model
components and NSP/NPDC EBPs were systematically
validated by experts in the field of autism. Using Qualtrics
software, Version 13, of the Qualtrics Research Suite (Qual-
trics, 2013), a link to an online survey was sent to seven
persons with extensive experience and training in autism
intervention, including university faculty members teaching
autism graduate courses, and public school and center-
based practitioners with board certification in ABA and/or
formal training in the TEACCH Model. The experts were
asked to validate that the socially validated IDEAL Model
components represented the same interventions and treat-
ments within the NSP/NPDC studies. Respondents were
instructed to indicate agreement (“Yes”) or disagreement
(“No”) that the NSP/NPDC EBP referred to the same treat-
ment or intervention as conceptualized within the IDEAL
Model. For example, the experts were asked to determine
whether the IDEAL Model intervention component, “The
use of incidental teaching or naturalistic teaching . . .,” rep-
resented the same intervention as the NSP’s “Naturalistic
Teaching Strategies” and the NPDC’s “Naturalistic inter-
ventions.” All but one expert validated 100% of the items as
representing the same interventions. The overall mean per-
centage agreement of the expert validators was 99.6%
(range = 96.7%–100%), indicating a very high level of
agreement that the socially validated IDEAL Model inter-
vention components represent the same treatments empiri-
cally validated by the NSP and NPDC.
Secondary analysis. Because the procedures implemented to
determine the social validity of the IDEAL Model compo-
nents addressed only a limited aspect of social validation—
that is, the perceived value of specific intervention
components by consumers within public school autism
programs—a second analysis was conducted to determine
whether broader evidence of social validation exists for
EBPs and emerging practices in autism. For this analysis,
an expanded research team consisting of the original mem-
bers and four additional PhD-level researchers reviewed all
articles cited within the NSP and NPDC reviews for the cor-
responding socially validated IDEAL Model components.
The purpose was to determine whether the author(s) directly
addressed social validation, and to identify the types of
social validity reported. A total of 828 unique articles within
the reference lists for the relevant NSP and NPDC EBPs
were reviewed.
Articles were searched using an automated search func-
tion and at least the following search terms: valid, social,
and satis. In situations where the automated search was
inoperable or failed to locate social validity information, the
researchers reviewed the full article to identify possible ref-
erences to social validity and social validation. Researchers
were directed to search every article for language and termi-
nology not captured in the automated searches.
The research team used an intentionally conservative
approach to reliably identify direct evidence of measure-
ment of social validation. That is, the researchers coded
articles as demonstrating direct evidence of social valida-
tion only if the author(s) specifically collected social valid-
ity data. The research team only considered the explicit
language of the author(s) using the words, headings,
phrases, and/or concepts appearing in the articles which
clearly addressed or referred to social validity or social vali-
dation. Most often, this information was located within sec-
tions of the article labeled “Social Validation,” or “Parent
Satisfaction,” or similar headings. In rare cases, the review-
ers located relevant information embedded within other
sections of the articles. In all instances in which the article
authors examined social validity, at least one Reichow et al.
(2011) category was identified and coded.
Finally, to determine the types of social validation most
often addressed by autism researchers, article reviewers
assessed the author(s)’ stated evidence of social validation
and assigned the information to one or more of the Reichow
et al. (2011) categories of social validity. For example, the
statement “Social validation measures provided by 30 raters
indicated that perceived positive changes in performance
had occurred in child behaviors, and training procedures
were acceptable” could be assigned to Reichow et al.’s cat-
egories of “Clinically significant behavioral change” and
“Consumers satisfied with the results,” respectively.
Inter-rater agreement. Of the 828 articles reviewed within
the reference lists of the NSP and NPDC EBP summary
reports, a total of 201 articles (24.3%) were reviewed and
coded by at least two different researchers to determine
inter-rater reliability in assessing evidence of social validity
and agreement among the reviewers for assignments of
social validity evidence to the Reichow et al. (2011) catego-
ries. A reliability coefficient was calculated for each article
by dividing the number of rater agreements by the number of
agreements plus disagreements and multiplying the results
by 100 to obtain a percentage of agreement. Agreements and
disagreements were compared for each of the seven Reichow
et al. (2011) categories, as well as for the basic question,
“Did the article contain evidence of social validity?” An
agreement was defined as both reviewers scoring the
Reichow et al. category and answering the evidence of social
validity question the same. There was 100% agreement
regarding whether articles contained evidence of social
validity. Across all articles in which social validation was
identified and those articles in which social validity was not
194 Focus on Autism and Other Developmental Disabilities 32(3)
identified, overall inter-rater agreement on the assignment of
social validity evidence to the Reichow et al. categories was
97.4%. For those articles only in which social validity was
identified, overall inter-rater agreement was 88.4% (range =
62.5%–100%). Within these articles, inter-rater agreement
for each Reichow et al. social validity category was as fol-
lows: (a) socially relevant Dependent Variables = 84.4%, (b)
time- and cost-effective = 88.8%, (c) persons with and with-
out disabilities = 93.3%, (d) clinically significant behavioral
change = 82.2%, (e) consumers satisfied with the results =
88.8%, (f) Independent Variable by people in typical context
= 86.6%, and (g) a natural context = 84.4%. When inter-rater
agreement was not reached, the raters discussed how they
coded the specific information in the article and whether it
fit the established coding criteria. After independently re-
rating these articles, consensus was reached in all cases.
Results
IDEAL Model Alignment With NSP and NPDC
Table 1 contains a list of the 31 socially validated IDEAL
Model autism interventions that correspond directly with
EBPs and emerging treatments empirically demonstrated to
be effective for individuals with ASD. As noted above, each
IDEAL Model intervention component was systematically
determined to represent the same specific EBP or treatment/
package validated by the NPDC and NAC, respectively.
Complete verbatim descriptions of the IDEAL Model
Intervention Components in Table 1 may be found in
Callahan et al. (2008). The social validation ratings on the
31 interventions range from 5.70 to 6.69 (M = 6.17) on a
7-point scale, indicating a high to very high level of support
by key consumers for the use of these treatments in autism
programming within schools, clinics, and homes.
Table 1 includes a designated model affiliation associ-
ated with each intervention, based on earlier research
(Callahan et al., 2010). Callahan and colleagues previously
analyzed whether socially validated autism interventions
were associated with ABA, TEACCH, both models, or nei-
ther model. Prior results suggested the primary consumers
of autism programming value intervention components
inherent within both ABA and TEACCH above treatments
associated with either model alone.
Several of the socially validated EBPs were identified
by the NSP as emerging, versus established, practices.
These EBPs were “Social Skills Training Groups/Social
Skill Package,” “Computer Aided Instruction/Technology-
Aided Instruction and Intervention,” “Picture Exchange
Communication,” and “Extinction.”
Four established EBPs had more than one corresponding
IDEAL Model intervention component. “Antecedent-Based
Intervention/Antecedent Package” was represented by six
practices socially validated within the IDEAL Model,
including the use of specific motivational procedures, stu-
dent preferences, stimulus control, physical modifications
to the instructional environment, and others. The effective
use of “Reinforcement/Behavioral Package” was supported
by three socially validated interventions.
Social Validity Within NSP and NPDC EBPs
As can be seen in Table 1, the secondary analysis identified
additional evidence of social validity for all of the NSP and
NPDC EBPs aligning with the IDEAL Model. Across all
EBPs, the mean percentage of articles in which the authors
directly reported evidence of social validity was 26.7%.
However, the percentage of articles in which the authors
reported social validity was variable, ranging from a low of
2.3% for the use of extinction to 74.2% for the use of Pictorial
Story Board Simulations/Social Stories (see Table 1).
Of the Reichow et al. (2011) categories, researchers
most often addressed social validity by reporting evidence
within the categories of “Consumers Satisfied with the
Results,” “Clinically Significant Behavioral Change,” and
“Socially Important Dependent Variables.” Across all
EBPs, more than 73% of the research articles in which
social validation evidence was present addressed consumer
satisfaction with the specific EBP, while approximately half
of the articles directly addressed whether the intervention
targeted socially important variables, and whether the treat-
ment had clinically significant benefits. Relatively few
researchers addressed the time and cost-effectiveness of
interventions, and where or with whom the interventions
were implemented. Table 2 shows the overall mean percent-
age of articles containing social validation evidence for
each of the Reichow et al. (2011) categories.
Analyses of social validity of the EBPs and emerging
treatments by their affiliated model (i.e., ABA, TEACCH, or
BOTH ABA and TEACCH) were conducted. EBPs associ-
ated with the TEACCH Model (N = 6; M = 48.9%) had a
much higher overall percentage of articles addressing social
validity than treatments associated with the ABA approach
alone (N = 13; M = 17.0%) or those representing BOTH
ABA and TEACCH (N = 12; M = 26.1%). The results of a
one-way, between-subjects analysis of variance indicated a
statistically significant difference between TEACCH and
ABA, and between TEACCH and the BOTH group of inter-
ventions, F(2, 28) = 12.949, p < .000, MSerror = 161.522, α =
.05. The analysis of variance did not identify a statistically
significant difference between the ABA treatments and those
in the BOTH category.
Discussion
This research provides new information for consumers of
autism interventions. For the first time, the literature
addressing EBPs and social validity has been systematically
Callahan et al. 195
integrated to help parents, educators, therapists, and others
access research findings that include both evidence of effec-
tiveness and validation of support for the goals, procedures,
and outcomes of autism treatments. As a result, service pro-
viders and administrators now have a menu of research-
validated interventions that can serve as a blueprint for
treatment implementation, strategic planning, and program
evaluation to improve outcomes, establish training priori-
ties (Hughes, Combes, & Mehta, 2012), and build effective
communities of practice (Cook & Odom, 2013; Hall, 2015).
Adding a social validation component to efforts by the
NAC, NPDC, and others to identify and define EBPs sup-
ports long-standing and renewed calls for making social
validity a routine part of program application and evalua-
tion (Carter, 2010; Foster & Mash, 1999; Kazdin, 1977;
Reichow, 2011; Schwartz & Baer, 1991; Wolf, 1978).
Recommendations to refine and expand social validation in
behavior change research have been made and, some might
argue, ignored, for decades beginning with the seminal
work of Kazdin (1977) and Wolf (1978). Fifteen years later,
Kennedy (1992) assessed that only 20% of articles pub-
lished in the journals Behavior Modification and the Journal
of Applied Behavior Analysis addressed measures of social
validity, leading to additional questions about how best to
define, conceptualize, and measure the construct of social
validation. The results of our research suggest the leading
behavioral journals continue to report a low rate of social
validity measurement in their autism treatment studies. For
example, of 182 articles published in the Journal of Applied
Behavior Analysis, only 19 studies (10.4%) directly reported
the measurement of social validation. If social validation
research is to have a maximum impact on improving the
effectiveness of applied interventions, researchers must
make renewed efforts to address fundamental issues about
“when to use social validity measures, and what measures
are most appropriate” (Kennedy, 1992, p. 154). In this
study, the low rates of social validity observed in several of
the Reichow categories underscores the continuing need for
exploring potentially important differences in the way
researchers value, select, implement, and report social vali-
dation measures.
It is important to note that establishing the social validity
of any particular treatment is not a substitute for fully exam-
ining the intervention’s efficacy in other relevant ways
(Schwartz & Baer, 1991). Further evaluation is necessary to
help specify the salient steps or components of EBPs which
result in maximum impact and widespread adoption by
practitioners—important elements of implementation sci-
ence (Odom, 2009). In addition, research to investigate
interactions between key factors, such as the age and sever-
ity of students and clients, and the types of treatment set-
tings, is warranted.
Limitations and Future Directions
Given the processes implemented in previous studies lead-
ing to this research, wherein researchers solicited social
validation ratings of autism intervention components with
widely disseminated positive outcomes, it is not surprising
key stakeholders would broadly support their use. Thus,
additional research is needed to extend these preliminary
findings to assess consumer satisfaction with the goals, pro-
cedures, and outcomes of a larger array of practices in
homes, schools, and clinics. In addition, the conservative
approach used by this research team to identify evidence of
social validity undoubtedly resulted in under-reporting the
percentage of NSP and NPDC articles containing social
validity evidence. For example, most of the articles
reviewed detailed the results of interventions with outcomes
that could be considered to be clinically significant.
However, because no effort was made by the authors to
directly address the concept of social validity, these articles
were not coded as providing evidence of social validation.
If future research was conducted with a broader definition
of what constitutes social validity, the results would likely
indicate higher levels of social validation than reported
here. Furthermore, these results are limited to only those
studies included in the NSP and NPDC reviews. It is possi-
ble different levels of social validation could be reported in
autism research not cited by the NSP and NPDC. In addi-
tion, our analyses of the combined articles including both
the NAC packages (e.g., the 228 articles within the
Behavioral Package) with the more discrete NPDC treat-
ments (e.g., the 10 articles for Functional Behavior
Assessment [FBA]) are problematic. In this example, with
only two articles in common, the majority of the Behavioral
Package studies clearly targeted interventions other than
FBA. Future research should attempt to identify specific
Table 2. Mean Percentage of Articles Reporting Social Validity Within Reichow, Doehring, Cicchetti, and Volkmar (2011) Categories
Across All EBPs.
Socially important
dependent
variables
Time- and
cost-effective
Individuals
with/without
disabilities
Clinically
significant
behavioral change
Consumers
satisfied with
the results
IV by people
in typical
contact
A natural
context
Mean % of articles 48.4 27.4 11.5 55.1 73.3 17.4 19.7
Note. EBPs = evidence-based practices.
196 Focus on Autism and Other Developmental Disabilities 32(3)
articles within the larger NAC packages, which more
closely match the corresponding treatments in the NPDC
and IDEAL research. Finally, an important future direction
for autism treatment research is to maximize the potential
value of lists of effective interventions such as those pre-
sented here and disseminated by the NAC and NPDC. It is
potentially problematic for service providers that the listed
interventions have been classified using a variety of con-
ceptualizations and methods. Autism researchers should
identify inherent problems with such classification schemes
and move toward investigating common principles and
practices that underlie the effective application of evidence-
based treatments.
Conclusion
These results affirm that empirically demonstrated autism
treatments believed to be important by parents, teachers,
therapists, and administrators have evidence of social valid-
ity. It is encouraging that all of the EBPs identified as estab-
lished, confirmed, or emerging by the NAC and NPDC
have at least a minimal level of social validation to comple-
ment the empirical evidence of effectiveness. At the same
time, with less than 27% of the reviewed research articles
directly addressing social validity, and in light of the wide
range of reported social validity among EBPs, it is clear that
the overall evidence for social validation remains limited.
In addition, the relatively low rates of social validity for
some treatments considered to be standard parts of compre-
hensive autism programming (e.g., the use of positive and
differential reinforcement, Discrete Trial Training, and
extinction) suggests much more work needs to be accom-
plished in this important area.
Although it is beyond the scope of this article, we sup-
port the future development of improved processes, includ-
ing standardized frameworks for socially validating EBPs
in autism intervention (Schlosser, 1999). For now, however,
immediate actions can include consideration of policy
changes by the editors of journals in applied research, which
would require authors to directly and clearly report how
social validity was measured. Researchers themselves
should also begin to include descriptive social validity
information within all intervention research manuscripts
submitted for publication.
As part of the evolution of autism research and interven-
tion from a focus on what works to why and how specific
treatments can be implemented with fidelity to bridge the
research-to-practice gap, it is important that researchers and
practitioners consider social validity when making instruc-
tional and therapeutic decisions.
Acknowledgments
The authors wish to thank the expert validators for their valuable
assistance with this research.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) received no financial support for the research,
authorship, and/or publication of this article.
References
Alberto, P. A., & Troutman, A. C. (2008). Applied behavior analy-
sis for teachers (8th ed.). Columbus, OH: Pearson.
American Occupational Therapy Association. (2010).
Occupational therapy code of ethics and ethics standards
(2010). Retrieved from http://www.aota.org/Practice/Ethics.
aspx
Behavior Analyst Certification Board. (2010). BACB guidelines
for responsible conduct for behavior analysts. Retrieved from
http://www.bacd.com/index.php?page=57
Bodfish, J. W. (2004). Treating the core features of autism: Are we
there yet? Mental Retardation and Developmental Disabilities
Research Reviews, 10, 318–326.
Callahan, K., Henson, R., & Cowan, A. (2008). Social validation
of evidence-based practices in autism by parents, teachers,
and administrators. Journal of Autism and Developmental
Disorders, 38, 678–692.
Callahan, K., Shukla-Mehta, S., Magee, S., & Wie, M. (2010).
ABA versus TEACCH: The case for defining and validating
comprehensive treatment models in autism. Journal of Autism
and Developmental Disorders, 40, 74–88.
Carter, S. L. (2010). The social validity manual: A guide to sub-
jective evaluation of behavioral interventions. Boston, MA:
Academic Press.
Cook, B. G., & Cook, C. C. (2011). Unraveling evidence-based
practices in special education. The Journal of Special
Education, 47, 71–82.
Cook, B. G., & Odom, S. L. (2013). Evidence-based practices
and implementation science in special education. Exceptional
Children, 79, 135–144.
Council for Exceptional Children. (2013). Special education pro-
fessional ethical principles and practice standards for spe-
cial education professionals. Retrieved from http://www.
cec.sped.org/Standards/Ethical-Principles-and-Practice-
Standards?sc_lang=en
Dickson, C. A., MacDonald, R. P. F., Mansfield, R., Guilhardi, P.,
Johnson, C., & Ahearn, W. H. (2014). Social validation of the
New England Center for Children-Core Skills Assessment.
Journal of Autism and Developmental Disorders, 44, 65-74.
doi:10.1007/s10803-013-1852-5
Dingfelder, H. E., & Mandell, D. S. (2011). Bridging the
research-to-practice gap in autism intervention: An applica-
tion of diffusion of innovation theory. Journal of Autism and
Developmental Disorder, 41, 597–609.
Foster, S. L., & Mash, E. J. (1999). Assessing social validity in
clinical treatment research: Issues and procedures. Journal of
Consulting and Clinical Psychology, 67, 308–319.
Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood,
C., & Innocenti, M. S. (2005). Quality indicators for group
Callahan et al. 197
experimental and quasi-experimental research in special edu-
cation. Exceptional Children, 71, 149–164.
Hall, L. J. (2015). Sustaining evidence-based practices by gradu-
ated special educators of students with ASD: Creating a com-
munity of practice. Teacher Education and Special Education,
38, 28–43. doi:10.1177/0888406414558883
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., &
Wolery, M. (2005). The use of single-subject research to iden-
tify evidence-based practice in special education. Exceptional
Children, 71, 165–179.
Hughes, H. L., Combes, B. H., & Mehta, S. S. (2012). Managing
autism: Knowledge and training in autism spectrum disorders
among special education administrators in Texas. Journal of
Special Education Leadership, 25(2), 90–98.
Individuals With Disabilities Education Improvement Act (2004).
20 U.S.C., § 1415 (reauthorization of the Individuals With
Disabilities Education Act of 1990).
Kazdin, A. E. (1977). Assessing the clinical or applied signifi-
cance of behavior change through social validation. Behavior
Modification, 1, 427–452.
Kennedy, C. H. (1992). Trends in the measurement of social valid-
ity. The Behavior Analyst, 15, 147–156.
Kern, L., & Manz, P. (2004). A look at current validity issues of
school-wide behavior support. Behavioral Disorders, 30, 47–59.
Lilienfeld, S. O., Marshall, J., Todd, J. T., & Shane, H. C. (2015).
The persistence of fad interventions in the face of negative
scientific evidence: Facilitated communication for autism as a
case example. Evidence-Based Communication Assessment and
Intervention, 8, 62–101. doi:10.1080/17489539.2014.976332
Mesibov, G. B., & Shea, V. (2011). Evidence-based practices and
autism. Autism, 15, 114–133.
National Autism Center. (2009). Addressing the need for evi-
dence-based practice guidelines for autism spectrum dis-
orders (National Standards Report: The National Standards
Project). Randolph, MA: Author.
National Professional Development Center. (2013). The NPDC
on ASD and the National Standards Project. Retrieved from
http://autismpdc.fpg.unc.edu/evidence-based-practices
No Child Left Behind (NCLB) Act of 2001, Publ. L. No. 107–110,
§ 115, Stat.1425 (2002).
Odom, S. L. (2009). The tie that binds: Evidence-based prac-
tice, implementation science, and outcomes for children.
Topics in Early Childhood Special Education, 29, 53–61.
doi:10.1177/0271121408329171
Odom, S. L., Collet-Klingenberg, L., Rogers, S., & Hatton, D.
(2010). Evidence-based practices in interventions for children
and youth with autism spectrum disorders. Preventing School
Failure, 54, 275–282.
Odom, S. L., Cox, A. W., Brock, M. E., & the National
Professional Development Center. (2013). Implementation
science, professional development, and autism spectrum dis-
orders. Exceptional Children, 79, 233–251.
Qualtrics. (2013). Qualtrics (Version 13). Provo, UT: Author.
Rapp, C. A., Etzel-Wise, D., Marty, D., Coffman, M., Carlson,
L., Asher, D., . . . Holter, M. (2010). Barriers to evidence-
based practice implementation: Results of a qualitative study.
Community Mental Health Journal, 46, 112–118.
Reichow, B. (2011). Developments, procedures, and application of
the evaluative method for determining evidence-based prac-
tices in autism. In B. Reichow, P. Doehring, D. V. Cicchetti
& F. R. Volkmar (Eds.), Evidence-based practices and treat-
ments for children with autism (pp. 25–39). New York, NY:
Springer.
Reichow, B., Doehring, P., Cicchetti, D. V., & Volkmar, F. R.
(2011). Evidence-based practices and treatments for children
with autism. New York, NY: Springer.
Reichow, B., Volkmar, F. R., & Cicchetti, D. V. (2008).
Development of the evaluative method for evaluating and
determining evidence-based practices in autism. Journal
of Autism and Developmental Disorders, 38, 1311–1319.
doi:10.1007/s10803-007-0517-7
Schlosser, R. W. (1999). Social validation of interventions in aug-
mentative and alternative communication. Alternative and
Augmentative Communication, 15, 234–247.
Schwartz, I. S., & Baer, D. M. (1991). Social validity assessments:
Is current practice the state of the art? Journal of Applied
Behavior Analysis, 24, 189–204.
Wolf, M. M. (1978). Social validity: The case for subjective mea-
surement or how applied behavior analysis is finding its heart.
Journal of Applied Behavior Analysis, 11, 203–214.
Wong, C., Odom, S. L., Hume, K. A., Cox, A. W., Fettig, A.,
Kucharczyk, S., . . . Schultz, T. R. (2014). Evidence-based
practices for children, youth, and young adults with autism
spectrum disorder. Chapel Hill: Frank Porter Graham Child
Development Institute, Autism Evidence-Based Practice
Review Group, The University of North Carolina.
Wong, C., Odom, S. L., Hume, K. A., Cox, A. W., Fettig, A.,
Kucharczyk, S., . . . Schultz, T. R. (2015). Evidence-based
practices for children, youth, and young adults with autism
spectrum disorder: A comprehensive review. Journal of
Autism and Developmental Disorders, 45, 1951–1966.
doi:10.1007/s10803-014-2351-z
... Featured articles in this cluster, such as that of [51] present a systematic review on alternative and augmentative communication systems to support children with autism spectrum disorder. On the other hand, articles such as that of [52] also analyze through a review of the literature by which the social validity of evidence-based practices in children with ASD is analyzed. In his search, the authors focus on the analysis of seven categories associated with social validity. ...
... With regard to the thematic axes, the importance of social validity in the field of autism spectrum disorder is highlighted [52]. Several authors have developed tools to ensure the validity of the resources used in interventions with children. ...
... Additionally, aspects such as the effectiveness of intervention programs, such as the implementation of communication systems [21,69], reading habits and behavior management [70,71] or training programs to support people with ASD or training parents on specific learning strategies such as the studies of [20,22,27,72] have also been frequent in this field. Thus, it is recommended to pay attention and deepen its study in order to develop evidence-based practices [52]. ...
Article
Full-text available
The field of telepractice is generating increasing interest in recent years. In recent years, early childhood services have used resources such as support for interventions in families with children with autism spectrum disorder. Due to this situation, the social validity of such resources has emerged in this sector, receiving individual attention from academics and practitioners. However, a further deepening of such analyses is needed. Therefore, the main objective of this work is to analyze papers published in the Web of Science on social validity and telepractice in autism spectrum disorders. Bibliometric analysis allows us to discover the current state of a research field, to identify the main authors, articles, and topics, and to propose future lines of research to develop it further. Articles published between 2000 and 2021 were analyzed quantitatively, and by co-occurrence of words and authors. Subsequently, through bibliographic linking, the articles were grouped into different clusters. Five central themes were found, with social skills and the effectiveness of intervention programs being the most developed areas of research. Furthermore, studies focusing on evidence-based practices are necessary for the development of this research field. Thus, the analysis of social validity in the field of telepractice in children with autism spectrum disorders is a developing field within the early childhood sector.
... While behavior analysts have historically prioritized and prided themselves on developing effective treatments (Baer et al., 1968), less emphasis has historically been placed on the aspect of social validity (Foxx, 1996). Nevertheless, this is changing as behavior analysts increasingly recognize the importance of social validity as a means of attracting new consumers, sustaining relationships with existing consumers, and as an important contributor to successful treatment in its own right (Baer et al., 1987;Callahan et al., 2017;Chadwell et al., 2019;Critchfield et al., 2017;LeBlanc et al., 2020). ...
Article
Social validity refers to the extent to which applied behavior analytic (ABA) treatment goals, methods, and/or outcomes are acceptable and meaningful to stakeholders. The aim of this conceptual paper is to suggest how social validity may be promoted during behavioral consultation in school systems and how the assessment of various aspects of social validity contributes to this throughout the consultation process. Literature pertaining to the concept of social validity, rapport‐building, collaboration with stakeholders in behavioral school consultation, and the assessment of social validity are reviewed and synthesized. Social validity may be promoted in several domains: communication with consultees, collaborative selection of treatment methods and student objectives, consultee training, and measurement of outcomes meaningful to stakeholders. Social validity assessment can occur in all of these domains and is ideally a continuous process that informs all phases of consultation. There is a particular need for direct measurement of the social significance of behavioral consultation services. Social validity is integral to all aspects of the behavioral school consultation process, and it may be facilitated by incorporating valid social validity assessments into every stage of the consultation process. Social validity is integral to all aspects of the behavioral school consultation process, and it may be facilitated by incorporating valid social validity assessments into every stage of the consultation process. Social validity in schools may be promoted in several domains: communication with consultees, collaborative selection of treatment methods and student objectives, consultee training, and measurement of outcomes meaningful to stakeholders. There is a need for tools that allow for the direct measurement of social validity on a continuous basis. Social validity is integral to all aspects of the behavioral school consultation process, and it may be facilitated by incorporating valid social validity assessments into every stage of the consultation process. Social validity in schools may be promoted in several domains: communication with consultees, collaborative selection of treatment methods and student objectives, consultee training, and measurement of outcomes meaningful to stakeholders. There is a need for tools that allow for the direct measurement of social validity on a continuous basis.
... The results of our study indicate that, as a whole, researchers are still not collecting social validity data as a part of their study (Armstrong et al., 1997;Callahan et al., 2017;Ferguson et al., 2019), and more subjective than normative measures have been used (Carter & Wheeler, 2019;Ferguson et al., 2019;Ledford et al., 2016). Evidently, the value of using multiple types of stakeholder feedback to inform an intervention has yet to be discovered. ...
Article
Purpose The authors conducted a systematic review of single-case experimental designs that included individuals with autism spectrum disorder and/or intellectual disability who used speech-generating devices (SGDs) for communication. The purpose of this study was to review subjective and normative pre- and post-intervention social validity data, in addition to the cost, acceptability, and feasibility of the SGDs used in the studies. The authors also studied trends in the reporting of pre- and post-intervention data over time. Method A systematic review of 7,327 articles resulted in 86 articles that met design quality criteria and included participants who used SGDs. A group of raters completed interrater reliability for all stages of the review. Results Researchers reported more subjective than normative data. Few studies reported on the price of the SGD or the person who purchased the SGD. More researchers reported using an SGD with more than one use, but few solicited feedback about the SGD used during the intervention. Few researchers reported information about the portability of the device or the operation effort. Conclusions Reporting on social validity represents a substantial limitation in experimental studies. Future work incorporating less biased/more objective measures is important for the creation of socially valid interventions.
... Similarly, determining individuals' preferences regarding the use of AAC is also crucial to maintaining the use of such communication tools (Hyppa-Martin et al., 2016). The approval and acceptance of a particular AAC by users and their social circles and the level of satisfaction of these stakeholders are accepted as indicators of the social validity of the preferred AAC (Callahan et al., 2017). ...
Article
Full-text available
This study aimed to review the studies determining the preferences of individuals with ASD, to determine how the preference assessment sessions were conducted, and which AAC system the participants preferred to use among the commonly used AAC systems (sign language, speech-generating device, and picture exchange-based systems). In this study, descriptive analysis method was used to analyze the data. According to the findings, most of the participants with autism spectrum disorder preferred speech-generating devices compared to sign language and picture systems. In addition, most of the participants tended to prefer the augmentative and alternative communication system that they learned the fastest. The findings show that individuals with ASD mostly prefer to use speechgenerating devices when requesting an object/action. However, some questions still need to be answered to determine the AAC preferences of individuals with ASD. It is believed that answering these questions will help special education teachers, speech and language therapists, and parents to choose the most appropriate AAC system for individuals with ASD.
Chapter
Full-text available
Naturalistic Developmental Behavioral Interventions (NDBIs) are child-centered and motivation-based interventions for autistic children. Though they hold great promise due to their naturalistic implementation and focus on children’s strengths, there have been recent calls to improve NDBIs such that they are more aligned with the neurodiversity approach. Central to this argument is the notion that autistic clients and their families should find the intervention acceptable. This chapter describes how NDBIs differ from other behavioral interventions, what their strengths are, and how they can continue to be improved. More specifically, we focus on expanding target NDBI outcomes to include more distal, family centered variables (such as family quality of life); improving the social validity of NDBIs; and how NDBIs can be implemented using a tiered support system.
Article
Full-text available
The social validity of autism behavioral intervention has been questioned. Naturalistic Developmental Behavioral Interventions (NDBIs) attempt to address some concerns, but it is unclear whether autistic people consider NDBIs socially valid. Social validity of an NDBI, Pivotal Response Treatment (PRT), was investigated through autistic adults commenting on videos of autistic children receiving PRT. Qualitative coding of responses generated three themes: respect for individuals; assessment of intervention implementation; and socioemotional considerations. Although video brevity limits the scope of the present study’s conclusions, participants highlighted PRT components that appeared socially valid (e.g., reinforcing attempts, following the child’s lead) and aspects appearing invalid (e.g., overemphasis on spoken language). Therefore, adjustments appear necessary for PRT to be fully acceptable to the autistic community.
Article
Many people with learning disabilities have poor health and die younger compared to the general population so having regular health checks is very important. Sometimes doctors (general practitioners or GPs) and nurses do not know a person has a learning disability so cannot offer them a suitable health check. We asked GPs and a nurse for their views on ways they can find out who has learning disabilities and whether those ways of finding people work We found that the doctors and the nurses did not use ways to find out who has learning disabilities very often, but thought they were a good idea. Finding people with learning disabilities meant they could be offered support to attend appointments more easily. Many people with learning disabilities have poor health and die younger compared to the general population so having regular health checks is very important. Sometimes doctors (general practitioners or GPs) and nurses do not know a person has a learning disability so cannot offer them a suitable health check. We asked GPs and a nurse for their views on ways they can find out who has learning disabilities and whether those ways of finding people work We found that the doctors and the nurses did not use ways to find out who has learning disabilities very often, but thought they were a good idea. Finding people with learning disabilities meant they could be offered support to attend appointments more easily. People with learning disabilities are at increased risk of physical health conditions and mortality compared to the general population. Initiatives to address these health inequalities include the introduction of learning disability registers, through which people with learning disabilities can be identified and offered annual health checks and reasonable adjustments in their healthcare provision. A barrier to offering such initiatives to people with learning disabilities is that practitioners/providers may be unaware of the presence of a learning disability, and people who meet criteria may not be entered onto general practice learning disability registers. Screening tools can be used to help identify people with learning disabilities in order that they can be offered appropriate health services and reasonable adjustments. This study aimed to explore the awareness and views of UK primary care staff about the existence and role of screening tools for learning disability in their services. A qualitative approach was used; semistructured online interviews were conducted with a purposive sample of primary care staff recruited via a regional professional network of primary care practitioners. Five general practitioners and one nurse practitioner, representing five primary care practices in the North‐East of England, participated. Interview transcripts were analysed using thematic analysis. Two themes and associated subthemes were identified. Within Theme 1 (‘I haven't had anything much to do with them') two subthemes related to reasons why screening tools were not routinely used. Theme 2 (‘I think they're great that they exist’) comprised two subthemes that explored the benefits of screening tools, for example, for those practitioners less experienced with people with learning disabilities, and also explored potential reluctance to use them. The study found that most practitioners had some knowledge of screening tools and the general consensus was that they are beneficial. However, the tendency of participants was not to use screening tools in a consistent and/or systematic way, instead relying on informal approaches or other services for identifying the presence of learning disabilities. The study findings highlight the need for changes in practice to support primary care staff to access and systematically use evidence‐based effective and efficient screening tools for learning disability.
Article
Full-text available
Високофункционални аутизам припада групи поремећаја из спектра аутизма и не представља засебну нозолошку категорију у званичним дијагностичким приручницима, мада се врло често користи у стручном жаргону и истраживачким чланцима. Социјалне приче представљају једну од многих интервенција које се препоручују у раду са особама са аутизмом. Приче се израђују комбиновањем различитих типова реченица. Циљ овог истраживања је утврдити у којој мери социјалне приче које су студенти примењивали у раду са младим особама са високофункционалним аутизмом задовољавају постављене критеријуме, као и испитати њихову социјалну валидност. У ово истраживање биле су укључене две групе учесника – шест испитаника са високофункционалним аутизмом и 46 студената завршне године основних студија. Анализом је утврђено да све приче задовољавају критеријуме који се односе на следећа два питања: Да ли прича даје одговоре на питања шта, када, како, зашто? Да ли садржи илустрације које одговарају значењу текста? У већини прича биле су заступљене позитивне језичке формулације и обраћање у првом или трећем лицу једнине. На основу приказаних резултата примећујемо да су студентска искуства са овом интервенцијом у највећој мери позитивна. Резултати овог истраживања показују да приче у највећој мери задовољавају препоручене критеријуме и да је њихова структура у складу са онима које су приказане у иностраним истраживањима. Такође, шаљу и охрабрујућу поруку професионалцима да се опробају у изради и примени социјалних прича. Наредним истраживањима би требало планирати детаљнију процену таргетираних понашања, мерење ефеката, њихове трајности, као и могућности генерализације стечених вештина. Кључне речи: аутизам, студенти, социјалне приче.
Article
Children initiate social interactions to greet, ask questions, invite peers to play, and for many other purposes. However, failure to initiate social interactions is one of the social communication criteria that are required to diagnose autism spectrum disorder (ASD) and may limit social opportunities for children with ASD. Several researchers examined different interventions to teach social initiations to elementary‐aged children with ASD. The purpose of this systematic review was to examine the interventions that were used to teach social initiations to elementary‐aged children with ASD and their outcomes in social initiations, maintenance, generalization, and social validity. The reviewed interventions were generally helpful in teaching, generalizing, and maintaining social initiations for elementary‐aged children with ASD. Although limited, social validity evaluations support the interventions, target behaviors, and/or outcomes. Gaps and implications for professionals and researchers were discussed.
Article
Full-text available
Stereotipna ponašanja pojavljuju se kod većine osoba s poremećajem iz spektra autizma (PSA) i često znatno ometaju njihovo funkcioniranje. Cilj je ovog rada prikazati najčešće korištene bihevioralne metode i postupke za redukciju stereotipija kod djece s PSA-om te sažeti rezultate recentnih sistematskih preglednih studija u kojima je procijenjena njihova učinkovitost. Rezultati recentnih preglednih studija sugeriraju da u području tretmana motornih stereotipija najviše empirijske potpore imaju antecedentni postupci mijenjanja okoline te diferencijalno potkrepljenje alternativnog ponašanja (Akers i sur., 2020), a u području tretmana vokalnih stereotipija antecedentne metode auditorne stimulacije, kao i intervencije temeljene na posljedicama (Wang i sur., 2020). U radu se raspravlja o nedostacima dosad provedenih studija učinkovitosti te su ponuđene neke smjernice za planiranje budućih istraživanja (provedba funkcionalne analize prije implementacije tretmana, provjera integriteta provedenog tretmana, ispitivanje trajnosti postignutih učinaka, provjera učinaka tretmana na druga ponašanja, generalizacije učinaka na različite lokacije, situacije i u prisustvu različitih ljudi itd.). Nadalje, za sigurnije zaključke o dokaznoj utemeljenosti pojedinih tretmana, preporuka je planirati studije učinkovitosti koje uključuju provedbu randomiziranih kontroliranih pokusa. U kliničkoj praksi, potrebno je osigurati da se djeci s PSAom omogući pristup onim terapijskim postupcima koji su znanstveno utemeljeni, i to pravodobno te odgovarajućim intenzitetom.
Article
Full-text available
Single-subject research plays an important role in the development of evidence-based practice in special education. The defining features of single-subject research are presented, the contributions of single-subject research for special education are reviewed, and a specific proposal is offered for using single-subject research to document evidence-based practice. This article allows readers to determine if a specific study is a credible example of single-subject research and if a specific practice or procedure has been validated as "evidence-based" via single-subject research.
Article
Full-text available
Establishing a process for identifying evidence-based practices (EBPs) in special education has been a significant advance for the field because it has the potential for generating more effective educational programs and producing more positive outcomes for students with disabilities. However, the potential benefit of EBPs is bounded by the quality, reach, and maintenance of implementation. The cross-disciplinary field of implementation science has great relevance for translating the promise of EBPs into positive outcomes for children and youth with disabilities. This article examines the history, extent, and limitations of EBPs and describes the emergence and current state of implementation science as applied in special education. Subsequent articles in this special issue of Exceptional Children address a range of issues related to implementation science in special education: the research-to-practice gap, dissemination and diffusion, adherence and sustainability, scaling up, a model for state-level implementation, and fostering implementation through professional development.
Article
Full-text available
Evidence-based practices (EBPs) are instructional techniques that meet prescribed criteria related to the research design, quality, quantity, and effect size of supporting research, which have the potential to help bridge the research-to-practice gap and improve student outcomes. In this article, the authors (a) discuss the importance of clear understanding and communication regarding EBPs and what works in special education; (b) define EBPs and discuss how they are identified; (c) differentiate the term EBP from related terms such as research-based, best, and recommended practices; (d) consider the purview of EBPs; and (e) present relevant caveats related to EBPs. The authors conclude by providing recommendations to facilitate special educators' clear and effectual thinking and communication about EBPs.
Article
The increased prevalence of autism spectrum disorders (ASD) has intensified the need for high-quality special education services designed for children and youth with ASD and their families. Implementation science provides guidance for moving innovation, such as utilizing evidence-based practices for students with ASD, into regular practice in schools. The National Professional Development Center on ASD (NPDC) incorporated the principles of implementation science, the scientific knowledge about evidence-based practices, and the measurement of program quality into an intervention approach for students with ASD. This article presents the NPDC model as an example of using implementation science to build systems of professional development that increase the quality of services and promote teachers’ use of evidence-based practices.
Article
Establishing a process for identifying evidence-based practices (EBPs) in special education has been a significant advance for the field because it has the potential for generating more effective educational programs and producing more positive outcomes for students with disabilities. However, the potential benefit of EBPs is bounded by the quality, reach, and maintenance of implementation. The cross-disciplinary field of implementation science has great relevance for translating the promise of EBPs into positive outcomes for children and youth with disabilities. This article examines the history, extent, and limitations of EBPs and describes the emergence and current state of implementation science as applied in special education. Subsequent articles in this special issue of Exceptional Children address a range of issues related to implementation science in special education: the research-to-practice gap, dissemination and diffusion, adherence and sustainability, scaling up, a model for state-level implementation, and fostering implementation through professional development.
Book
Autism spectrum disorders (ASDs) have been increasingly diagnosed in recent years and carries with it far reaching social and financial implications. With this in mind, educators, physicians, and parents are searching for the best practices and most effective treatments. But because the symptoms of ASDs span multiple domains (e.g., communication and language, social, behavioral), successfully meeting the needs of a child with autism can be quite challenging. Evidence-Based Practices and Treatments for Children with Autism offers an insightful and balanced perspective on topics ranging from the historical underpinnings of autism treatment to the use of psychopharmacology and the implementation of evidence-based practices (EBPs). An evaluation methodology is also offered to reduce the risks and inconsistencies associated with the varying definitions of key autism terminology. This commitment to clearly addressing the complex issues associated with ASDs continues throughout the volume and provides opportunities for further research. Additional issues addressed include: Behavioral excesses and deficits treatment Communication treatment Social awareness and social skills treatment Dietary, complementary, and alternative treatments Implementation of EBPs in school settings Interventions for sensory dysfunction With its holistic and accessible approach, Evidence-Based Practices and Treatments for Children with Autism is a vital resource for school psychologists and special education professionals as well as allied mental health professionals, including clinical child and developmental psychologists, psychiatrist, pediatricians, primary care and community providers. © Springer Science+Business Media, LLC 2011. All rights reserved.
Article
The increased prevalence of autism spectrum disorders (ASD) has intensified the need for high-quality special education services designed for children and youth with ASD and their families. Implementation science provides guidance for moving innovation, such as utilizing evidence-based practices for students with ASD, into regular practice in schools. The National Professional Development Center on ASD (NPDC) incorporated the principles of implementation science, the scientific knowledge about evidence-based practices, and the measurement of program quality into an intervention approach for students with ASD. This article presents the NPDC model as an example of using implementation science to build systems of professional development that increase the quality of services and promote teachers' use of evidence-based practices.
Article
Supporting special educators' sustained use of evidence-based practices (EBPs) is a priority for the field. In this study, the authors used multiple measures to evaluate the first graduated cohort from a university program 6 years after graduation with a master's degree with a specialization in autism, and at least 8 years working as special educators, to determine whether they (a) remain in the field working with students with autism spectrum disorders (ASD) and (b) sustain effective practices. Survey and interview responses and content from viewed data sheets and graphs were summarized. All 12 graduates remain in the field. All continued to collect data for progress monitoring purposes and continue to use the EBPs identified by the National Professional Development Center (NPDC) on ASD with fidelity. All graduates indicated that the university program assisted in their capacity to implement and sustain their use of EBPs and attribute support from a community of practice as a sustaining factor.
Article
Communication disorder and mental health professionals may assume that once novel clinical techniques have been refuted by research, they will be promptly abandoned. Using facilitated communication (FC) for autism as a recent case example, we provide evidence to the contrary. Although FC was scientifically discredited by the mid-to-late 1990s, data we review demonstrate that it is still frequently administered in clinical and educational settings. We examine evidence for FC’s (a) continued use as an intervention for autism, (b) persistence in academic and institutional settings, (c) popularity in online and print sources, (d) promotion in the media, and (e) ongoing risk to caregivers accused of sexual abuse. We analyze the sources of these troubling developments, explore their ethical implications, and offer recommendations for addressing the spread of FC and other fad interventions.