Content uploaded by Shuqi Zhou
Author content
All content in this area was uploaded by Shuqi Zhou on Dec 20, 2024
Content may be subject to copyright.
Vol.:(0123456789)
Research in Science Education
https://doi.org/10.1007/s11165-024-10216-y
A Meta‑analysis ofSTEM Integration onStudent Academic
Achievement
ShuqiZhou1· ZehuaDong2· HuiHuiWang3· MingMingChiu4
Accepted: 10 November 2024
© The Author(s), under exclusive licence to Springer Nature B.V. 2024
Abstract
This meta-analysis examined whether learning outcomes differ (a) for STEM integration
versus traditional instruction and (b) across STEM integration implementations. Based on
79 effect sizes from 40 studies of 15,577 students, those learning via STEM integration
outperformed other students on academic achievement tests (g = 0.661; 95% CI [0.548,
0.774]). The effect sizes of STEM integration on achievement were largest for context
integration, smaller for content integration, and smallest for tool integration. They were
largest for inquiry-based learning, and progressively smaller for problem-based learning,
designed-based learning, and project-based learning. They were largest for STEM subject
achievement, and progressively smaller for science achievement, math achievement, and
engineering achievement. They were larger for collectivist countries than for individualistic
countries. Engineering design skills and grade level were not significant moderators. These
results can inform integrated STEM instructional design and improve student learning.
Keywords STEM integration· Achievement· Meta-analysis
Integrated Science, Technology, Engineering, and Mathematics (STEM) education
“combine[s] the four disciplines of science, technology, engineering, and mathematics
into one class, unit, or lesson that is based on connections among these disciplines and
real-world problems” (Moore & Smith, 2014, p.5). Past studies have shown that STEM
integration can increase students’ STEM interests (e.g., Struyf etal., 2019), promote posi-
tive attitudes toward STEM (Guzey etal., 2016), improve conceptual understanding (e.g.,
Aminger etal., 2020), foster higher-order thinking skills (English etal., 2017), raise learn-
ing outcomes (e.g., Robinson etal., 2014), and aid college preparation and STEM-related
* Zehua Dong
andyzehua@126.com
1 College ofForeign Languages, Donghua University, Shanghai, China
2 Jing Hengyi School ofEducation, Hangzhou Normal University, Hangzhou, Zhejiang, China
3 College ofAgriculture & College ofEducation, Purdue University, WestLafayette, IN, USA
4 Special Education andCounseling, Analytics\Assessment Research Centre, The Education
University ofHong Kong, PokFuLam, HongKong
Research in Science Education
career choices (NRC, 2013). Although STEM integration improved 4th grade students’ sci-
ence and mathematics achievements (Acar etal., 2018), it did not affect high school stu-
dents’ technology, mathematics, and science achievements (Merrill, 2001)—even worse, it
showed a negative effect on high school students’ science achievements (McGehee, 2015).
Different types of integrated STEM instruction might account for such results: (a) types
of integration (context, content, tool; Moore etal., 2020), (b) learning activities (inquiry-
based, problem-based, design-based, project-based; Roehrig, etal., 2021), (c) engineering
design (Bryan etal., 2016), and so on.
Moreover, past meta-analyses omitted many studies or had technical flaws. Following
Becker and Park’s (2011) systematic review of STEM integration, meta-analyses of subsets
of STEM activities on learning outcomes excluded some grade levels, academic subjects,
or regions (early childhood education, ES = 0.556, Yücelyiğit & Toker, 2021; mathemat-
ics achievement, Hedge’s g = 0.242, Siregar etal., 2019; Asian students, Cohen’s d = 0.69,
Wahono etal., 2020). Moreover, three meta-analyses had technical flaws. Saraç (2018) did
not specify clear selection criteria. Furthermore, Zeng and Yao (2020) did a limited search
for original studies, and did not consider pre-post effect sizes. Siregar etal. (2019) did not
consistently use fixed effects and mixed effects for moderators.
Thus, this up-to-date meta-analysis uses all available original studies of STEM integra-
tion and K-12 student achievement published by January 2024 to determine the effects of
STEM integration on student achievement. To account for past studies’ different results,
we test for moderator effects. These results will inform STEM education researchers and
teachers to make wiser choices as they implement STEM integration.
Theoretical Perspective
Bybee (2010) broadly defines STEM education to include any event, policy, program, or
practices with one or more of the STEM disciplines, but we prefer Vasquez etal.’s (2015)
disciplinary knowledge continuum of a single discipline, multidisciplinary, interdiscipli-
nary, and transdisciplinary. Most classes stick to a single subject like math. By contrast,
students tackle separate subjects before combining them to solve a problem in multidis-
ciplinary approaches. Interdisciplinary approaches goes a step further, blending concepts
across fields. Lastly, students combine disciplines to address real-world problems in trans-
disciplinary approaches. In this paper, we embrace a transdisciplinary definition of STEM
integration as mixing two or more STEM disciplines to address real-world problems. For
this study, learning outcome (aka achievement) captures how well a student has accom-
plished specific learning goals via STEM integration.
STEM Integration andLearning
STEM integration grabs students’ interest, which motivates them to sharpen their higher-
order thinking skills and learn more. Each subject has its fans, but when we mix them,
more students are interested in at least one of them. Some are hooked by the real-world
STEM problem or its relevance, even if the subjects do not grab them. Solving a real prob-
lem can be satisfying and have real-world benefits, which can motivate still more students.
So, compared to a regular school task in one subject, a STEM project gets more students
excited and eager to learn (Struyf etal., 2019). Also, STEM projects offer more to learn
Research in Science Education
than single-subject tasks. Moreover, real-world problems are messier and more challenging
than school exercises, offering richer learning experiences (though they can be overwhelm-
ing). As these projects engage more students and provide more learning opportunities, stu-
dents can learn more from them than from traditional schoolwork (Robinson etal., 2014).
Hence, we propose hypothesis H-1.
H-1. Students in STEM integration activities outperform other students on learning out-
comes.
Moderators
Many scholars are studying STEM education, but they do not agree on a single framework or
clear definition of STEM integration (Moore etal., 2020). Some researchers detail their inte-
grated STEM learning units but not the instructional principles behind their design (e.g., Barrett
etal., 2014; Gentile et al., 2012). Conversely, others give detailed accounts of their teaching
methods but lack a theoretical basis (e.g., Moore & Smith, 2014). Hence, Moore etal.’s (2020)
reviewed the literature to see how researchers define and use STEM integration. They synthe-
sized 109 articles, noting that most agree on the importance of tackling real-world problems.
However, views differ on: (a) the degree of integration, (b) learning activities, (c) the number
of included disciplines, (d) implementation strategies, and (e) the roles of individual disciplines
(Moore etal., 2020). The original studies had enough data to test the degree of integration and
learning activities for moderation effects (but lacked information on the number of disciplines,
roles of each discipline, and implementation strategies). Thus, we test whether these moderators
and others (demographics, study design) account for the different effect sizes in past studies.
Degree ofIntegration: Context, Content, andTool Integration
STEM integration can occur across contexts, content or tools. Context integration uses a
meaningful, relevant, concrete, authentic problem situation in one subject to learn about
another (Bryan etal., 2016; Moore & Smith, 2014). For example, a teacher (Ms. T) teaches
her students about temperature (physics). She asks them to find the temperature ranges at
which poodles thrive (biology). As Ms. T brings real life into the classroom, learning about
temperature becomes meaningful to her students, so they are more eager to learn (Nadelson
& Seifert,2017). Furthermore, they create more neural connections about temperature, cats
and dogs in their brains, which aids their later recall and novel uses (Li & Wang, 2021).
By using these ideas across diverse problems, they learn how to apply them in new ways
(transfer, Perkins & Salomon, 2012).
Whereas context integration only uses a context from another subject, content inte-
gration mixes ideas across subjects (Wang & Knobloch, 2018, 2022). For example, Ms.
T asks her students to model heat indices (H) of temperature (T) and relative humidity
(R) across environments that poodles can tolerate. To understand their relations, students
must integrate physics and math (H = f[T, R]), which helps them see how these subjects fit
together. Hence, they also build more neural connections across subjects as well as with the
real-world situation (Li & Wang, 2021). All of this can help them solve complex problems
(Siverling etal., 2019). (Roehrig etal. [2021] argued for intertwining of content and con-
text integration to make STEM integration more meaningful to students.)
Research in Science Education
During tool integration, teachers help students learn one subject with a novel tool from
another one (without necessarily adding its context or content; Moore etal., 2020). For
example, Ms. T asks her students to use thermocouples(Bajzek, 2005) to measure tem-
peratures in different places—kitchen, pool, beach—and think about where their poodles
can live. Thus, students learn how to use tools to solve real-world problems. Learning
how to use the thermocouple to measure the temperature was the focus of the instruc-
tion.Although chemistry knowledge (esp. relation between thermocouples’s two metals)
spurred the invention of the thermometer, Ms. T does not explain its origin, so her students
do not learn the links between mercury and temperature.
No past study showed whether the degree of STEM integration affects student learn-
ing. As context and content integration offer more links across subjects than tool integra-
tion, student can learn more from the first two. Moreover, teachers might plan for and use
context integration more easily than content integration. As many teachers did not study a
STEM subject in college, they do not know enough STEM content or feel confident enough
(self-efficacy) to guide students in project-based learning or design-based learning (Brophy
etal.,2008; Frank etal., 2003; Nadelson etal., 2016). For example, some teachers do not
know enough about engineering to include it in their lessons (Hamad etal., 2022). Time
and curriculum limits (Helle etal., 2006) also obstruct content integration (e.g., integrating
math into science/engineering classes; Roehriget al.,2021).
In short, (a) context integration improves learning outcomes and is easy to use, (b)
content integration is harder to use but still aids learning, and (c) tool integration offers
the least benefit. While context integration, content integration, and tool integration can
overlap, they differ substantially. So, we test whether context, content, or tools integration
accounts for the differences in results across studies. Thus, we propose this hypothesis.
H-2. Student learning outcomes are highest for context integration, lower for content
integration, and lowest for tool integration.
Learning Activities
Some teachers use student-centered design (i.e., problem-based learning, project-based
learning, design-based learning) to help students learn. They ask them to solve complex,
real-world problems with scientific methods or engineering design processes (Bryan etal.,
2016; Wang & Knobloch, 2022). In these lessons, students learn and apply cross-cutting
concepts, think critically, and solve problems across contexts (e.g., Margot & Kettler, 2019;
Robinson etal., 2014). They connect concepts and practices across subjects through a com-
mon problem (Roehrig, etal., 2021; Vasquez etal., 2013) to learn more efficiently (e.g.,
You, 2017). Teachers have taught STEM integration via problem-based, inquiry-based,
project-based or design-based learning (Moore, etal., 2020). In all of these methods, stu-
dents build understanding and meaning from prior experiences, supported by tools in their
teachers’ learning environment (constructivism; Cobern, 1993).
Students can learn by working on a complex problem (problem-based learning; Serv-
ant-Miklos, 2020). For example, hot, humid days in Phoenix, Arizona can harm poodles
(and humans!). So, students tried to identify such days. They measured the temperature,
humidity, wind speed, and cloudiness at noon each day. Then, they brought their poodle
outside and recorded how quickly it sought the cool refuge of their air-conditioned class-
room. They identified undesirable weather combinations and shared their findings with the
local newspaper to help their community. During problem-based learning, students face a
Research in Science Education
problem, think critically, gather more information, try out solutions, and share their results
en route to learning both subject content and problem solving processes (e.g.,Yew & Goh,
2016).
During inquiry-based learning, students act like scientists (Pedaste etal., 2012). For
example, they wondered what affects poodles’ discomfort on hot day? Perhaps heavier poo-
dles suffered more? To test this idea, they gathered several poodles, measured the tempera-
ture at noon, brought them all outside, and recorded how quickly they went back inside.
In short, they ask questions, generate hypotheses, design experiments, collect and analyze
data, and accept or reject their hypotheses (Pedaste etal., 2012).Like problem-based learn-
ing, inquiry-based learning deals with real, relevant problems. The teacher scaffolds stu-
dents, helping them make sense of their ideas, develop explanations based on evidence, and
share their ideas (Hmelo-Silver etal., 2007). But unlike problem-based learning, inquiry-
based learning pushes students into scientific discovery processes (Pedaste etal., 2012).
In project-based learning, students take on bigger real-world challenges and often cre-
ate a product (Hess etal., 2016), like making a doghouse for their poodle. These students
discussed criteria for a comfortable doghouse, evaluated many doghouse designs before
choosing one, bought materials, built the doghouse, and observed how their poodle used
it. During such longer, personally meaningful projects, students have more autonomy to
address complex issues, often needing support and input from peers and their teacher
(Kokotsaki etal., 2016).
Design-based learning is a subset of project-based learning in which students learn and
apply theoretical knowledge to address larger problems and design solutions (Puente etal.,
2013). For example, Ms. T asked groups of students to design cooling mats for their poo-
dles. Scaffolded by Ms. T as needed, they researched mats and possible materials (esp.
their heat transfer), specified design criteria, brainstormed, built prototypes, and tested
them with their poodle. Then, they refined their cool mat design via cycles of redesign,
building, testing, and evaluating. Lastly, they shared their best cooling mat with their
class. During the design process, their teachers help students identify/clarify the problem;
gather information about it; develop possible solutions; select the best one(s); construct
prototype(s); and cycle through testing, evaluating, and revising, before explaining their
final solution (Massachusetts, 2006).
Many teachers struggle with projects and design activities, which are typically much
longer and more complex than problem-based or inquiry-based learning activities. Such
teachers often do not have enough STEM knowledge (Brophy et al., 2008; Dare et al.,
2018), experience with project-based or design-based curricula (Frank etal., 2003; Nadel-
son etal., 2016), time, or curriculum flexibility (Helle etal., 2006). These hurdles obstruct
them from helping their students complete these complex activities and learn from them
(Lewis etal., 2021). By contrast, problem-based and inquiry-based learning are simpler to
manage, pop up more often in classrooms (Albion, 2015), and yield more student learning
(Panasan & Nuangchaler, 2010). Thus, we propose this hypothesis:
H-3. Students in problem-based or inquiry-based lessons outperform students in project
or design-based lessons during STEM integration.
Engineering Design Skills
Scholars argue that integrating an iterative, engineering design process into K-12 STEM
education helps students learn engineering design skills to tackle real-world problems:
Research in Science Education
identify engineering constraints, propose engineering solutions, make decisions, and com-
municate effectively (e.g., Bryan etal., 2016; Guzey et al., 2020; Moore etal., 2020).
These students learn to (a) link science and math through engineering principles and (b)
crafting evidence-based engineering solutions for complex problems (Guzey etal., 2020).
As scientific inquiry focuses on understanding phenomena and engineering design focuses
on creating solutions (Ali & Tse, 2023), the Next Generation Science Standards (NGSS)
in the United States urged their integration for greater resource and time efficiency (Dare
etal., 2018; NRC, 2013). So, science teachers (unlike other teachers) did so for their cur-
ricula (Ross etal., 2018). However, many teachers do not have enough engineering design
expertise to implement it (Hamad etal., 2022), which can reduce its impact on student
learning. Thus, we propose this hypothesis:
H-4. The STEM integration-achievement link is not stronger in interventions with engi-
neering design skills.
Cultural Values
Cultural values vary across countries. Some societies favor group interests over individual
interests (collectivism), while others favor individual interests (individualism; LeTendre
et al., 2003). In collectivist cultures, people pay more attention to one another, so they
work together more effectively to attain group goals (e.g., collaborative learning with class-
mates; Chiu & Chow, 2015). Indeed, classmates’ mean past achievement is linked to a stu-
dent’s current achievement more strongly in collectivist cultures than in individualist ones
(Chiu & Chow, 2015). As STEM integration often includes group activities, classmates
might collaborate more effectively and learn more in collectivist cultures than in individu-
alist ones (e.g., McAtavey & Nikolovska, 2010; Tian etal., 2015). However, students in
collectivist cultures avoid criticizing their peers to save face (Wong, 2011), which hinders
their collaborative problem-solving (Wu etal., 2022). Hence, this remains an open ques-
tion that we test in this study.
H-5: The STEM integration-achievement link is stronger in collectivist cultures than
individualist ones.
Achievement Measures: Academic Subject andStandardized Tests
Achievement measures can differ across academic subjects and are not always standard-
ized tests. Researchers often create custom tests to match their learning activities, so their
greater coherence can yield stronger effect sizes. Conversely, using off-the-shelf tests that
do not align as well with the activities can yield weaker effect sizes. For example, Fan
and Yu (2017) designed a STEM unit and a corresponding conceptual test to assess learn-
ers’ understanding of mechanics, science, and math. Also, students more readily perceive,
interact with, enjoy, learn from and remember activities that are more visible, accessible,
and concrete (e.g., biology) rather than abstract ones (e.g., physics; Dong etal., 2024).
Thus, we test whether measures of achievement content (STEM and academic subjects) are
significant moderators.
Unlike one-time, customized tests for a research study, some tests are psychometrically
validated and standardized, with uniform administration, scoring, and interpretation pro-
cedures (e.g., National Assessment of Educational Progress [NAEP] or Programme for
Research in Science Education
International Student Assessment [PISA]). NAEP assesses student achievement across sub-
jects (NCES, 2024). PISA measures 15-year-olds’ ability to use their reading, mathemat-
ics, and science knowledge and skills to meet real-life challenges (OECD, 2024). Hence,
non-standardized tests like class tests are often more closely aligned than standardized tests
to instruction and might yield larger intervention effect sizes (Li & Ma, 2010). Thus, we
tested whether standardized (vs. unstandardized) tests are significant moderators.
Other Study Differences: Designs, Intervention Duration, andParticipant Grade
Levels
Following standard quality control protocols for meta-analyses, we also test for differences
across study designs, intervention durations, and grade levels. Study designs included: (a)
a controlled experiment with both pre- and post-tests (e.g., Hsiao etal., 2017), (b) only the
experimental group’s pre- and post-tests (which ignores informal learning across time, e.g.,
Pozarski Connolly, 2017), or (c) only post-tests of control and experimental groups (which
ignores their prior learning, e.g., Capobianco etal., 2021. As controlled experiments likely
show less bias (e.g., maturation bias, history bias; Kirk, 2009) than others (Torday &
Baluška, 2019), study design might moderate the STEM integration-achievement link.
Longer intervention duration allows for more learning time and better learning out-
comes (e.g., Wahono etal. [2020]). However, it also allows for more external events that
interfere with learning and drive weaker results (Nahmias etal., 2019). Hence, the impact
of intervention duration on learning is unclear.
Past studies have not hypothesized how grade level (e.g., middle school vs. high school)
might moderate the effect of STEM integration on learning outcomes. Still, following com-
mon meta-analysis practices, we test grade level as a moderator.
Present Study
This meta-analysis determines the overall link between STEM integration and student
achievement. Furthermore, we test whether the following moderators account for past stud-
ies’ different results: degree of STEM integration, learning activities, engineering design
skills, cultural values, academic subject, standardized test, study design, intervention dura-
tion, and grade level.
Method
Literature Search Procedure
We used the PRISMA guidelines in our literature search and selection processes (see Fig.1,
Moher etal., 2009). To identify original studies up to January 8, 2024 inclusive, we searched
six electronic databases: Education Resources Information Center (ERIC), PsychINFO,
JSTOR, Web of Science, Institute of Electrical and Electronics Engineers (IEEE) Xplore,
ProQuest Dissertations and Theses. Our searches used these keyword combinations: (student
success, student achievement, academic achievement, or student performance), AND (STEM
Research in Science Education
integration, or integrated STEM). We checked the references of published studies of STEM
integration but did not find additional original studies. Then, we removed duplicates.
Inclusion Criteria
We included articles that met all six of the following criteria (see Table1). First, the study
was related to STEM integration. Second, the study reported at least one measured student
achievement outcome. Third, the study was a controlled experiment (post-tests only or both
pre- and post-tests) or a treatment only condition with both pre- and post-tests. Fourth, the
study reported means, standard deviations, t values, F values, or other sufficient information to
compute effect sizes. Fifth, the study specified the type of STEM integration (e.g., Olivarez,
2012; Williams, 2020). Our final data set has 79 effect sizes from 40 studies.
Records identified through database
searching
(n = 1,175)
ScreeningIncluded Eligibility Identification
Additional records identified
through other sources
(n = 0)
Records after duplicates removed
(n = 1,170)
Records screened
(n = 1,170)
Records excluded
(n = 982)
Full-text articles assessed
for eligibility
(n = 188)
Full-text articles excluded,
with reasons
(n =148)
Studies included in
quantitative synthesis
(meta-analysis)
(n = 40)
Fig. 1 PRISMA 2009 Flow Diagram (Moher etal., 2009)
Research in Science Education
Table 1 Studies Included in the Meta-analysis
Study Effect size (g) Degree of
integration
Learning
ActivityaEngineering
design skill
Intervention
Duration
Cultural valuesbAcademic subjectcGrade
LeveldStudy DesigneStandard-
ized test or
notf
Acar etal., 2018 1.958 Content NA No NA Collect. Science Element. 1 2
Acar etal., 2018 1.473 Content NA No NA Collect. Science Element. 1 2
Acar etal., 2018 0.299 Content NA No NA Collect. Math Element. 1 2
Acar etal., 2018 1.556 Content NA No NA Collect. Math Element. 1 2
Akkaya & Benzer, 2020 2.044 Content NA Yes 4–8weeks Collect. Science Middle 1 2
Alameh, 2018 0.379 Content 3 Yes 4–8weeks Indiv. Science Middle 1 2
Alameh, 2018 0.051 Content 3 Yes 4–8weeks Indiv. Science Middle 1 2
Angwal etal., 2019 1.650 Context mix Yes NA Collect. Science High 1 2
Anwar etal., 2022 0.742 Content 2 Yes ≤ 4weeks Indiv. Science Middle 1 2
Capobianco etal., 2021 1.656 Content 4 Yes ≤ 4weeks Indiv. Science Element. 3 2
Capobianco etal., 2021 1.514 Content 4 Yes ≤ 4weeks Indiv. Science Element. 3 2
Cotabish etal., 2013 1.192 Content 3 No > 8weeks Indiv. Science Element. 1 2
Fan & Yu, 2017 1.199 Content 4 Yes > 8weeks Collect. STEM High 1 2
Gazibeyoglu & Aydin,
2019
0.934 Context 3 No 4–8weeks Collect. Science Middle 2 2
Guzey etal., 2017 0.261 Content 4 Yes > 8weeks Indiv. Engineer Element. 1 2
Guzey etal., 2017 0.047 Content 4 Yes > 8weeks Indiv. Math Element. 1 2
Guzey etal., 2017 0.381 Content 4 Yes > 8weeks Indiv. Science Element. 1 2
Guzey etal., 2017 0.566 Content 4 Yes > 8weeks Indiv. Science Element. 1 2
Guzey etal., 2017 0.320 Content 4 Yes > 8weeks Indiv. Science Element. 1 2
Guzey etal., 2017 0.136 Content 4 Yes > 8weeks Indiv. Science Element. 1 2
Guzey etal., 2017 −0.383 Content 4 Yes > 8weeks Indiv. Math Element. 2 2
Guzey etal., 2017 0.233 Content 4 Yes > 8weeks Indiv. Engineer Element. 1 2
Guzey etal., 2017 0.128 Content 4 Yes > 8weeks Indiv. Math Element. 1 2
Research in Science Education
Table 1 (continued)
Study Effect size (g) Degree of
integration
Learning
ActivityaEngineering
design skill
Intervention
Duration
Cultural valuesbAcademic subjectcGrade
LeveldStudy DesigneStandard-
ized test or
notf
Guzey etal., 2017 0.752 Content 4 Yes > 8weeks Indiv. Science Element. 1 2
Guzey etal., 2017 0.236 Content 4 Yes > 8weeks Indiv. Science Element. 1 2
Guzey etal., 2017 0.315 Content 4 Yes > 8weeks Indiv. Science Element. 1 2
Guzey etal., 2017 0.517 Content 4 Yes > 8weeks Indiv. Science Element. 1 2
Guzey etal., 2017 −0.204 Content 4 Yes > 8weeks Indiv. Math Element. 2 2
Hsiao etal., 2017 0.466 Content 4 Yes ≤ 4weeks Collect. STEM High 1 2
Izgi & Kalayci,2020 1.326 Context 3 No 4–8weeks Collect. Science Middle 1 1
Jahan, 2018 0.018 Content 2 Yes > 8weeks Indiv. Science High 3 1
Jahan, 2018 0.241 Content 2 Yes > 8weeks Indiv. Science High 3 1
Jahan, 2018 0.640 Content 2 Yes > 8weeks Indiv. Science High 3 1
James, 2014 0.297 Content 1 No NA Indiv. Math Middle 2 1
James, 2014 0.443 Content 1 No NA Indiv. Science Middle 2 1
Kağnıcı & Sadi, 2021 1.103 Context 3 No NA Collect. Science High 1 2
Kelley etal., 2023 0.470 Content 2 Ye s > 8weeks Indiv. STEM High 1 2
Kurt & Benzer, 2020 1.489 Content 4 No 4–8weeks Collect. Science Middle 1 2
Li etal., 2016 0.037 Context 4 Yes NA Collect. Science Element. 1 2
Li etal., 2018 0.364 Tool 2 No NA Indiv. Math High 1 2
Li etal., 2018 0.513 Tool 2 No NA Indiv. Science High 1 2
Lie etal., 2019 0.140 Content 4 Yes > 8weeks Indiv. Engineer Element. 3 2
Lie etal., 2019 0.246 Content 4 Yes > 8weeks Indiv. Engineer Middle 3 2
Lie etal., 2019 0.228 Content 4 Yes > 8weeks Indiv. Engineer Middle 3 2
Lie etal., 2019 0.045 Content 4 Yes > 8weeks Indiv. Engineer Middle 3 2
Lippert & Seals, 2023 0.170 Content 2 Yes 4–8weeks Indiv. Math Middle 1 2
McClain, 2015 0.260 Context 2 No > 8weeks Indiv. Math Element. 1 1
Research in Science Education
Table 1 (continued)
Study Effect size (g) Degree of
integration
Learning
ActivityaEngineering
design skill
Intervention
Duration
Cultural valuesbAcademic subjectcGrade
LeveldStudy DesigneStandard-
ized test or
notf
McGehee, 2015 0.218 NA 2No NA Indiv. Math High 2 1
McGehee, 2015 −0.055 NA 2No NA Indiv. Science High 2 1
McGehee, 2015 −0.069 NA 2No NA Indiv. Math High 2 1
McGehee, 2015 −0.153 NA 2No NA Indiv. Science High 2 1
McHugh, 2016 1.876 Content 3 No > 8weeks Indiv. Science High 1 2
McHugh, 2016 −0.221 Content 3 No > 8weeks Indiv. Science High 2 1
Parlakay & Koç, 2020 0.559 Context 2 No NA Collect. Science Middle 1 2
Pozarski Connolly, 2017 1.266 Content 3 Yes > 8weeks Indiv. Science Middle 3 2
Pozarski Connolly, 2017 0.669 Content 3 Yes > 8weeks Indiv. Science High 3 2
Rehmat & Hartley, 2020 2.275 Context 1 Yes > 8weeks Indiv. STEM Element. 1 2
Rehmat & Hartley, 2020 1.294 Context 1 Yes > 8weeks Indiv. STEM Element. 1 2
Robinson, 2016 0.536 Content 2 Yes 4–8weeks Indiv. Math Middle 3 2
Robison etal., 2014 1.513 Content 3 Yes 4–8weeks Indiv. Science Element. 1 2
Robison etal., 2014 2.616 Content 3 Yes 4–8weeks Indiv. Science Element. 1 2
Robison etal., 2014 2.230 Content 3 Yes 4–8weeks Indiv. Science Element. 1 2
Sabag & Trotskovsky,
2013
0.533 Content NA No NA Collect. Math High 3 2
Sarican & Akgunduz,
2018
0.381 Content 4 Yes 4–8weeks Collect. Science Element. 1 2
Sauder, 2023 0.802 Content 2 Yes ≤ 4weeks Indiv. Science Element. 1 2
Shelden, 2021 0.044 Content 2 Yes 4–8weeks Indiv. Math Element. 3 1
Shelden, 2021 0.159 Content 2 Yes 4–8weeks Indiv. Math High 3 1
Srihongsa etal., 2017 2.150 Content NA Yes 4–8weeks Collect. Science Element. 3 2
Stitham, 2018 0.466 Content mix No > 8weeks Indiv. Math Element. 1 1
Research in Science Education
Table 1 (continued)
Study Effect size (g) Degree of
integration
Learning
ActivityaEngineering
design skill
Intervention
Duration
Cultural valuesbAcademic subjectcGrade
LeveldStudy DesigneStandard-
ized test or
notf
Sunyoung etal., 2016 0.351 Content 2 No > 8weeks Collect. Math High 1 1
Tati etal., 2017 0.801 Content 2 Ye s NA Collect. Science Middle 2 2
Tati etal., 2017 0.876 Content 2 Ye s NA Collect. T & Engineer Middle 2 2
Tati etal., 2017 0.820 Content 2 Ye s NA Collect. Math Middle 2 2
Tsai etal., 2021 0.547 Content 3 No 4–8weeks Collect. Science Middle 1 2
Vallera&Bodzin 2020 1.136 Context 2 No ≤ 4weeks Indiv. STEM Element. 1 2
Yaki etal., 2019 0.329 Content 4 Yes 4–8weeks Collect. Science High 1 2
Yaki etal., 2019 1.277 Content 4 Yes 4–8weeks Collect. Science High 1 2
Yaki etal., 2019 1.596 Content 4 Yes 4–8weeks Collect. Science High 1 2
Hasançebi etal., 2021 0.795 NA 3No NA Collect. Science Middle 1 2
Note. a1 = problem-based learning, 2 = project-based lear ning, 3 = inquiry-based learning, 4 = design-based learning; b Collect. = Collectivism, Indiv. = Individualism.cT = tech-
nology, Engineer = engineering; d Element. = elementary school, middle = middle school, high = high school; e1 = Pre-post experimental and control, 2 = Control and experi-
mental group’s posts, 3 = Experimental group’s pre and post; f T = technology, Engineer = engineering;d1 = standardized, 2 = non-standardized
Research in Science Education
Coding
Three authors coded the included studies according to the above moderators:
author, learning activity (problem-based learning, project-based learning, inquiry-
based learning, and design-based learning), intervention duration(≤ 4 weeks,
4–8 weeks, > 8 weeks), degree of integration (content integration, context integra-
tion, and tool integration), academic subjects [e.g., science, math, engineering design,
STEM], standardized test (or non-standardized), cultural values (collectivist vs. indi-
vidualist), engineering design skill (yes or no), and grade level (elementary school,
middle school, high school). See Table1. Although most studies explicitly specified
the learning activity as either problem-based learning, project-based learning, inquiry-
based learning, or design-based learning, some studies used other terminology, so
we carefully read their definitions and coded them accordingly (e.g., Angwal etal.’s
[2019] 5E model is a type of inquiry-based learning). Notably, if a study included more
than one subject, we coded it as STEM. Also, many of these interventions covered a
few lessons (days), a science unit (four or more weeks), or a semester (or longer), so
we categorized interventions into three categories: less than 4weeks, 4–8weeks, and
more than 8weeks.
Initially, three authors independently coded one study in the first week, another two
studies in the second week, and another two studies in the third week. Each week, they
discussed coding discrepancies, resolved the disagreement by consensus, and speci-
fied/updated definitions of codes. Then, they independently coded randomly assigned
studies. Next, the other two authors checked each author’s independent coding. These
three authors resolved all disagreements through discussion and consensus.
Calculation ofEffect Size
We used the effect size Hedge’s g. For studies with only the experiment group’s pre-
and post-test scores, we computed its standardized mean difference of test scores
before and after STEM intervention:
g
=
(
1−
3
4df −1
)d
, with
d
=
x
post
−x
pre
S
within
,
S
within =
S
diff
√
2
(
1
−r)
. For studies with only post-test scores of the experiment and control groups,
g
=
(
1−
3
4df −1
)
d , where
d
=
x
1
−x2
S
within
,
S
within =
√
(n1−1)S2
1+(n2−1)S2
2
n1+n2−2 , with the two groups’
respective sample means
x1
and
x2
, standard deviations S1 and S2, and sample sizes n1
and n2 (Borenstein etal., 2021).
For studies with both pre- and post-test scores of the experiment and control groups,
g
=
(
1−
3
4
df −
1
)
d , where
d
=
(x
1_post
−x
1_pre
)−(x
2_post
−x
2_pre
)
SD
post
, and
SD
post =
√
(n2_post−1)S2
2_post+(n1_post −1)S2
1_post
n2_post+n1_post −2 , with the respective mean scores of the pretests and
posttests of the experimental group
x
1_
pre and x
1_
post
and of the control group
x2_pre and x2_post
(Sung etal., 2016).
We used the inverse-variance weights to calculate the overall effect size. For studies
with two or more control groups, we treated them independently. If a study was pub-
lished both as a journal article and as a dissertation, we used the journal article results.
Research in Science Education
Statistical Analyses
We analyzed these data with Comprehensive Meta Analysis software (CMA 3.0). As
these studies used a wide range of STEM intervention approaches and sample types, we
used a random effects model with inverse-variance weights to calculate the mean effect
size (Borenstein etal., 2010). A statistically significantQstatistic or large I2 indicates
effect size heterogeneity (Higgins etal., 2003).
We tested for publication bias with a funnel plot, Egger’s regression, and fail-safe
number (Nfs). The following results would suggest a low risk of publication bias: (a)
effect sizes are mostly within the funnel and show axis symmetry; (b) Egger’s linear
regression intercept is not significant and near-zero; and (c) Nfs > 5k + 10 (k = number
of original studies, Rothstein etal., 2005).
Results
Overall Effect Size
STEM integration has a positive effect on student achievement (g = 0.661, 95% CI
[0.548,0.774], p < 0.001), supporting H-1 (students in STEM integration activities out-
perform other students on learning outcomes). The effect size varied significantly across
studies (Q[78] = 2,954, p < 0.001; I2 = 97, τ2 = 0.222, see Table2). See the forest plot in
Fig.2.
Publication Bias
The funnel plot, Egger’s regression, and Nfs showed no evidence of publication bias. The
effect sizes were mostly within their funnel without any sharp asymmetry (see Fig.3).
Egger’s regression showed a non-significant intercept value of 1.429 (95% CI [−0.606,
3.464], p > 0.05). Nfs(40,437) far exceeded its respective threshold (405 = 5*79 + 10).
Moderator
Degree of STEM Integration The degree of integration moderated the link between
STEM integration and achievement (Qbetween = 7.227, df = 2, p < 0.05, see Table3). Specifi-
cally, context integration yielded the largest effect size (g = 1.063, 95% CI [0.668, 1.457],
p < 0.001) followed content integration (g = 0.654, 95% CI [0.528, 0.780], p < 0.001) and
then tool integration (g = 0.441, 95% CI [0.207, 0.676], p < 0.001), supporting H-2 (signifi-
cant differences in integrations: context vs. content vs. tool).
Learning Activities The STEM integration-achievement link differed across learning
activities (Qbetween = 21.347, df = 3, p < 0.001, see Table 3). Specifically, inquiry-based
learning produced the largest effect size (g = 1.051, 95% CI [0.754, 1.349], p < 0.001), fol-
lowed closely by problem-based learning (g = 0.941, 95% CI [0.561, 1.321], p < 0.001).
Design-based learning (g = 0.442, 95% CI [0.294, 0.589], p < 0.001) and project-based
learning (g = 0.401, 95% CI [0.255, 0.547], p < 0.001) both showed much smaller effect
Research in Science Education
Table 2 Random Model of the STEM Integration’s Effect on Student STEM Achievement
Note. *p < .001
Weighted mean g k Effect size and 95%
interval
Test of null
(2-Tail)
Heterogeneity Tau-squared
LL UL z-Value Q-Value I-squared Tau Squared SE Tau
0.661 79 0.548 0.774 11.463* 2954.147*97.360 0.222 0.073 0.471
Research in Science Education
Fig. 2 Forest Plot for the Random-effects Model.Note: This figure is an output from CMA software. CMA
uses “a”, “b”, “c” etc., after the authors to differentiate different dataused in the same study
Research in Science Education
sizes. These results support H-3 (students in problem-based or inquiry-based learning
activities outperform other students).
Engineering Design Skills Whether engineering design skills were included (or not) did
not affect the STEM integration-achievement link (Qbetween = 0.018, df = 1, p = 0.892, see
Table3), showing no support for H-4.
Cultural Values Cultural values also moderated the STEM integration-achievement link
(Qbetween = 11.366, df = 1, p < 0.01, see Table 3). Specifically, collectivist culture yielded
a larger effect size (g = 1.007,95% CI [0.760, 1.253], p < 0.001) than an individualistic
culture (g = 0.526, 95% CI [0.394, 0.657], p < 0.001), supporting our H-5 that the STEM
integration-achievement link is stronger in collectivist cultures than in individualistic ones.
Academic Subject Academic subjects moderated the STEM integration-achievement link
(Qbetween = 99.575, df = 4, p < 0.001, see Table3). Specifically, STEM subject achievement
produced the largest effect size (g = 1.107, 95% CI [0.645, 1.570], p < 0.001) followed
by science achievement (g = 0.843, 95% CI [0.679, 1.008], p < 0.001), math achievement
(g = 0.240, 95% CI [0.091, 0.389], p < 0.01), and engineering achievement (g = 0.203, 95%
CI [0.122, 0.285], p < 0.001).
Standardized Test Test standardization moderated the STEM integration-achievement
link (Qbetween = 28.395, df = 1, p < 0.001, see Table 3). Specifically, the effect sizes were
larger for non-standardized tests (g = 0.784, 95% CI [0.645, 0.923], p < 0.001) than for
standardized tests (g = 0.229, 95%CI[0.080, 0.379], p = 0.003).
Fig. 3 Funnel Plot of the Effect Sizes of STEM Integration on Students’ Academic Achievement
Research in Science Education
Table 3 The Effect of STEM Integration on Student STEM Achievement: Univariate Analysis of Variance for Moderator Variables
Moderator Qbetween kSE Weighted mean g95% CI Qwithin I2τ2
LL UL
Degree of Integration 7.227*
Content integration 62 0.064 0.654*0.528 0.780 2794.903*97.817 0.219
Context integration 10 0.201 1.063*0.668 1.457 55.459*83.772 0.335
Tool integration 2 0.120 0.441*0.207 0.676 0.388 0.000 0.000
Learning Activities 21.347*
Problem-based learning 4 0.194 0.941*0.561 1.321 75.576*96.030 0.126
Project-based learning 23 0.074 0.401*0.255 0.547 352.342*93.756 0.098
Inquiry-based learning 15 0.152 1.051*0.754 1.349 752.728*98.140 0.291
Design-based learning 28 0.075 0.442*0.294 0.589 571.077*95.272 0.130
Involving engineering design skills or not 0.018
Not involving 27 0.100 0.650*0.454 0.847 510.743*94.909 0.221
Involving 52 0.072 0.667*0.526 0.808 2443.262*97.913 0.231
Cultural values 11.366*
Collectivism 26 0.126 1.007*0.760 1.253 197.479 87.340 0.325
Individualism 53 0.067 0.526*0.394 0.657 2692.186 98.068 0.215
Academic subject 99.575*
Engineering 7 0.041 0.203*0.122 0.285 23.170*74.104 0.008
Math 19 0.076 0.240*0.091 0.389 181.750*90.096 0.079
Science 46 0.084 0.843*0.679 1.008 1974.328*97.721 0.274
Science & Math 1 0.067 0.802*0.671 0.933 0.000 0.000 0.000
STEM 6 0.236 1.107 *0.645 1.570 78.053*93.594 0.300
Achievement type 28.395*
Standardized 16 0.076 0.229*0.080 0.379 299.745*** 94.996 0.073
Non-standardized 63 0.071 0.784*0.645 0.923 2319.761*** 97.327 0.271
Research in Science Education
Table 3 (continued)
Moderator Qbetween kSE Weighted mean g95% CI Qwithin I2τ2
LL UL
Study Design 22.755*
Pre-post experimental and control 49 0.073 0.795* 0.651 0.938 670.865 92.845 0.210
Control and experimental group’s posts 13 0.105 0.185* −0.021 0.391 233.120 94.852 0.119
Experimental group’s pre and post 17 0.122 0.648* 0.409 0.887 1703.600 99.061 0.234
Intervention duration 14.224*
Less than 4weeks 6 0.173 1.047*0.708 1.387 44.672*93.284 0.295
4–8weeks 19 0.190 1.029*0.656 1.402 173.251*89.610 0.594
Above 8weeks 33 0.084 0.461*0.296 0.626 2341.134*98.634 0.221
Grade level 2.990
Elementary 29 0.118 0.817*0.586 1.048 821.832 96.593 0.354
Middle school 23 0.091 0.600*0.421 0.779 767.405 97.133 0.160
High school 27 0.561 0.561*0.351 0.770 1321130 98.032 0.267
Note. *p < .05
Research in Science Education
Study Design The study design moderated the STEM integration-achievement link
(Qbetween = 22.755, df = 2, p < 0.001, see Table3). Specifically, the effect sizes were largest
for controlled experiments with both pre- and post-tests (g = 0.795, 95%CI[0.651, 0.938],
p < 0.001), smaller for a single group with pre- and post-tests (g = 0.648, 95% CI [0.409,
0.887], p < 0.001) and smallest for a controlled experiment with only post-tests (g = 0.185,
95% CI [−0.021, 0.391], p > 0.05).
Intervention Duration Intervention duration moderated the link between STEM integra-
tion and achievement (Qbetween = 14.224, df = 2, p < 0.001, see Table3). Specifically, inter-
ventions of less than four weeks (g = 1.047, 95% CI [0.708, 1.387], p < 0.001) or 4–8weeks
(g = 1.029, 95% CI [0.656, 1.402], p < 0.001) showed larger effect sizes than those exceed-
ing 8weeks (g = 0.461, 95% CI [0.296, 0.626], p < 0.001).
Grade Level Grade level did not affect the STEM integration-achievement link
(Qbetween = 2.990, df = 2, p = 0.224).
Discussion
This meta-analysis of 79 effect sizes in 40 studies showed a positive link between STEM
integration and academic achievement. This STEM integration-achievement link differed
across STEM integration type, learning activities, cultural values, academic subjects,
standardized versus non-standardized tests, intervention duration, and study design.
STEM Integration andStudent Achievement
The positive effect size of STEM integration on student achievement (g = 0.661) supports
the view that STEM integrated activities motivate students by addressing real-world prob-
lems, making interdisciplinary connections, and aiding their application of multidisci-
plinary knowledge—thereby helping them learn more than otherwise (Margot & Kettler,
2019). This result aligns with those of past meta-analyses (e.g., Guzey etal., 2017; Robin-
son etal., 2014). Hence, educators should advocate training teachers (e.g., via professional
development) to teach students via STEM integration to help them learn more.
Moderators
Degree ofSTEM Integration
Among the three degrees of STEM integration, context integration had the largest effect
on student achievement. These results align with the following two views. First, a famil-
iar, meaningful, authentic problem situation can enhance students’ motivation (Nadelson
& Seifert,2017) and connect familiar, concrete experiences via more neural links in their
brains (Li & Wang, 2021) to help them learn STEM concepts (Bryan etal., 2016; Moore
& Smith, 2014). Second, applying them to different contexts helps students understand
when and how to apply them to unfamiliar problem contexts (transfer, Perkins & Salomon,
2012). Both are consistent with context integration showing the largest STEM integration
effect on student achievement.
Research in Science Education
The weaker, positive results of content integration cohere with teachers’ limited content
knowledge (Brophy et al.,2008) and self-efficacy (Lewis et al., 2021), curriculum con-
straints (Helle etal., 2006), and time constraints, which hinder effective STEM content
integration and hence, student learning (Roehrig etal., 2012). For instance, many science
teachers who use integrated STEM content activities in their classes lack sufficient knowl-
edge and skills about the novel content (e.g., engineering) to effectively help their students
learn it (Hamad etal., 2022).
These results suggest future research testing whether pre-service training or professional
development can enhance teachers’ STEM knowledge and improve the effectiveness of
their STEM content integration for their students’ learning. Likewise, future studies can
also test whether aligning STEM content integration lessons with the curriculum or giv-
ing teachers more time can improve their students’ learning outcomes. Among degree of
STEM integration, tool integration had the smallest, positive effect on student achieve-
ment. This result is consistent with both (a) the fewer links across STEM concepts via tool
integration compared to context or content integration, and (b) teachers’ unfamiliarity with
new technological tools that can hinder their teaching and their students’ learning. Still, as
only one original study examined tool integration (two effect sizes), the tool integration
results require cautious interpretation.
Learning Activities
Among STEM integration learning activities, inquiry-based learning had the largest posi-
tive effect on student achievement, followed closely by problem-based learning, with much
smaller positive effects for design-based learning and for project-based learning. These
results align with the following views. First, teachers can readily manage the simpler prob-
lem-based and inquiry-based learning activities, use them more often (Albion, 2015), and
help their student learn from them (Panasan & Nuangchaler, 2010). By contrast, teachers
often lack STEM knowledge (Dare etal., 2018), experience with project-based or design-
based curricula (Frank etal., 2003; Nadelson etal., 2016), time, or curriculum flexibility
(Helle etal., 2006). So, they struggle with projects, design activities, and how to help their
students learn from them (Lewis etal., 2021). The largest positive effect of inquiry-based
learning also aligns with the common training of science teachers to teach via inquiry-
based learning (NRC, 1996) and their greater experience with it.
Engineering Design Skills
Whether STEM integration includes engineering design skills or not does not moderate its
effect on achievement. This result does not support some researchers’ claims that students’
STEM learning benefits from engineering design skills (e.g., Li etal., 2019), despite the
NGSS emphasis on integrating engineering design processes with the scientific method
(Dare, etal., 2018; NRC, 2013). As these teachers often have limited engineering design
expertise they might not implement STEM integration lessons with engineering design
skills effectively (Nadelson etal., 2016).
Cultural Values
Cultural values moderated the STEM integration-achievement link. Effect sizes in collec-
tivist cultures nearly doubled those in individualist cultures (1.007 > 0.526). This result
Research in Science Education
aligns with the view that students pay more attention to their classmates and support group
goals more in collectivist cultures than in individualist cultures (Chiu & Chow, 2015).
Such attention and support enhances learning in integrated STEM activities, which are
often done in groups. As students in collectivist cultures especially benefit from integrated
STEM group activities, teachers in such cultures should especially consider using them.
In individualistic cultures, studies can test whether interventions that foster such student
processes (attend to classmates, support group goals) can make STEM integration more
effective for student learning.
Other meta-analyses also show that culture affects learning interventions (e.g., Lei
etal.,2022a, b; Dong etal., 2024). For example, mobile learning had a bigger impact on
students’ science achievement in collectivist countries than in individualistic ones (e.g.,Lu
etal., 2023). Our results match these findings. They are consistent with the view that stu-
dents in collectivistic cultures value group goals more than individual goals, pay closer
attention to one another, recognize one another’s problems more readily, and help one
another learn more.
Achievement Measure: Academic Subject andStandardized Tests
The positive link between STEM integration and achievement was strongest for STEM,
a bit less for science, and much less for math or engineering design. These results align
with the views that (a) STEM tests are better suited for measuring STEM learning, and (b)
students engage with, like, and learn more from concrete activities than abstract ones (e.g.,
physics, Dong etal., 2024). The stronger concreteness results also align with those of a
meta-analysis showing that mobile learning worked best for biology, and progressively less
for earth and space sciences, chemistry, and physics (Dong etal., 2024). As only two stud-
ies (with six independent samples) assessed engineering design skills, so those results need
cautious interpretation.
The positive STEM integration-achievement link was larger in studies using non-stand-
ardized tests than those using standardized tests. This result coheres with the view that
researcher-designed non-standardized tests fit the content of the STEM activities better but
have worse psychometric properties (Li & Ma, 2010), yielding larger effect sizes compared
to uniform, standardized tests.
Study Design
The STEM integration-achievement link was largest in controlled experiments with both
pre- and post-tests, smaller for a single group’s pre and post-test scores, and far smaller for
a controlled experiment with only post-tests. These results suggest that biases in the latter
two types of studies affected the effect size, consistent with past studies (e.g., Kirk, 2009;
Torday & Baluška, 2019). Hence, future studies should use controlled experiments with
both pre- and post-tests.
Intervention Duration
Compared to longer interventions, shorter ones lasting eight weeks or less had much
larger effect sizes. This result fits the claim that more external events during longer
interventions interfere with them and dilute their impact (e.g., Nahmias etal., 2019).
As shorter interventions are typically more cost-effective and easier to implement,
Research in Science Education
educators should begin with shorter STEM integration interventions. Conversely, edu-
cators can consider about how to increase the effectiveness of longer STEM integra-
tion interventions. As only five studies had interventions of less than four weeks in our
analysis, those results require cautious interpretation.
Implications
These results suggest three implications. First, students learn more from STEM integra-
tion than traditional teaching methods. Hence, any comprehensive theory of learning
must include STEM integration. Also, this result suggests that educators help teachers
teach via STEM integration to help their students learn more. Given the larger effect
sizes of shorter interventions (eight weeks or less), educators can help teachers start
with these, as they are likely more cost-effective and easier to implement than longer
ones.
Second, students learn more via context integration than other types of integration.
Thus, educators should use diverse STEM contexts to motivate their students to learn
more. As many teachers have limited content knowledge across disciplines, integrating
a new context from another STEM discipline is likely easier for them than integrating
content from another STEM discipline.
Third, inquiry-based learning and problem-based learning have larger effects than
project- or design-based learning. As the latter two are newer, teachers might under-
stand them less. Hence, educators seeking to use project- or design-based learning
must understand and tackle this issue before implementing these activities. Other-
wise, inquiry- or problem-based learning activities might yield better student learning
outcomes.
Limitations andFuture Research
This study has five major limitations: conceptual taxonomy, limited reporting of key vari-
ables in original studies, single methodology, language, and few original studies. First, we
started classifying STEM integration interventions by types of integration and learning
activities, so future scholars can build and test more detailed taxonomies for them. Second,
many studies did not report attributes of STEM integration, teachers (e.g., teachers’ pro-
fessional development for STEM), students (e.g., gender, socio-economic status [e.g., Li
etal., 2018; Robinson, 2016]), or intervention procedures (e.g., teacher interactions with
students) so we could not test for their moderation effects. Notably, teachers with better
training in STEM subjects might help their students learn more. Future studies can include
such information, especially for participants from diverse backgrounds or cultures, to
inform meta-analyses examining such differences. Third, as meta-analyses can only show
coarse patterns, future studies can also use other methods (e.g., ethnographies, micro-
genetic analyses of videotapes of students’ problem-solving) to detail intricate mechanisms
and contextual differences (e.g., within and across cultures). Fourth, we only examined
studies published in English. Future meta-analyses with authors fluent in more languages
can include studies published in other languages. Fifth, we only have two studies about tool
integration, two studies with less than four weeks’ intervention, and two studies with engi-
neering achievement. Hence, we should interpret these results cautiously.
Research in Science Education
Conclusion
This study meta-analyzed STEM integration’s effect on student achievement using
79 effect sizes from 40 studies. The results showed a positive overall effect of STEM
integration on student STEM achievement, indicating its value for improving student
learning outcomes. Effect sizes varied across degrees of STEM integration, learning
activities, academic subjects, standardization of tests, cultural values, study design, and
intervention duration. Whether the integration involved engineering design skills or not
did not moderate the STEM integration’s effect on student achievement. These results
help teacher educators, professional development programs, and teachers design better
STEM instruction to improve student learning outcomes.
Funding This research was supported by the 2023 Humanities and Social Science Program sponsored by
the Ministry of Education of the People’s Republic of China (23YJC880019).
Data Availability The data used in this research is presented in Table1 in the Table file. Inquiries about the
specific data used in this study can be directed to the corresponding author.
Declarations
Ethical Approval It is not applicable to this study.
Consent and Participate It is not applicable to this study.
Conflicts of Interest The authors declare that we have no conflict of interest.
References
*Primary studies included in the meta‑analysis.
*Acar, D., Tertemiz, N., & Taşdemir, A. (2018). The effects of STEM training on the academic achieve-
ment of 4th graders in science and mathematics and their views on STEM training.International
Electronic Journal of Elementary Education, 10(4), 505–513. https:// doi. org/ 10. 26822/ iejee.
20184 38141
*Akkaya, M. M., & Benzer, S. (2020). The effect of STEM practices on academic achievement and atti-
tudes of sixth grade students.MOJES: Malaysian Online Journal of Educational Sciences, 8(2),
36–47.
Albion, P. (2015). Project-, problem-, and inquiry-based learning. In M. Henderson, M. J. Henderson, &
G. Romeo (Eds.), Teaching and digital technologies: Big issues and critical questions (pp. 240–
252). Cambridge University Press.
*Alameh, S. H. (2018). Effect of science and engineering practices in biology on students attitudes,
achievement and engineering design skills [Doctoral dissertation, American University of Beirut].
ProQuest Dissertations Publishing.
Ali, M., & Tse, W. C. (2023). Research trends and issues of engineering design process for STEM edu-
cation in K-12: A bibliometric analysis. International Journal of Education in Mathematics, Sci-
ence, and Technology (IJEMST), 11(3), 695–727. https:// doi. org/ 10. 46328/ ijemst. 2794
Aminger, W., Hough, S., Roberts, S. A., Meier, V., Spina, A. D., Pajela, H.,…, & Bianchini, J. A.
(2021). Preservice secondary science teachers’ implementation of an NGSS practice: Using math-
ematics and computational thinking. Journal of Science Teacher Education, 32(2), 188–209.
https:// doi. org/ 10. 1080/ 10465 60X. 2020. 18052 00
Research in Science Education
*Anwar, S., Menekse, M., Guzey, S., & Bryan, L. A. (2022). The effectiveness of an integrated STEM
curriculum unit on middle school students’ life science learning.Journal of Research in Science
Teaching,59(7), 1204-1234. https:// doi. org/ 10. 1002/ tea. 21756
*Angwal, Y. A., Saat, R. M., & Sathasivam, R. V. (2019). Preparation and validation of an integrated
STEM instructional material for genetic instruction among year 11 science students. Malaysian
Online Journal of Educational Sciences, 7(2), 41–56.
Bajzek, T. J. (2005). Thermocouples: A sensor for measuring temperature. IEEE Instrumentation &
Measurement Magazine, 8(1), 35–40.
Barrett, B. S., Moran, A. L., & Woods, J. E. (2014). Meteorology meets engineering: An interdisci-
plinary STEM module for middle and early secondary school students. International Journal of
STEM Education, 1(1), 1–6. https:// doi. org/ 10. 1186/ 2196- 7822-1-6
Becker, K. H., & Park, K. (2011). Integrative approaches among science, technology, engineering, and
mathematics (STEM) subjects on students’ learning: A Meta-Analysis. Journal of STEM Educa-
tion: Innovations and Research, 12(5–6), 23–37.
Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2010). A basic introduction to fixed-
effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2), 97–111.
https:// doi. org/ 10. 1002/ jrsm. 12
Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2021). Introduction to meta-analysis.
John Wiley & Sons.
Brophy, S., Klein, S., Portsmore, M., & Rogers, C. (2008). Advancing engineering education in P-12
classroom. Journal of Engineering Education, 97(3), 369–387. https:// doi. org/ 10. 1002/j. 2168-
9830. 2008. tb009 85.x
Bryan, L. A., Moore, T. J., Johnson, C. C., & Roehrig, G. H. (2016). Integrated STEM education. In C.
C. Johnson, E. E. Peters-Burton, & T. J. Moore (Eds.), STEM road map: A framework for inte-
grated STEM education(pp. 23–37). Routledge.
Bybee, R. W. (2010). Advancing STEM education: A 2020 vision. Technology and Engineering Teacher,
70(1), 30–35.
*Capobianco, B. M., Radloff, J., & Lehman, J. D. (2021). Elementary science teachers’ sense-making
with learning to implement engineering design and its impact on students’ science achievement.
Journal of Science Teacher Education, 32(1), 39–61. https:// doi. org/ 10. 1080/ 10465 60X. 2020.
17892 67
Chiu, M. M., & Chow, B. W. Y. (2015). International comparisons of student achievement. In Nata, R.
V. (Eds.), Progress in Education(pp. 93–108). Nova Science Publishers, Inc.
Cobern, W. W. (1993). Constructivism. Journal of Educational and Psychological Consultation, 4(1),
105–112. https:// doi. org/ 10. 1207/ s1532 768xj epc04 01_8
*Cotabish, A., Dailey, D., Robinson, A., & Hughes, G. (2013). The effects of a STEM intervention on
elementary students’ science knowledge and skills. School Science and Mathematics, 113(5), 215–
226. https:// doi. org/ 10. 1111/ ssm. 12023
Dare, E. A., Ellis, J. A., & Roehrig, G. H. (2018). Understanding science teachers’ implementations of
integrated STEM curricular units through a phenomenological multiple case study.International
Journal of STEM Education, 5(4).https:// doi. org/ 10. 1186/ s40594- 018- 0101-z.
Dong, Z., Chiu, M. M., Zhou, S., & Zhang, Z. (2024). The effect of mobile learning on school-aged
students’ science achievement: A meta-analysis. Education and Information Technologies, 29(1),
517–544. https:// doi. org/ 10. 1007/ s10639- 023- 12240-3
English, L. D., King, D., & Smeed, J. (2017). Advancing integrated STEM learning through engineering
design. The Journal of Educational Research, 110(3), 255–271. https:// doi. org/ 10. 1080/ 00220 671.
2016. 12640 53
*Fan, S.-C., & Yu, K.-C. (2017). How an integrative STEM curriculum can benefit students in engineer-
ing design practices. International Journal of Technology and Design Education, 27(1), 107–129.
https:// doi. org/ 10. 1007/ s10798- 015- 9328-x
Frank, M., Lavy, I., & Elata, D. (2003). Implementing the project-based learning approach in an aca-
demic engineering course. International Journal of Technology and Design Education, 13(3),
273–288. https:// doi. org/ 10. 1023/A: 10261 92113 732
*Gazibeyoglu, T., & Aydin, A. (2019). The effect of STEM-based activities on 7th grade students’ aca-
demic achievement in force and energy unit and students’ opinions about these activities. Univer-
sal Journal of Educational Research, 7(5), 1275–1285. https:// doi. org/ 10. 13189/ ujer. 2019. 070513
Gentile, L., Caudill, L., Fetea, M., Hill, A., Hoke, K., Lawson, B., & Szajda, D. (2012). Challenging
disciplinary boundaries in the first year: A new introductory integrated science course for STEM
majors. Journal of College Science Teaching, 41(5), 44–50.
Research in Science Education
Guzey, S. S., Moore, T. J., Harwell, M., & Moreno, M. (2016). STEM integration in middle school life
science. Journal of Science Education and Technology, 25(4), 550–560. https:// doi. org/ 10. 1007/
s10956- 016- 9612-x
*Guzey, S. S., Harwell, M., Moreno, M., Peralta, Y., & Moore, T. J. (2017). The impact of design-based
STEM integration curricula on student achievement in engineering, science, and mathematics.Jour-
nal of Science Education and Technology,26(2), 207-222. https:// doi. org/ 10. 1007/ s10956- 016- 9673-x
Guzey, S. S., Caskurlu, S., & Kozan, K. (2020). Integrated STEM pedagogies and student learning. In C.
Johnson, M. Mohr-Schroeder, T. Moore, & L. English (Eds.), Handbook of research on STEM educa-
tion (pp. 65–75). Routledge.
*Hasançebi, F., Güner, Ö., Kutru, C., & Hasancebi, M. (2021). Impact of Stem integrated argumentation-
based inquiry applications on students’ academic success, reflective thinking and creative thinking
skills. Participatory Educational Research, 8(4), 274–296. https:// doi. org/ 10. 17275/ per. 21. 90.8.4
Hamad, S., Tairab, H., Wardat, Y., Lutfieh, R., AlArabi, K., Yousif, M., Abu-Al-Aish, A., & Stoica, G.
(2022). Understanding science teachers’ implementations of integrated STEM. Sustainability, 14(6),
3594. https:// doi. org/ 10. 3390/ su140 63594
Helle, L., Tynjälä, P., & Olkinuora, E. (2006). Project-based learning in post-secondary education–the-
ory, practice and rubber sling shots. Higher Education, 51(2), 287–314. https:// doi. org/ 10. 1007/
s10734- 004- 6386-5
Hess, J. L., & Sorge, B., & Feldhaus, C. (2016). The efficacy of Project Lead the Way. Paper presented at
2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. https:// doi. org/ 10. 18260/p.
26151
Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring Inconsistency in Meta-
Analyses. Bmj, 327(7414), 557–560. https:// doi. org/ 10. 1136/ bmj. 327. 7414. 557
Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based
and inquiry learning. Educational Psychologist, 42(2), 99–107. https:// doi. org/ 10. 1080/ 00461 52070
126336
*Hsiao, H.-S., Yu, K.-C., Chang, Y.-S., Chien, Y.-H., Lin, K.-Y., Lin, C.-Y., Chen, J.-C., Chen, J.-H., &
Lin, Y.-W. (2017). The study on integrating the design thinking model and STEM activity unit for
senior high school living technology course. 2017 7th World Engineering Education Forum (WEEF),
383–390. https:// doi. org/ 10. 1109/ WEEF. 2017. 84671 11
*Izgi, S. & Kalayci, S. (2020). The effect of the STEM approach based on the 5E model on academic
achievement and scientific process skills: the transformation of electrical energy. International Jour-
nal of Education Technology and Scientific Researches. 5(13), 1578–1629. Retrieved Nov 18, 2024,
from https:// www. ijets ar. com/ Makal eler/ 78038 4334_ 11.% 201578- 1629% 20ser pil% 20kal ayc% c4% b1.
pdf
*Jahan, A. (2018).Impact of project Lead the Way™ engineering program on student achievement (Publi-
cation No. 10931041) [Doctoral dissertation, Lamar University-Beaumont]. ProQuest Dissertations
Publishing.
*James, J. S. (2014).Science, technology, engineering, and mathematics (STEM) curriculum and seventh
grade mathematics and science achievement(Publication No. 3614935) [Doctoral dissertation, Grand
Canyon University]. ProQuest Dissertations Publishing.
*Kağnıcı, A., & Sadi, Ö. (2021). Students’ conceptions of learning biology and achievement after STEM
activity–enriched instruction. IE: Inquiry in Education,13(1), Article 7 . https:// digit alcom mons. nl.
edu/ ie/ vol13/ iss1/7
*Kelley, T. R., Sung, E., Han, J., & Knowles, J. G. (2023). Impacting secondary students’ STEM knowledge
through collaborative STEM teacher partnerships.International Journal of Technology and Design
Education,33(4), 1563-1584. https:// doi. org/ 10. 1007/ s10798- 022- 09783-w
Kirk, R. E. (2009). Experimental design. In R. E. Millsap & A. Maydeu-Olivares (Eds.), The sage handbook
of quantitative methods in psychology (pp. 23–45). Sage Publications.
Kokotsaki, D., Menzies, V., & Wiggins, A. (2016). Project-based learning. Improving Schools, 19(3), 267–
277. https:// doi. org/ 10. 1177/ 13654 80216 659733
*Kurt, M., & Benzer, S. (2020). An Investigation on the effect of STEM practices on sixth grade students’
academic achievement, problem solving skills, and attitudes towards STEM. Journal of Science
Learning, 3(2), 79–88. https:// doi. org/ 10. 17509/ jsl. v3i2. 21419
Lei, H., Wang, C., Chiu, M. M., & Chen, S. (2022a). Do educational games affect students’ achievement
emotions? Evidence from a meta-analysis. Journal of Computer Assisted Learning, 38(4), 946–959.
https:// doi. org/ 10. 1111/ jcal. 12664
Lei, H., Chiu, M. M., Wang, D., Wang, C., & Xie, T. (2022b). Effects of Game-Based Learning on Students’
Achievement in Science: A Meta-Analysis. Journal of Educational Computing Research, 60(6),
1373–1398. https:// doi. org/ 10. 1177/ 07356 33121 10645 43
Research in Science Education
LeTendre, G. K., Hofer, B. K., & Shimizu, H. (2003). What is tracking? American Educational Research
Journal, 40, 43–89. https:// doi. org/ 10. 3102/ 00028 31204 00010 43
Lewis, F., Edmonds, J., & Fogg-Rogers, L. (2021). Engineering science education. International Journal of
Science Education, 43(5), 793–822. https:// doi. org/ 10. 1080/ 09500 693. 2021. 18875 44
*Li, L., Tripathy, R., Salguero, K., & McCarthy, B. (2018). Evaluation of learning by making i3 project:
STEM success for rural schools. WestEd. Retrieved Nov 18, 2024, from https:// files. eric. ed. gov/ fullt
ext/ ED594 016. pdf
Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’
mathematics learning. Educational Psychology Review, 22, 215–243. https:// doi. org/ 10. 1007/
s10648- 010- 9125-8
*Li, Y., Huang, Z., Jiang, M., & Chang, T. W. (2016). The effect on pupils’ science performance and prob-
lem-solving ability through Lego. Journal of Educational Technology & Society, 19(3), 143–156.
https:// doi. org/ 10. 2307/ jeduc techs oci. 19.3. 143
Li, Y., Schoenfeld, A. H., diSessa, A. A., Grasser, A. C., Benson, L. C., English, L. D., & Duschl, R. A.
(2019). Design and design thinking in STEM education. Journal for STEM Education Research, 2(2),
93–104. https:// doi. org/ 10. 1007/ s41979- 019- 00020-z
Li, X., & Wang, W. (2021). Exploring spatial cognitive process among STEM students and its role in STEM
education: A cognitive neuroscience perspective. Science & Education, 30(1), 121–145. https:// doi.
org/ 10. 1007/ s11191- 020- 00167-x
*Lie, R., Selcen Guzey, S., & Moore, T. J. (2019). Implementing engineering in diverse upper elementary
and middle school science classrooms. Journal of Science Education and Technology, 28(2), 104–
117. https:// doi. org/ 10. 1007/ s10956- 018- 9751-3
*Lippert, R. J., & Seals, T. R. (2023). Impacting African American Student Achievement in the Middle
School STEM Classroom by Teaching Mathematics Through Arts Integration and Design Thinking
(Doctoral dissertation, University of Missouri-Saint Louis). Retrieved Nov 17, 2024, from https:// irl.
umsl. edu/ disse rtati on/ 1301
Lu, Z., Chiu, M. M., Cui, Y., Mao, W., & Lei, H. (2023). Effects of game-based learning on students’ com-
putational thinking: A meta-analysis. Journal of Educational Computing Research, 61(1), 235–256.
https:// doi. org/ 10. 1177/ 07356 33122 11007
Margot, K. C., & Kettler, T. (2019). Teachers’ perception of STEM integration and education. International
Journal of STEM Education, 6(1), 1–16. https:// doi. org/ 10. 1186/ s40594- 018- 0151-2
Massachusetts Department of Education. (2006). Massachusetts science and technology/engineering cur-
riculum framework. Massachusetts Department of Education. Retrieved Nov 18, 2024, from https://
files. eric. ed. gov/ fullt ext/ ED508 413. pdf
McAtavey, J., & Nikolovska, I. (2010). Team collectivist culture: A remedy for creating team effectiveness.
Human Resource Development Quarterly, 21(3), 307–316. https:// doi. org/ 10. 1002/ hrdq. 20039
*McClain, M. L. (2015).The effect of STEM education on mathematics achievement of fourth-grade under-
represented minority students (Publication No. 3723284) [Doctoral dissertation, Capella University].
ProQuest Dissertations Publishing.
*McGehee, N. J. L. (2015). Project-based learning in STEM and act achievement [Doctoral dissertation,
Tennessee Technological University]. Retrieved Nov 17, 2024, from https:// www. proqu est. com/ docvi
ew/ 16896 59663? pq- origs ite= gscho lar& fromo penvi ew= true
*McHugh, L. (2016). The integration of mathematics in middle school science (Publication
No.10140739)[Doctoral dissertation, State University of New York at Stony Brook]. ProQuest Dis-
sertations Publishing.
Merrill, C. (2001). Integrated technology, mathematics, and science education. Journal of Industrial
Teacher Education, 38, 45–61. Retrieved Nov 17, 2024, from https:// schol ar. lib. vt. edu/ ejour nals/
JITE/ v38n3/ merri ll. html
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group*. (2009).Preferred Reporting Items
for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Annals of Internal Medi-
cine,151(4), 264–269. https:// doi. org/ 10. 7326/ 0003- 4819- 151-4- 20090 8180- 00135
Moore, T. J., Johnston, A. C., & Glancy, A. W. (2020). STEM integration: A synthesis of conceptual frame-
works and definitions. In C. C. Johnson, M. J. Mohr-Schroeder, T. J. Moore, & L. D. English (Eds.),
Handbook of research on STEM education (pp. 3–16). Routledge.
Moore, T. J., & Smith, K. A. (2014). Advancing the State of the Art of STEM Integration. Journal of STEM
Education, 15(1), 5. Retrieved Nov 17, 2024, from https:// karls mithmn. org/ wp- conte nt/ uploa ds/ 2017/
08/ Moore- Smith- JSTEM Ed- Guest Edito rialF. pdf
Nadelson, L. S., & Seifert, A. L. (2017). Integrated STEM defined: Contexts, challenges, and the future. The
Journal of Educational Research, 110(3), 221–223. https:// doi. org/ 10. 1080/ 00220 671. 2017. 12897 75
Research in Science Education
Nadelson, L.,Sias, C. M., & Seifert, A.(2016).Challenges for integrating engineering into K-12 cur-
riculum: Indicators of K-12 teachers’ propensity to adopt innovation. Paper presented at the ASEE
Annual Conference & Exposition, New Orleans, LA. Retrieved Sept 14, 2023, fromhttps:// peer.
asee. org/ chall enges- for- integ rating- engin eering- into- the-k- 12- curri culum- indic ators- of-k- 12- teach
ers- prope nsity- to- adopt- innov ation
Nahmias, A. S., Pellecchia, M., Stahmer, A. C., & Mandell, D. S. (2019). Effectiveness of community-
based early intervention for children with autism spectrum disorder: A meta-analysis. Journal of
Child Psychology and Psychiatry, 60(11), 1200–1209. https:// doi. org/ 10. 1111/ jcpp. 13073
National Research Council (NRC). (1996). National science education standards. National Academies
Press.
National Research Council [NRC]. (2013). Next generation science standards: For states, by states.
National Academies Press.
National Center for Education Statistics (NCES). (2024). What is NAEP? Retrieved Aug 9, 2024, from
https:// nces. ed. gov/ natio nsrep ortca rd/
Organization for Economic Cooperation and Development (OECD). (2024).About PISA. Retrieved from
Aug 9, 2024, https:// www. oecd. org/ en/ about/ progr ammes/ pisa. html
Olivarez, N. (2012).The Impact of a STEM program on academic achievement of eighth grade students
in a south Texas middle school (Publication No. 3549798) [Doctoral dissertation, Texas A&M
University-Corpus Christi]. ProQuest Dissertations Publishing.
Panasan, M., & Nuangchalerm, P. (2010). Learning outcomes of project-based and inquiry-based learn-
ing activities. Online Submission, 6(2), 252–255.
*Parlakay, E. S., & Koç, Y. (2020). An investigation of the effectiveness of STEM practices on fifth
grade students’ academic achievement and motivations at the unit "Exploring and Knowing The
World of Living Creatures". International Journal of Progressive Education, 16(1), 125–137.
https:// doi. org/ 10. 29329/ ijpe. 2020. 228.1
Pedaste, M., Mäeots, M., Leijen, Ä., & Sarapuu, S. (2012). Improving students’ inquiry skills through
reflection and self-regulation scaffolds. Technology, Instruction, Cognition and Learning, 9(1–2),
81–95.
Perkins, D. N., & Salomon, G. (2012). Knowledge to go: A Motivational and Dispositional View of Trans-
fer. Educational Psychologist, 47(3), 248–258. https:// doi. org/ 10. 1080/ 00461 520. 2012. 693354
*Pozarski Connolly, C. J. (2017). Evaluating the effectiveness of project recharge [Doctoral dissertation,
University of Nevada, Reno]. ProQuest Dissertations Publishing.
Puente, S. G., Van Eijck, M., & Jochems, W. (2013). Empirical validation of characteristics of design-based
learning in higher education. International Journal of Engineering Education, 29(2), 491–503.
*Rehmat, A. P., & Hartley, K. (2020). Building engineering awareness. Interdisciplinary Journal of
Problem-Based Learning, 14(1). https:// doi. org/ 10. 14434/ ijpbl. v14i1. 28636
*Robinson, N. (2016). A case study exploring the effects of using an integrative STEM curriculum on
eighth grade students? Performance and engagement in the mathematics classroom [Doctoral dis-
sertation, Georgia State University]. ProQuest Dissertations Publishing.
*Robinson, A., Dailey, D., Hughes, G., & Cotabish, A. (2014). The effects of a science-focused STEM
intervention on gifted elementary students’ science knowledge and skills. Journal of Advanced
Academics, 25(3), 189–213. https:// doi. org/ 10. 1177/ 19322 02X14 533799
Roehrig, G. H., Dare, E. A., Ring-Whalen, E., & Wieselmann, J. R. (2021). Understanding coherence
and integration in integrated STEM curriculum. International Journal of STEM Education, 8,
1–21. https:// doi. org/ 10. 1186/ s40594- 020- 00259-8
Roehrig, G. H., Moore, T. J., Wang, H. H., & Park, M. S. (2012). Is adding the E enough? Investigating
the impact of K‐12 engineering standards on the implementation of STEM integration. School Sci-
ence and Mathematics, 112(1), 31–44. https:// doi. org/ 10. 1111/j. 1949- 8594. 2011. 00112.x
Ross, J. M., Peterman, K., Daugherty, J. L., & Custer, R. L. (2018). An engineering innovation tool: pro-
viding science educators a picture of engineering in their classroom. Journal of STEM Education:
Innovations & Research, 19(2), 13–18.
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication bias in meta-analysis: A brief over-
view. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis:
Prevention, assessment, and adjustments (pp. 1–10). Wiley.
*Sabag, N., & Trotskovsky, E. (2013). Using lab experiments in electric circuits to promote achieve-
ments in mathematics. 2013 IEEE Global Engineering Education Conference (EDUCON), 123–
129. https:// doi. org/ 10. 1109/ EduCon. 2013. 65300 96
Saraç, H. (2018). The Effect of Science, Technology, Engineering and Mathematics-STEM Educational
Practices on Students’ Learning Outcomes: A Meta-Analysis Study. Turkish Online Journal of
Educational Technology-TOJET, 17(2), 125–142.
Research in Science Education
*Sarican, G., & Akgunduz, D. (2018). The impact of integrated STEM education on academic achieve-
ment, reflective thinking skills towards problem solving and permanence in learning in science
education.Cypriot Journal of Educational Sciences,13(1), 94-113.
*Sauder, L. D. (2023). Integrated STEM Learning Activity: Effect on Student Engagement and Learning
(Publication No. 30489534) (Doctoral dissertation, University of South Dakota). ProQuest Dis-
sertations Publishing.
Servant-Miklos, V. (2020). Problem-oriented project work and problem-based learning: “Mind the
Gap!”. Interdisciplinary Journal of Problem-Based Learning, 14(1). https:// doi. org/ 10. 14434/
ijpbl. v14i1. 28596
*Shelden, T. C. (2021). Making and Math: Exploring How Integrated Stem Maker Units Impact Upper
Elementary Math Development [Doctoral dissertation, University of Massachusetts Lowell]. Pro-
Quest Dissertations Publishing.
Siverling, E. A., Suazo-Flores, E., Mathis, C. A., & Moore, T. J. (2019). Students’ use of STEM content
in design justifications during engineering design-based STEM integration. School Science and
Mathematics, 119(8), 457–474.
Siregar, N. C., Rosli, R., Maat, S. M., & Capraro, M. M. (2019). The effect of science, technology,
engineering and mathematics (STEM) program on students’ achievement in mathematics: A meta-
analysis. International Electronic Journal of Mathematics Education, 15(1), em0549.
*Srihongsa, C., Santiboon, T., & Ponkham, K. (2017). Assessing students’ critical thinking abilities and
science attitudes for enhancing their achievements through the instructional approaching manage-
ment with the STEM education instructional method of secondary students at the 10th grade level.
European Journal of Education Studies, 3(5),377-404. https:// doi. org/ 10. 5281/ ZENODO. 546597
*Stitham, R. (2018).The Effects of an Elementary STEM Intervention on Fourth-Grade Student Out-
comes in Language Arts and Math (Publication No.13419109)[Doctoral dissertation, Concordia
University (Oregon)]. ProQuest Dissertations Publishing.
Struyf, A., De Loof, H., Boeve-de Pauw, J., & Van Petegem, P. (2019). Students’ engagement in different
STEM learning environments: Integrated STEM education as promising practice? International
Journal of Science Education, 41(10), 1387–1407. https:// doi. org/ 10. 1080/ 09500 693. 2019. 16079
83
Sung, Y. T., Chang, K. E., & Liu, T. C. (2016). The effects of integrating mobile devices with teaching
and learning on students’ learning performance. Computers & Education, 94, 252–275. https://
doi. org/ 10. 1016/j. compe du. 2015. 11. 008
*Sunyoung, H. A. N., Rosli, R., Capraro, M. M., & Capraro, R. M. (2016). The effect of science, tech-
nology, engineering and mathematics (STEM) project based learning (PBL) on students’ achieve-
ment in four mathematics topics. Journal of Turkish Science Education, 13(special), 3–29.
*Tati, T., Firman, H., & Riandi, R. I. O. P. (2017). The effect of STEM learning through the project
of designing boat model toward student STEM literacy. Journal of Physics: Conference Series,
895(1), 012157. https://doi.org/10.1088/17426596/895/1/012157
Tian, L., Li, Y., Li, P. P., & Bodla, A. A. (2015). Leader–member skill distance, team cooperation, and
team performance: A cross-culture study in the context of sport teams. International Journal of
Intercultural Relations, 49, 183–197. https:// doi. org/ 10. 1016/j. ijint rel. 2015. 10. 005
Torday, J. S., & Baluška, F. (2019). Why control an experiment?: From empiricism, via consciousness,
toward Implicate Order.EMBO Reports, 20(10), e49110. https:// doi. org/ 10. 1080/ 00461 520. 2012.
693354
*Tsai, L. T., Chang, C. C., & Cheng, H. T. (2021). Effect of a STEM-oriented course on students’
marine science motivation, interest, and achievements.Journal of Baltic Science Education,20(1),
134–145. https:// doi. org/ 10. 33225/ jbse/ 21. 20. 134
*Vallera, F. L., & Bodzin, A. M. (2020). Integrating STEM with AgLIT (agricultural literacy through
innovative technology): The efficacy of a project-based curriculum for upper-primary students.
International Journal of Science and Mathematics Education, 18(3), 419-439.https:// doi. org/ 10.
1007/ s10763- 019- 09979-y
Vasquez, J. A., Cary, S., & Comer, M. (2013). STEM lesson essentials, grades 3–8: Integrating science,
technology, engineering, and mathematics. Heinemann.
Vasquez, J. A. (2015). STEM–Beyond the Acronym. Educational Leadership, 72(4), 10–15.
Wahono, B., Lin, P. L., & Chang, C. Y. (2020). Evidence of STEM enactment effectiveness in Asian
student learning outcomes. International Journal of STEM Education, 7(1), 1–18. https:// doi. org/
10. 1186/ s40594- 020- 00236-1
Wang, H. H., & Knobloch, N. A. (2018). Levels of STEM integration through agriculture, food, and
natural resources. Journal of Agricultural Education, 59(3), 258–277. https:// doi. org/ 10. 5032/ jae.
2018. 03258
Research in Science Education
Wang, H. H., & Knobloch, N. A. (2022). Preservice educators’ interpretations and pedagogical benefits of
a STEM integration through agriculture, food and natural resources rubric. Journal of Pedagogical
Research, 6(2), 4–28. https:// doi. org/ 10. 33902/ JPR. 20221 3513
Williams, E. V. (2020). Investigating the impact of the integrated STEM program on student test scores in
Jamaica[Doctoral dissertation, William Howard Taft University]. ProQuest Dissertations Publishing.
Wong, H. L. H. (2011). Critique: A communicative event in design education. Visible Language, 45(3),
221–247.
Wu, Y., Zhao, B., Wei, B., & Li, Y. (2022). Cultural or economic factors? Which matters more for collabo-
rative problem-solving skills: Evidence from 31 countries. Personality and Individual Differences,
190, 111497. https:// doi. org/ 10. 1016/j. paid. 2021. 111497
*Yaki, A. A., Saat, R. M., Sathasivam, R. V., & Zulnaidi, H. (2019). Enhancing science achievement utilis-
ing an integrated STEM approach.Malaysian Journal of Learning and Instruction,16(1), 181-205.
Yew, E. H. J., & Goh, K. (2016). Problem-based learning: An overview of its process and impact on learn-
ing. Health Professions Education, 2(2), 75–79. https:// doi. org/ 10. 1016/j. hpe. 2016. 01. 004
You, H. S. (2017). Why teach science with an interdisciplinary approach: History, trends, and conceptual
frameworks. Journal of Education and Learning, 6(4), 66. https:// doi. org/ 10. 5539/ jel. v6n4p 66
Yücelyiğit, S., & Toker, Z. (2021). A meta-analysis on STEM studies in early childhood education. Turkish
Journal of Education, 10(1), 23–36.
Zeng, S.B. & Yao, J.J. (2020). Identifying the “Best Evidence”: How to use meta-analysis to conduct a
literature review—A case of STEM education’s effect on students’ academic achievement]. Journal
of East China Normal University (Educational Sciences Edition), 38(6), 70–85. https:// doi. org/ 10.
16382/j. cnki. 1000- 5560. 2020. 06. 005
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under
a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted
manuscript version of this article is solely governed by the terms of such publishing agreement and applicable
law.
A preview of this full-text is provided by Springer Nature.
Content available from Research in Science Education
This content is subject to copyright. Terms and conditions apply.