Teacher training, coaching and school libraries in rural indigenous Guatemala: A multi-pronged approach to improving reading proficiency

Abstract

Efforts to address persistent intergenerational poverty in the Global South have focused, in part, on improving both access to and quality of schooling for all children, often including teacher training and provision of materials. Child Aid supports literacy development in hundreds of primary schools in indigenous communities in the rural highlands of Guatemala through an innovative and scalable teacher training program. The program works in over 100 schools at a time offering a four-year intervention with a three-pronged approach: workshops; professional instructional coaching; and providing thousands children's literature books to school libraries and classrooms. Child Aid's program is uniquely focused on improving not only basic reading ability, but reading comprehension skill and critical thinking among students. This study examined whether Child Aid's multifaceted but scalable intervention had a positive effect on children's reading comprehension through two large-scale quasi-experimental studies, with the first serving as the initial study and the second serving as a replication study. Hierarchical linear modeling was used to explore differences in reading comprehension gains between two large samples comparing students' reading comprehension gains in Child Aid schools with those of students in control schools. In both studies, students in Child Aid schools consistently demonstrated significantly greater gains in their reading comprehension skills than students who were not in Child Aid schools. Additionally, we learned students with weaker skills at the start of the year had the greatest gains. These findings will be presented and implications for the Child Aid program and other literacy interventions will be discussed.

Full text

Teacher training, coaching and school libraries in rural indigenous
Guatemala: A multi-pronged approach to improving reading prociency
Sarah L. Peller
a,*
, Amanda M. Marcotte
b
, Ketan
b
, Craig S. Wells
b
, Nancy Press
c
, Maciej Kos
d
a
Child Aid, Guatemala
b
College of Education, University of Massachusetts Amherst, MA, USA
c
Child Aid, Portland, OR, USA
d
College of Science, Northeastern University, Boston, MA, USA
ARTICLE INFO
Keywords:
Literacy
Coaching
Teacher training
Global south
Guatemala
Reading comprehension
Hierarchical linear modeling
Children’s literature
Critical thinking
ABSTRACT
Efforts to address persistent intergenerational poverty in the Global South have focused, in part, on improving
both access to and quality of schooling for all children, often including teacher training and provision of ma-
terials. Child Aid supports literacy development in hundreds of primary schools in indigenous communities in the
rural highlands of Guatemala through an innovative and scalable teacher training program. The program works
in over 100 schools at a time offering a four-year intervention with a three-pronged approach: workshops;
professional instructional coaching; and providing thousands children’s literature books to school libraries and
classrooms. Child Aid’s program is uniquely focused on improving not only basic reading ability, but reading
comprehension skill and critical thinking among students. This study examined whether Child Aid’s multifaceted
but scalable intervention had a positive effect on children’s reading comprehension through two large-scale
quasi-experimental studies, with the rst serving as the initial study and the second serving as a replication
study. Hierarchical linear modeling was used to explore differences in reading comprehension gains between two
large samples comparing students’ reading comprehension gains in Child Aid schools with those of students in
control schools. In both studies, students in Child Aid schools consistently demonstrated signicantly greater
gains in their reading comprehension skills than students who were not in Child Aid schools. Additionally, we
learned students with weaker skills at the start of the year had the greatest gains. These ndings will be presented
and implications for the Child Aid program and other literacy interventions will be discussed.
1. Introduction
1.1. Reading prociency
Efforts to address persistent and widespread poverty in the world, in
particular in developing countries, have focused in part on improving
access to schooling. In recent years, this focus has paid off, with
educational coverage now more expansive than ever before in educa-
tional history (Vasquez-Lopez & Huerta-Manzanilla, 2021). However, a
new problem has been revealed. Although students are in school, it is not
clear they are learning in meaningful ways (Unesco Institute for Statis-
tics, 2017). According to the World Bank, 60 % of students globally
demonstrate a lack of basic reading prociency (World Bank, 2017).
Reading prociency is important to governments and societies
because of its impact on the educational and career prospects of
individuals and on the economic wellbeing of developing countries
(Hanushek & Woessmann, 2007). Additionally, research has demon-
strated connections between higher literacy ability and greater quality
of life, knowledge about health and accessing of health care, and
reduction of behavioral health risks (Berkman et al., 2004). Procient
reading is also key to learning in all areas, from middle elementary
school on.
However, reading prociency eludes easy denition since it is a skill
that lies on an extensive developmental continuum from decoding the
written word to the more advanced ability to interpret and analyze
complex texts. UNESCO’s Programme for International Student Assess-
ment (PISA) has been used to evaluate the basic educational compe-
tencies of students in over 70 countries at the end of their formal
schooling opportunities, which for most students is at approximately age
15, and it has helped to dene various degrees of reading prociency.
* Corresponding author: Nuevo San Gaspar Casa #1, San Gaspar Vivar, La Antigua Guatemala, Guatemala, Central America.
E-mail address: sarah.peller@caguatemala.org (S.L. Peller).
Contents lists available at ScienceDirect
International Journal of Educational Research Open
journal homepage: www.elsevier.com/locate/ijedro
https://doi.org/10.1016/j.ijedro.2025.100437
Received 2 January 2024; Received in revised form 13 January 2025; Accepted 14 January 2025
International Journal of Educational Research Open 8 (2025) 100437
Available online 17 January 2025
2666-3740/© 2025 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-
nc-nd/4.0/ ).

Students obtaining level two on the test are considered to have achieved
basic competenc. Yet due to global advances in information technology
and knowledge, it is thought that a higher skill level than this basic
prociency is now necessary for modern life and employment (Elliott,
2017). According to the PISA test results, advanced reading prociency
is associated with certain metacognitive abilities: understanding and
remembering; generating abstracts; and evaluating the quality and
credibility of information in a text (Vasquez-Lopez &
Huerta-Manzanilla, 2021).
Procient reading develops throughout the lifetime when a child
grows up in a language-rich environment with access to good early
reading instruction and to texts that are graded in complexity over time.
Both early verbal abilities and socioeconomic status are highly predic-
tive of later achievement in reading comprehension (Morrow et al.,
2023a; Blanchard, 2023; Hoover & Gough, 1990). Adaptation to in-
struction and reading enjoyment are also signicant characteristics of
students who demonstrate advanced reading prociency
(Vasquez-Lopez & Huerta-Manzanilla, 2021).
The socioeconomic status of the community inuences the quality of
education provided at its schools. Community-level socioeconomic fac-
tors include governmental investment in education, teacher salaries,
average educational attainment and training in the community and
among teachers and parents, prioritization of education and literacy in
the culture, among others. The quality of education provided at school
has been shown to impact the reading prociency of its students.
Research has consistently also shown that the quality of instruction
provided by individual teachers, or teacher quality, is one of the most
important variables affecting student achievement (Cornett & Knight,
2009; Goe, 2007; Wenglinsky, 2000). Reading achievement specically
is also affected by teacher quality. The PISA investigation found that the
student-teacher relationship as well as overall educational quality
impact reading prociency achieved by students. The degree of support
that students perceive from their teacher was found to be an important
enabling factor, as well as the perceptions of students that their teacher
is interested in them (OECD Unesco, 2019; Vasquez-Lopez &
Huerta-Manzanilla, 2021).
1.2. Reading instruction
Decades of research have identied best practices for teaching chil-
dren to read using a range of linguistic- and literacy-based activities that
primary school teachers should implement throughout each school day
and across the primary grades. In 2000, the United States commissioned
the National Reading Panel to review the extensive empirical research
on teaching reading. The panel identied ve teaching components that
comprise comprehensive literacy instruction. They found that explicit
and systematic instruction in phonemic awareness and phonics, guided
reading with feedback to work on uency and accuracy when reading
connected text, direct instruction in academic vocabulary, and instruc-
tion of comprehension strategies, are the elements of evidence-based
reading instruction (National Reading Panel, 2000). Although there
has been some controversy regarding best practices in reading instruc-
tion, even resulting in the "reading wars" of the 1980
′
s, a systematic and
explicit approach that provides in-depth teaching to all ve components
is considered best practice and is of particular importance for children
presenting with difculties learning to read. The basic ndings of the
Panel have held up over time and continue to be considered best prac-
tices in literacy instruction (Morrow et al., 2023b).
Advanced reading comprehension or reading prociency, the ulti-
mate goal of literacy instruction, is a multidimensional skill that has
been divided theoretically into two independent strands of learning:
decoding and comprehension (Hoover & Gough, 1990). During the early
grades, students must learn the sounds the letters make and then
combine them to begin to decode words. With modeling, instruction,
focused practice, and incidental exposure to text, they then develop the
ability to decode in an increasingly automatic fashion, eventually
reading with sufcient uency (speed, accuracy and prosody) to be able
to simultaneously comprehend what they are reading (Morrow et al.,
2023b). Children vary in the speed at which they learn to read and the
amount and intensity of direct instruction they need. Many infer the
code and have a love for reading that seems inborn, quickly becoming
automatic decoders and enjoying reading on their own, while others
struggle with each phase of skill acquisition (Pfost et al., 2014).
Developmental reading theory identies the third grade as an
important transition point when children should transition from the
"learning to read" (decoding) stage to the "reading to learn" (compre-
hension) stage. Procient reading comprehension is a complex skill that
depends on the successful integration of lower-level word recognition
skills with higher-level cognitive abilities such as verbal comprehension,
linguistic ability, "critical thinking", and background knowledge. Tex-
tual characteristics such as the cultural match between the reader and
the text, text difculty, and text genre familiarity also impact students’
ability to make meaning from text. Therefore, what constitutes "reading"
in the early grades is qualitatively different from the advanced uency
and comprehension required in later grades and beyond (Morrow et al.,
2023b; Hoover & Gough, 1990).
It follows that what constitutes reading instruction changes as stu-
dents progress through primary school and through the phases of liter-
acy acquisition: from sounding out letters; to automatic and uent
recognition of words; to comprehending, summarizing and synthesizing
progressively advanced and complex texts (Morrow et al., 2023b). At an
advanced level, children must be able to uently and automatically
decode connected text on the one hand, and to have simultaneously
developed academic vocabulary, critical thinking ability, and advanced
verbal skills on the other. These two skill sets merge to inform their
reading comprehension ability and overall prociency.
1.3. Interventions to improve reading prociency in developing countries
Research on primary school interventions in the developing world
has established a strong relationship between the provision of sustained
and intensive professional development and support to teachers and
improved student learning outcomes (Krishnaratne et. al., 2013; McE-
wan, 2015; Torrente et. al., 2019; Kremer et al., 2013). However, re-
views of the international empirical literature attempting to identify the
most effective primary school interventions in developing countries
affecting student learning have arrived at divergent conclusions. In an
effort to resolve the question, the World Bank commissioned a review of
the reviews, nding the largest impact on student learning for "peda-
gogical interventions that match teaching to students’ learning" and the
second-largest impact for "individualized, repeated teacher training,
associated with a specic method or task" (Evans & Popova, 2015). In
other words, the top two types of interventions that impact student
learning both involve providing professional development to teachers,
especially when the training includes matching instruction to student
needs and is individualized, repeated, and trains teachers on the use of
specic methodologies and techniques.
Early-grade reading interventions involving teacher training in pri-
mary schools in developing countries have overall reported a positive
impact on students’ reading ability (Kim et. al., 2019; Graham & Kelly,
2013). Interventions demonstrating the highest impact are those which
include ongoing pedagogical support or instructional coaching of
teachers as opposed to stand-alone workshops. A great deal of evidence
supports the power of instructional coaching to increase teacher quality
and impact student learning in resource-rich environments (Bean, 2014;
Pepsen, 2019; Knight, 2009; Smiley et al., 2020). A small but growing
body of evidence also has demonstrated its effectiveness as an important
component of early-grade literacy interventions in developing countries
(Cornett & Knight, 2009; Evans & Popova, 2015; Pepsen, 2019).
Although the ultimate goal of all reading instruction is to eventually
achieve advanced reading comprehension, the majority of interventions
focus on emergent reading skills (decoding), or "learning to read". These
S.L. Peller et al.
International Journal of Educational Research Open 8 (2025) 100437
2

skills are foundational and prerequisite to becoming a procient reader,
so a focus on them is thought to give students a head start so that they get
to "read to learn" sooner and don’t fail to learn to read uently. How-
ever, many phonics-level interventions demonstrate impact on emergent
skills in the short term (knowledge of letters, basic decoding skills, or
gains in reading uency), but not on later reading comprehension skills
(Kim et. al., 2019; Rodriguez-Segura, 2022; Pfost et al., 2014). Also, tests
of foundational skills such as decoding and uency tend to be more
sensitive to small changes over a short time span, so these would be
expected to detect an intervention’s impact in the short term and to
report large effects for interventions focused only on decoding-level
skills.
Reading comprehension, while more nebulous and difcult to mea-
sure, is the ultimate goal of reading instruction, and thus arguably a
more meaningful outcome variable to measure the success of a literacy
intervention (Rodriguez-Segura, 2022). According to one review of 67
international studies, literacy interventions in developing countries
(with various foci) on average demonstrated a signicant impact on
reading comprehension test performance of participants, with an
average effect size of 0.25, which is considered a moderate (Kim et. al.,
2019), albeit smaller than the average effect size found on measures of
phonics tests or reading uency probes. However, the review took all
types of interventions together; in the present study we will report the
impact of an intervention focused specically on reading
comprehension.
1.4. The Guatemalan context
For students in low-SES communities globally, access to quality ed-
ucation and adequately trained teachers is a common problem. The
world faces a chronic shortage of trained, qualied teachers with the
skills to improve student learning (UNESCO, 2016). Enhancing teacher
preparation and providing ongoing support to teachers in their class-
rooms is critically important to achieving global goals to improve the
quality of education. These include the U.S. Government’s and USAID’s
goal to provide primary age children equitable access to quality edu-
cation, specically by improving the teaching and learning of literacy
and numeracy skills in the early grades (USAID, 2018).
In Guatemala, for example, most teachers only have a high school
degree, are trained in outdated teaching methods, and are placed in
classrooms lacking important resources, such as books. Guatemala is
Central America’s largest economy, yet it has the attributes of some of
the poorest nations in the world. In regards to education, it has the
lowest levels of public investment in children and adolescents in Central
America (USAID & fhi360, 2017). Guatemalans who are 15 years and
older have an average of 5.6 years of schooling. Indigenous Guatemalan
women have an average of 3.4 years of schooling. Although access to
education appears to be improving, Guatemala, like other developing
countries, also faces a learning crisis within its schools. At the end of the
rst grade, approximately 60 % of students cannot read a three-word
sentence. In 2015, only about one quarter of graduating students had
passed the national reading test (USAID & fhi360, 2017; Direcci´
on
General de Evaluaci´
on e Investigaci´
on Educativa DIGEDUCA, 2016).
In primary schools in many developing countries, literacy curriculum
frequently neglects the comprehension-based side of literacy education,
along with instruction which provokes critical thinking
(Rodriguez-Segura, 2022), even though many countries like Guatemala
have recently instituted reforms and new national curricula to address
this problem. Guatemala’s Ministry of Education has implemented
several national campaigns over recent decades to attempt to ameliorate
the country’s literacy crisis. The most recent was a one-time effort that
sent boxes of children’s books to schools across the country. However,
research has repeatedly shown that the simple provision of nancial or
physical resources without follow-up or professional development has
little effect on student learning (Masino & Ni˜
no-Zarazúa, 2016). For
example in Guatemala it was widely reported that the boxes of books
often stayed closed up in the school director’s ofce for fear of the books
being damaged or lost and teachers then charged; the students rarely
used them.
This realization has led to increased interest among governments to
increase the skill of the teaching workforce in pursuit of improving the
education of students (United Nations 2018). Many approaches have
been tried from supplying teachers with laptops loaded with excellent
teaching content to government-run teacher certication programs
(Curricula Nacional Base, n.d.). Several years ago, the Guatemalan
government instituted a requirement for some college-level work for
new teachers, but this would have little impact on the skill set of current
teachers or the many teachers already certied who are waiting for
teaching positions to open up. Thus, the effect of this additional training
has yet to be tested.
Child Aid began supporting literacy development in Guatemala by
providing books to community libraries and school libraries. But sub-
sequent observation revealed extremely limited use of community li-
braries and a larger problem. Not only were books almost totally
lacking, or sitting unused as previously mentioned, but literacy in-
struction was inadequate. Reading instruction was mostly limited to the
rst grade and focused on the mechanical lower-level literacy skills such
as letter sounds and decoding, or "learning to read". Instruction
including instilling a love for literature with read-alouds, improving
vocabulary knowledge, teaching comprehension strategies, and devel-
oping critical thinking, was absent because teachers had not been
trained in those areas.
The limited focus of instruction was compounded by a lack of literacy
resources. Guatemalan schools rarely have libraries, and there is very
little access to children’s literature or other types of texts. Textbooks are
provided but in insufcient quantity thus never to take home, often
don’t arrive until the middle of the school year, and then with many
courses and grades missing.
Additionally, teachers were observed to use a "traditional" approach
to instruction in general, sometimes termed "stand-and-deliver" or
"transmission" instruction, in which children sat in rows and spent most
of the day listening to the teacher lecture or waiting in line to have their
homework checked. Very little time each day was devoted to children
actively reading (opportunities for students to have their "eyes on text"),
and little or no exposure provided to high-interest children’s literature.
1.5. The intervention: child aid’s reading for life program
In order to address the problems of ill-trained teachers, under-
resourced schools, and limited access to quality books and reading
materials, Child Aid provides an integrated, four-year program of
teacher training, coaching, and provision of quality children’s literature
and other literacy resources, built on accepted best practices in
comprehensive primary school literacy pedagogy.
Child Aid works in small rural schools in agricultural communities in
the Guatemalan Highlands, the central/southwestern part of the coun-
try. The population is nearly one hundred percent Mayan and speak one
of Guatemala’s 23 Mayan dialects as a rst language, typically learning
Spanish at school. Both socioeconomic and second-language-learner
status are predictors of low reading achievement in terms of reading
comprehension ability, as children’s linguistic development and vo-
cabulary knowledge in both their rst and second languages are affected
by poverty. Noting the decit in the comprehension side of literacy in-
struction, combined with the low socioeconomic status of the commu-
nity, Child Aid found it prudent to focus its literacy intervention on
reading comprehension and critical thinking rather than on decoding.
Educational reform research has demonstrated that interventions
have the most impact when the goals of the intervention are aligned
with the national curriculum (Cohen & Hill, 2001). The content of the
Child Aid Program aligns with the national curriculum of Guatemala (htt
ps://www.mineduc.gob.gt/digecur/?p=CNB.asp), as both are based on
Marzano’s re-working of the Bloom taxonomy for education. Marzano’s
S.L. Peller et al.
International Journal of Educational Research Open 8 (2025) 100437
3

framework emphasizes the development of higher-order linguistic and
critical thinking skills that comprise a literacy focus beyond the more
mechanical early skills, with a focus on comprehension strategies, lin-
guistic development, vocabulary, writing, and meaningful engagement
with children’s literature (Marzano, 2007).
With certication by the Ministry of Education to offer the program
in three departments (states) in the Western highlands of Guatemala –
Chimaltenango; Solol´
a; and Totonicap´
an – Child Aid directs its resources
to public primary schools in these areas. Before the Child Aid program is
implemented, support from the School District Supervisor and then the
school director is obtained, and teacher meetings are held to secure the
“buy-in” of the teacher participants. Those initial agreements, plus the
designation of Child Aid workshops as “in-service” professional devel-
opment days when teacher attendance is mandatory and their students
are not in school, yields a program completion rate well above 90 %.
The specic program activities include participation in eight full-
day, highly interactive workshops attended by all the teachers in the
school twice each year, over the course of four years. The workshops are
facilitated by Child Aid Literacy Trainers assigned to each school. In
order to affect a paradigmatic shift in the approach of teachers to
teaching and literacy instruction, one-on-one coaching is an important
aspect of the program (Cornett & Knight, 2009). Following the full day
professional development workshops, each teacher is visited at least
three times by the Literacy Trainer for coaching sessions in her class-
room. These coaching sessions allow teachers to practice skills learned
in the workshop, while the Literacy Trainer coaches the teacher’s
application of the techniques and together they work on things like
lesson plans and small group work that are not part of teacher training in
Guatemala.
To address the lack of literacy resources in schools, Child Aid pro-
vides high quality, grade appropriate, ction and non-ction books to
participating schools. Working with teachers, Child Aid devised an easy
classication system, denoting grade level, content topic, and ction
versus nonction volumes. The program provides moveable bookshelves
so that books can be shared between classrooms when there is no
designated school library. These shelves can also be moved into the
schoolyard for access to reading during recess. In addition, Child Aid
strongly encourages book-lending programs in each school, and
schoolwide activities such as book fairs, reading clubs, and reading
competitions. Finally, we work with teachers on ways to incorporate
books across all classroom subjects and activities.
1.6. Purpose of the study
The purpose of the present study was to examine whether im-
provements in the reading comprehension skills of second and third
grade students in rural areas of Guatemala can be observed during
teachers’ rst year of participation in the Child Aid program. To deter-
mine whether Child Aid’s multifaceted intervention had a positive effect
on children’s reading comprehension ability, we conducted two large-
scale pre-post quasi-experimental studies. The two studies were virtu-
ally identical in design, with the rst serving as the initial study and the
second serving as a replication study. The latter study was designed to
evaluate whether similar effects would be observed in a different
geographical area in Guatemala which shares similar characteristics to
the rst region. Both studies measured students’ improvement on a
reading comprehension test during their schools’ rst year participating
in the Child Aid Reading for Life program, with expectations that students
in schools with Child Aid support would demonstrate greater gains in
their reading comprehension skills from the start of the school year to
the end. In addition to our evaluation of the main effects of Child Aid on
student’s reading comprehension skills, we also examined whether dif-
ferential effects were found for students in second and third grades, and
for students in the most remote schools compared to students in the
schools located in community centers. Hierarchical linear modeling
allowed us to explore these relationships within the nested structure of
the schools in our study, so we could explore these main effects and the
interactions between these variables in our two studies.
2. Methods and materials
2.1. Research design
We used a non-equivalent, quasi-experimental group research design
to compare growth in reading comprehension performance across one
school year, comparing the performance of students in schools entering
the Child Aid program with those in matched comparison schools. This
design is a useful research design for studies of educational interventions
where random assignment to treatment groups is not logistically feasible
(children attend their community school). In adherence to the standard
procedures of the testing protocol, all aspects of the testing process were
kept equal between groups to the extent possible.
2.2. Selection of comparison and experimental schools
Both the initial study and the replication study compared gains in
students’ reading between rst-year Child Aid schools and a set of
comparison schools. Schools were chosen to participate in the Child Aid
program based on logistical factors affecting program operations. Child
Aid works mainly with the poorer, more rural, smaller, and more
indigenous schools in the given area – the Western Highlands of
Guatemala. Within the states and districts where Child Aid was oper-
ating, new schools were selected based on proximity to Child Aid staff
who provide the services at the schools, and the size of the school to
balance staff travel and workloads.
Comparison schools were selected to match the intervention schools
on the following characteristics. First, geographic vicinity to the inter-
vention school is a proxy for many other similarities as the children
attending both schools are from the same community and thus share
similar background characteristics. Second, we attempted to match the
school size using the number of teachers as a proxy for size. Thirdly we
considered the rural-ness or urban-ness of the school. For this
geographical variable, most of the regions we worked in were rural
communities. Schools were coded as urban if they were situated in the
town or community center; schools were coded as rural if they were
situated in remote areas of the country. Comparison schools also shared
membership in the same linguistic and ethnic communities, the same
municipalities, and the same districts as the Child Aid schools, sug-
gesting they are comparable in terms of the cultural group membership
of students and staff, average economic level of families, parent
educational attainment, language, cultural value placed on schooling,
and access to print and computers/internet, among other variables.
Comparison schools were put on a waitlist to receive the Child Aid
Program within ve years of participating in the study.
2.3. Dependent variable
Choosing a measurement tool to assess reading skill in children is a
challenging decision, because the construct to be assessed – in this case,
reading comprehension - is not readily or easily dened, and the skills
being measured change with the age of the child and the focus of literacy
instruction in each grade. Although many literacy interventions focus on
foundational skills in the early grades such as alphabet knowledge and
decoding, Child Aid’s literacy intervention is focused on advanced lit-
eracy skills such as linguistic ability, reading comprehension and critical
thinking. Thus, for this study, we decided to test the effects of Child Aid
on the construct of reading comprehension directly.
Our Reading Comprehension Test was developed by the Guatemalan
Ministry of Education (MINEDUC) and USAID. The test items were
originally used in national assessments for rst- and third-grade students
in Guatemala. In 2014, a second-grade version was developed by a
doctoral student working in the Ministry, by combining the difcult
S.L. Peller et al.
International Journal of Educational Research Open 8 (2025) 100437
4

items from the rst-grade test with easier items from the third-grade
test. Three forms of the second-grade version were piloted to collect
evidence of validity on 75 potential items. We selected six of the four-
item sections to represent the full range of difculty and question type
for a total of 24 test items. We selected those sections with items
demonstrating better internal validity as reported by the Ministry, as
well as those with higher face validity, clearer presentation, and rep-
resenting each component ability we wished to assess.
The rst section asks students to identify an antonym for a given
word (vocabulary). The next three sections ask students to read a very
short passage, from one to three sentences long, and answer a question.
The second section asks for literal recall of information in the passage,
the third section asks students to identify the main idea of the passage,
and the fourth asks them to make a prediction regarding what could
happen next in the story. The fth section presents students with four
lines of text listing sequential actions out of order, and students are
asked to select the response that presents the actions in a logical order of
events (sequencing). The nal section of the test presents students with a
vocabulary word in a sentence or short passage and asks them to identify
which response contains a word that means the same or almost the same
as the target word.
Students were allowed one hour to complete the test. The test used a
group administration format. All 24 items were presented in a multiple-
choice format with three or four possible responses listed. Students were
given standardized directions and the evaluator completed two sample
questions with them along with detailed instructions. Teachers were
instructed not to offer or assist students as they took the test. Students’
rst language, frequently a Mayan dialect that the teacher also spoke,
could be used for instructional purposes only as needed, as we wish to
measure reading ability in Spanish, the main language of literacy in-
struction in Guatemala.
2.4. Data collection
For the data collection phase, a team of evaluators was hired and
trained prior to each testing session. The number of evaluators needed
was determined according to the number of schools participating in that
testing session. Each evaluator was assigned to 20–23 schools and
typically visited one to two schools each day to administer the test.
In February and March, the Reading Comprehension Test was
administered to all second and third grade students attending schools
entering Child Aid’s program that year, and a roughly equal number of
schools and students attending matched comparison schools. Evaluation
days were staggered between groups to ensure equivalency in terms of
the time-of-year the test was taken, and all tests were completed within
the six-week window. For schools participating in the Child Aid pro-
gram, the test was administered prior to the rst Child Aid teacher
workshop to minimize any potential inuence of the program at base-
line. During a six-week window in September and October, the same test
was administered to the same students for the post-test measurement.
During the year neither teachers nor students were given any access to
the test, to avoid the possibility of studying or discovering and
remembering the correct responses at the end of the year. For the
replication study, we followed identical data collection procedures,
albeit with some further procedures to ensure data integrity. In this
study, two different people entered all the data so each piece of data –
items and total scores – was compared for accuracy.
In the initial sample, 29 schools in the department of Solol´
a began
the Child Aid program and were included in the study, with 25 com-
parison schools. A total of 3271 students took the reading comprehen-
sion pre-test, including 1682 students in the intervention group and
1589 in the comparison group. Only students with both pre- and post-
tests were then included in the study. After eliminating students with
a missing posttest score, a total of 2861 were included in the initial
study, with 1460 participants in the intervention group and 1401 stu-
dents in the comparison group. The attrition rate for this sample was 13
%.
In the replication study, 32 schools began the Child Aid program and
were included, and 39 comparison schools participated. A total of 2869
students in all took the pre-test, including 1239 students in the Child Aid
group and 1630 students in the comparison group. Only students with
both pre- and post- tests were included in the study, causing some degree
of attrition in the ultimate sample (15.1 %). After eliminating students
with only one score, 1334 participants made up the intervention group
and 1668 made up the comparison group.
2.5. Data analysis
Our analyses examined whether differences in reading comprehen-
sion gain scores could be attributed to the Child Aid program, and
whether gain score differences were observed at different grade levels
(second and third grades) and in different geographical regions of
Guatemala (urban/centrally located schools or rural/remote schools).
To evaluate whether Child Aid’s intervention had a positive effect on
children’s reading prociency, we conducted three hierarchical multi-
level models for each separate dataset. We analyzed the two academic
years separately because we wanted to replicate the study in a different
geographical region in Guatemala to discern program effects in a region
that shared similar sociocultural characteristics to the rst region. Model
1 evaluated the unconditional differences to explore how much variance
in gain scores existed between schools and within schools? This un-
conditional or null model included no predictors and only partitioned
the variance into its within-group and between-group components to
determine the baseline variability in reading comprehension gain scores
and to calculate the intraclass correlation coefcient (ICC), indicating
the proportion of the total variance attributed at the school level. Model
2 explored the main effects to evaluate whether participation in Child
Aid affected student gains in reading comprehension outcomes and
whether the effects varied by students’ initial comprehension skills. In
this model, we looked at differences in gains scores of students in second
and third grades, differences in gains for students in urban schools
versus rural schools, and differences in gains for students in Child Aid
schools compared to schools that did not receive resources and training
from Child Aid. The nal model, Model 3, explored interactions between
the treatment status of the school (treatment vs. control), the
geographical region of the school (urban vs. rural), students’ grade level
(grade 3 vs. grade 2) affected reading comprehension gain scores, while
accounting for students’ initial reading comprehension skills as measure
in their pretest scores. These three models were tested separately for the
initial dataset and the replication dataset.
3. Results
Students in both the Child Aid Schools and the comparisons schools
showed improvement over the span of the school year, in both studies,
with average gains from pretest to post test of 3.21 and 3.79 in the
second-grade sample and 2.35 and 3.43 in the third-grade sample
(Table 1).
3.1. Initial study
Initial Model 1: Unconditional/null model. This model describes
the variance between schools (between-group) and students (within-
group). This model revealed that the average gain score across all
schools was estimated at 3.93 (SE =0.32, p <0.001), indicating a sta-
tistically signicant positive gain for students on average (see Table 2).
Gains such as these are expected when measuring growth in reading
because reading ability for elementary-aged children will nearly always
improve over time.
The variance in gain scores between schools was 5.25, with a stan-
dard deviation of 2.29, suggesting that there was substantial variability
in student performance across the schools in this sample. The residual
S.L. Peller et al.
International Journal of Educational Research Open 8 (2025) 100437
5

variance within schools (i.e., variability among students within the same
school) was 20.44, with a standard deviation of 4.52. This indicates that
most of the variation in gain scores occurred at the student level, but
there was still a meaningful variation between schools. The intraclass
correlation coefcient (ICC), which measures the proportion of total
variance attributable to differences between schools, was calculated as
0.20. Thus, approximately 20 % of the variability in gain scores can be
attributed to school-level differences, justifying the use of multilevel
modeling techniques for further analysis.
Initial Model 2: Random intercept model with student-level and
school-level predictors. This main effects model examined the inu-
ence of students’ pretest reading comprehension scores, their grade
level, treatment status, and the geographical region of their school
(urban or rural) on their reading comprehension gain scores. The
intercept, representing the mean gain score, was estimated at 9.21 (SE =
0.86, p <0.001), indicating a signicant average gain across all schools,
as evident in the rst unconditional model (see Table 3). Pre-test scores
had a negative effect on gain scores, with an estimate of −0.50 (SE =
0.02, p <0.001), suggesting that students with higher pre-test scores
made smaller gains across the year. Grade level had a positive effect on
gain scores, with third grade students showing an estimated gain that
was 1.10 points higher than second grade students (SE =0.16, p <
0.001). Importantly, this model found the treatment status of the school
was a signicant predictor, with students in Child Aid schools gaining an
estimated 1.37 points in their reading comprehension scores more than
students in control schools (SE =0.60, p =0.028). This effect did not
signicantly vary based on the geographical location of the schools,
when comparing gains scores between students in urban and rural
schools, (estimate = − 1.09, SE =0.84, p =0.202).
Initial Model 3: Random intercept model with student-level and
school-level predictors and interaction terms. The nal model
examined the effects of pre-test scores, students’ grade level, the treat-
ment condition, and geographical region, and the interactions of these
predictors on students reading comprehension gain scores. As observed
in Model 2, higher pre-test scores were associated with signicantly
lower gain scores (estimate = − 0.50, SE =0.02, p <0.001), indicating
that students who started with higher initial scores had less room for
improvement (see Table 4). However, when interactions were added to
the analyses in this model, the main effects of grade level (estimate =
−0.53, SE =0.36, p =0.141), school treatment (estimate =0.62, SE =
1.60, p =0.702), and school type (estimate = − 1.32, SE =1.22, p =
0.286) were not statistically signicant when considered individually.
This analysis revealed signicant interactions between grade level
and the treatment condition (estimate =3.06, SE =0.66, p <0.001) as
well as grade level and geographical region (estimate =1.25, SE =0.45,
p =0.005). Specically, third grade students who were in the Child Aid
schools showed signicantly higher gains than third-grade students in
the control schools. Additionally, third grade students enrolled in the
rural schools had higher gains than the third graders in the urban
schools in our sample. Although there was not an interaction between
the treatment condition and the geographical setting of the school (es-
timate =0.12, SE =1.74, p =0.947), the three-way interaction between
grade level, treatment condition, and geographical setting was signi-
cant (estimate = − 1.97, SE =0.75, p =0.008). This three-way inter-
action revealed that the effects of Child Aid varied depending on both
students’ grade level and whether the school was in a remote rural
setting or a more centrally located town center.
In conclusion, in the initial study, students in treatment schools
demonstrated better gains than students in control schools, particularly
for third-grade students, and the effect was more pronounced in remote
schools that were far from community centers where curricular re-
sources are often scarce.
Table 1
Descriptive statistics for the initial and replication samples.
Initial Study
Pre-test Mean
(SD)
Post-test Mean
(SD)
Mean Gain Score
(SD)
Second Third Second Third Second Third
Total Sample
(N =3271)
10.02
(4.48)
13.17
(5.05)
13.94
(5.27)
16.76
(4.93)
3.79
(4.95)
3.43
(4.81)
Child Aid
(n =1682)
9.26
(4.19)
12.36
(4.73)
13.56
(5.03)
17.13
(4.75)
4.10
(5.12)
4.62
(4.72)
Control
(n =1589)
10.82
(4.64)
14.04
(5.27)
14.33
(5.48)
16.36
(5.09)
3.46
(4.75)
2.18
(4.57)
Replication Study
PreTest Mean
(SD)
PostTest Mean
(SD)
Mean Gain Score
(SD)
Second Third Second Third Second Third
Total
Sample
(N =
2869)
11.15
(4.66)
13.94
(5.09)
14.35
(4.98)
16.29
(4.96)
3.21
(4.67)
2.35
(4.22)
Child Aid
(n =
1239
10.68
(4.55)
14.06
(5.18)
14.45
(4.97)
16.70
(4.93)
3.77
(4.29)
2.64
(3.95)
Control
(n =
1630)
11.50
(4.70)
13.84
(5.0)
14.28
(4.98)
15.97
(4.96)
2.78
(4.90)
2.13
(4.41)
Table 2
Fixed and random effects for Model 1 of the initial study.
Fixed effect Parameter Coefcient SE p-value
Intercept γ
00
3.93 0.32 p <0.001
    
Random effect Parameter Coefcient SD
Level 2 variance
τ
00
=Var(u
0j
) 5.25 2.29 
Level 1 variance
σ
2
=Var(r
ij
) 20.44 4.52 
Table 3
Fixed and random effects for Model 2 of the initial study.
Fixed effects Parameter Coefcient SE p-value
Intercept γ
00
9.21 0.86 p <0.001
Pre-test score γ
10
−0.50 0.02 p <0.001
Grade level γ
20
1.10 0.16 p <0.001
School treatment γ
01
1.37 0.60 p =0.028
School type γ
02
−1.10 0.84 p =0.202
Random effects Parameter Coefcient SD
Level 2 variance
τ
00
=Var(u
0j
) 4.50 2.12 
Level 1 variance
σ
2
=Var(r
ij
) 16.05 4.01 
Table 4
Fixed, random, and interaction effects for Model 3 of the initial study.
Fixed effects Parameter Coefcient SE p-value
Intercept γ
00
9.63 1.13 p <0.001
Pre-test score γ
10
−0.50 0.02 p <0.001
Grade level γ
20
−0.53 0.36 p =0.141
School treatment γ
01
0.62 1.60 p =0.702
School type γ
02
−1.32 1.22 p =0.286
Interaction terms    
Grade: Treatment γ
21
3.06 0.66 p <0.001
Grade: Type γ
22
1.25 0.45 p =0.005
Treatment: Type γ
03
0.12 1.74 p =0.947
Grade: Treatment: Type γ
23
−1.97 0.75 p =0.008
Random effects Parameter Coefcient SD
Level 2 variance
τ
00
=Var(u
0j
) 4.61 2.15 
Level 1 variance
σ
2
=Var(r
ij
) 15.84 3.98 
S.L. Peller et al.
International Journal of Educational Research Open 8 (2025) 100437
6

3.2. Replication study
Replication Model 1: Unconditional/null model. This analysis
included no predictors and only partitioned the variance into its within-
group and between-group components to describe the variability in
reading comprehension gain scores and to summarize the total variance
attributable to the school-level grouping variable. This analysis revealed
that the average gain score across all schools was estimated at 2.84 (SE =
0.24, p <0.001), indicating a statistically signicant positive gain for
students on average (see Table 5). The variance in gain scores between
schools was 2.53, with a standard deviation of 1.59, suggesting mod-
erate variability in student performance across schools. The residual
variance within schools (i.e., variability among students within the same
school) was found to be 18.47, with a standard deviation of 4.30.
Therefore, most of the variation in gain scores occurred at the student
level, yet there was still a meaningful variation between schools in this
sample. The intraclass correlation coefcient (ICC), which measures the
proportion of total variance attributable to differences between schools,
was calculated as 0.12, with approximately 12 % of the variability in
gain scores attributable to school-level differences, justifying the use of
multilevel modeling techniques for further analysis.
Replication Model 2: Random intercept model with student-
level and school-level predictors. The main effects model examined
the inuence of pre-test scores, students’ grade level, treatment condi-
tion, and geographical location of the student’s school on reading
comprehension gain scores. The intercept, representing the mean gain
score, was estimated at 7.79 (SE =0.40, p <0.001), indicating a sig-
nicant average gain across all schools (see Table 6). As in the initial
study, pre-test scores had a signicant negative effect on gain scores,
with an estimate of −0.43 (SE =0.02, p <0.001), suggesting that stu-
dents with higher pre-test scores had smaller gains. Although not as
large as the differences found in the initial study, Grade 3 students
showed an estimated gain that was 0.30 points higher than Grade 2
students (SE =0.1, p =0.050). The treatment status was also a signi-
cant predictor, with students in Child Aid schools having an estimated
0.88 points higher gain in reading comprehension performance than
students who were not in Child Aid schools (SE =0.39, p =0.029).
Similar to the ndings in the initial study, geographical setting of the
school (urban vs. rural) did not have a statistically signicant effect on
gain scores (estimate = − 0.41, SE =0.39, p =0.301).
Replication Model 3: Random intercept model with student-
level and school-level predictors and interaction terms. This nal
model examined the effects of pre-test scores, students’ grade level,
school treatment condition, and geographical region, as well as their
interactions, on gain scores. Again, the pre-test scores had a signicant
negative effect on gain scores (estimate = − 0.43, SE =0.02, p <0.001),
indicating that students with higher initial scores had smaller gains in
their scores (see Table 7). Grade level also showed a signicant positive
effect on gain scores (estimate =0.55, SE =0.24, p =0.022), with Grade
3 students showing higher gains compared to Grade 2 students. How-
ever, neither the treatment status of the school (estimate =0.98, SE =
0.63, p =0.128) nor the school type (urban vs. rural) (estimate = − 0.30,
SE =0.57, p =0.602) had statistically signicant effects on gain scores
when considered individually.
The interaction between grade level and school treatment was not
statistically signicant (estimate = − 0.48, SE =0.35, p =0.176), sug-
gesting that the relationship between treatment status and gain scores
did not vary signicantly by grade level in this replication study. The
interaction between grade level and school type was also not statistically
signicant (estimate = − 0.45, SE =0.40, p =0.272). Similarly, the
interaction between treatment status and school type was not signicant
(estimate = − 0.14, SE =0.86, p =0.871).
The three-way interaction between grade level, school treatment,
and school type was also not statistically signicant (estimate =0.77, SE
=0.64, p =0.232), indicating that the combined effects of these vari-
ables did not signicantly affect gain scores in this replication study. In
conclusion, pre-test scores and grade level were the primary signicant
predictors of gain scores in this model, while the interactions between
school treatment, school type, and grade level did not yield signicant
effects as was seen in the rst study.
4. Discussion
Students from low-SES communities around the world often do not
have access to quality education and instructional resources and mate-
rials such as books. When they do have access to schooling, their
teachers are frequently less well-trained than teachers in more resourced
communities and have less access to continuing their professional
development. Therefore, opportunities for high student achievement are
limited. Enhancing teachers’ skills while providing ongoing support to
them in their classrooms might be key to achieving global goals to
improve the quality of education for all children. Specically, the pre-
sent study shows that providing students with reading materials and
their teachers with evidence-based training and coaching to incorporate
student literacy activities into their instruction and classroom routines
may provide students with more opportunities to develop strong reading
skills. An important aspect of the Child Aid intervention is its scalability
and feasibility. By setting up a hierarchical training of teachers, Child
Aid was able to reach a signicant number of teachers and children.
Equally important is the need to evaluate the effectiveness of the
approach.
Previous research on interventions in primary schools in developing
countries that impact student learning supports Child Aid’s three-prong
intervention to improve reading prociency (Rodriguez-Segura, 2022).
Reviewing the literature suggests that teacher professional development
can have an impact on student learning when it is individualized and
Table 5
Fixed and random effects for Model 1 of the replication study.
Fixed effect Parameter Coefcient SE p-value
Intercept γ
00
2.84 0.24 p <0.001
    
Random effect Parameter Coefcient SD
Level 2 variance
τ
00
=Var(u
0j
) 2.53 1.59 
Level 1 variance
σ
2
=Var(r
ij
) 18.47 4.30 
Table 6
Fixed and random effects for Model 2 of the replication study.
Fixed effects Parameter Coefcient SE p-value
Intercept γ
00
7.79 0.40 p <0.001
Pre-test score γ
10
−0.43 0.02 p <0.001
Grade level γ
20
0.30 0.15 p =0.050
School treatment γ
01
0.88 0.39 p =0.029
School type γ
02
−0.41 0.39 p =0.301
Random effects Parameter Coefcient SD
Level 2 variance
τ
00
=Var(u
0j
) 1.65 1.28 
Level 1 variance
σ
2
=Var(r
ij
) 14.66 3.83 
Table 7
Fixed, random, and interaction effects for Model 3 of the replication study.
Fixed effects Parameter Coefcient SE p-value
Intercept γ
00
7.79 0.40 p <0.001
Pre-test score γ
10
−0.43 0.02 p <0.001
Grade level γ
20
0.30 0.15 p =0.050
School treatment γ
01
0.88 0.39 p =0.029
School type γ
02
−0.41 0.39 p =0.301
Random effects Parameter Coefcient SD 
Level 2 variance
τ
00
=Var(u
0j
) 1.65 1.28 
Level 1 variance
σ
2
=Var(r
ij
) 14.66 3.83 
S.L. Peller et al.
International Journal of Educational Research Open 8 (2025) 100437
7

repeated, evidence-based, when teachers are taught to match instruction
to student needs and how to use specic techniques and methodologies,
ongoing support is included, and aligned materials are provided. The
results of the present study add further evidence to support these
ndings.
Although the Child Aid program has a strong theoretical underpin-
ning, the outcomes of this program had not been tested prior to this
study. The purpose of the present study was to examine the effects of the
Child Aid program on the reading prociency of second- and third-grade
students throughout rural areas of Guatemala. Using two separate co-
horts of approximately 3000 students in each study, we tested whether
one year’s involvement with the Child Aid program and resources
resulted in better gains in reading comprehension scores on a reading
comprehension test compared to the gains of students in schools that
were not participating in Child Aid.
We tested the main effects of our four predictors and discovered
three signicant main effects. First, we tested whether students’ initial
reading comprehension skills at the start of the school year predicted
their gain scores at the end of the year. In both studies, we discovered
students who had higher initial reading comprehension skills demon-
strated the smallest gains across the year. This might indicate that stu-
dents with the strongest skills had less progress to make on this one test.
Then we examined whether there were differences in gain scores for
students in second grade compared to students in third grade. Again,
both studies revealed the same results; third graders made signicantly
greater gains in their reading comprehension scores than did second
graders. These grade level effects were most salient in the initial study,
when third graders had estimated gains of 1.10 higher than second
graders in this study compared to the 0.30 gain score difference
observed in the replication study. We also studied the main effects of
reading comprehension improvements predicted by geographical loca-
tion. In this study, “urban” describes schools in the center of commu-
nities, whereas “rural” describes smaller schools situated in remote
locations of the country. We wanted to see if there were general dif-
ferences in the gains of students from these two geographical designa-
tions, and neither study found a signicant difference. We can conclude
reading comprehension gains do not signicantly differ between stu-
dents in these more centrally-located schools compared to students in
more remote regions.
Finally, we tested the main effects of the Child Aid program.
Although the schools in the present study had only participated in the
Child Aid program for one year, with three additional years to follow,
students in Child Aid schools consistently demonstrated signicantly
greater gains in their reading comprehension skills than students who
were not in Child Aid schools. In both studies, the treatment status of the
school was a signicant predictor, with students in Child Aid showing
gains than students in control schools. In the initial study, on average,
both second and third graders made gains greater than 4 points on the
standardized reading comprehension test compared to gains of 3.5 and
2.1 respectively for second and third graders in the control condition. In
the replication study, signicant treatment effects were also observed,
albeit with smaller observed effects than were seen in the rst study. To
make sense of the differences in the overall effect of Child Aid, we looked
at the descriptive statistics and found that students in the initial study
had much lower scores at the start of the study than the students in the
replication study. The differences in the results of these two studies is
consistent with the ndings within each study, where we found that
students with strong initial skills made smaller gains across the year than
did students with lower initial reading comprehension skills. This
nding is important, as programs, such as Child Aid, have the goal of
improving the reading achievement of students who have the most need.
Lastly, we explored the interactions between these predictors. In
each study, we found that students in the Child Aid schools had more
gains in their reading comprehension scores than children who were not
in Child Aid Schools. However, this effect was not statistically signicant
when interaction terms were introduced to each dataset, and no
signicant interactions were found in either the initial study or the
replication study. This suggests that the effectiveness of the Child Aid
program was consistent across grade levels and geographical regions
(urban and rural). Overall, we can conclude from these two studies that
the Child Aid Program is benecial to students across Guatemalan
schools, and its impact does not seem to vary across different types of
schools or for students in different grades.
These positive ndings support implementing scalable programs
such as Child Aid in the developing world, with its multipronged
approach and focus on students’ critical thinking and reading compre-
hension skills. Child Aid’s program stands out because of this focus,
which is a frequently neglected area of literacy instruction in impov-
erished communities, as well as its scalability.
5. Limitations and further research needed
Despite all efforts to control threats to the valid interpretation of our
results, it is difcult to manage all the various interferences to a quasi-
experimental study in school settings, particularly in the vast rural set-
tings where this study was conducted. For this study, we observed the
largest effects for students with the greatest need. It is possible these
results are simply an artifact of regression to the mean for our lowest
performing students. We plan to continue to investigate the effects of the
Child Aid program across Guatemala to continue to test our current
ndings and further evaluate whether this is a true effect or simply a
statistical artifact. Important to the context of this study and future
replication opportunities was the global COVID-19 pandemic, during
which students across the world did not access schooling opportunities.
Children in Guatemala, like developing nations and less resourced
communities across the world, had no access to schools for almost two
full years. The result is that students in second and third grades in the
past few years are still learning how to read. It will take time before we
are able to test reading comprehension outcomes for these grade levels.
The most complicated threat to the results of our study are the high
attrition rates we see across rural parts of Guatemala. In our rst study,
approximately 1500 students were in our pre-test sample who were not
in school for the post-test. We examined whether there was a statistical
difference in the pre-test scores for students who participated in the full
study compared to those who were absent during the post-testing period.
We did nd a statistically signicant difference between the pretest
means of the initial student sample; students with missing data had a
slightly larger average score of 11.9 compared to students with complete
data who had average pretest scores of 11.6 (t(4285) = − 2.04, p =0.04);
however the effect size as measured by Cohen’s d for this difference may
be considered trivial (Cohen’s d = − 0.06) indicating that these two
groups are likely not meaningfully different in their reading compre-
hension skills.
For the replication study, we observed a much better study
completion rate, however there were still 571 students missing from our
post-test sample. For these students, we observed students with missing
data had signicantly lower scores than students who completed the
study, with pretest means of 11.6 compared to 12.6 (t(3539) =4.7, p <
0.001) with a small effect size (Cohen’s d =0.16). These high attrition
rates can be expected in rural parts of Guatemala where school-age
children in agricultural communities work with their families to pick
coffee or harvest food at the end of the school year when harvest seasons
begins, or for other economic and social priorities.
The mixed results we found between these two studies in the skills of
students from whom we have incomplete data suggest that we need to
further investigate whether some of the statistical effects we observed
were related variations in attrition or other reasons that may interfere
with the results of our studies. For this reason we continue to complete
pre-post case-control investigations to study the effect of the rst year of
the program each time we begin to work in a new area. In 2020 we
suspended the investigation as schools shut down due to the COVID-19
global pandemic. In 2023 we began working in a new area of Guatemala,
S.L. Peller et al.
International Journal of Educational Research Open 8 (2025) 100437
8

Quiche, and are in the process of analyzing the student testing data we
have collected during the rst year.
We also have further questions regarding the effectiveness and
impact of Reading for Life. As it is a multifaceted program, we would like
to analyze the relative impact of each of the three pillars of the program:
teacher training, coaching, and book/resource provision. It is also
possible that another factor such as encouraging book lending, children
reading during recess, or variation among teachers is responsible for a
portion of the effect. We are also in the process of conducting structured
observations of classrooms in schools enrolled in the program to deter-
mine what effect the program could have on teacher behavior, what
students are doing and assigned to do, and/or the amount of time spent
on various activities, especially the amount time spent on literacy ac-
tivities and incorporation of literacy activities into classes.
This study only tested differences in students’ gains across the rst
year of Child Aid intervention. This year can be described as an initial
implementation year, when schools receive books and teachers receive
initial training and coaching related to student reading development and
evidence-based instructional strategies. Our goal is to test student out-
comes in schools that have had the full three years of the program, and to
explore whether teachers who received training and support from the
Child Aid coaches have students who display stronger reading pro-
ciency skills than the students of teachers who are not involved in the
Child Aid program. We would also like to nd out if long-term benet is
derived from progressive years in the program, and whether it affects the
rate at which a participating schools’ students graduate from primary
school, or the rate at which they are promoted from one grade to the
next.
Child Aid provides a comprehensive school-wide literacy program in
rural Guatemala aimed at improving reading prociency and critical
thinking skills by increasing the quality of instruction provided by
teachers and access to quality literacy resources. This paper presented
two studies using a quasi-experimental design to answer whether Child
Aid’s Reading for Life program impacts growth in basic reading
comprehension skill during its rst year of implementation among sec-
ond and third-graders in rural parts of the Guatemalan highlands, where
the wide majority of the families are indigenous and speak a Mayan
dialect. Results demonstrated a positive overall effect in terms of greater
gains by students in Child Aid schools than comparison students. These
results are encouraging for interventions hoping to realize a positive
impact on student learning in developing countries with an evidence-
based, comprehensive approach.
Funding
This research did not receive any specic grant from funding
agencies in the public, commercial, or not-for-prot sectors. As specied
in the competing interests document, two of the four authors are em-
ployees of the organization whose intervention is the subject of the
program evaluation described in the manuscript.
CRediT authorship contribution statement
Sarah L. Peller: Writing – review & editing, Writing – original draft,
Visualization, Validation, Supervision, Resources, Project administra-
tion, Methodology, Investigation, Data curation, Conceptualization.
Amanda M. Marcotte: Writing – review & editing, Writing – original
draft, Validation, Methodology, Formal analysis, Data curation. Ketan:
Formal analysis, Writing – review & editing. Craig S. Wells: Validation,
Software, Methodology, Formal analysis, Data curation. Nancy Press:
Writing – review & editing, Writing – original draft, Supervision, Re-
sources, Project administration, Methodology, Investigation, Funding
acquisition, Conceptualization. Maciej Kos: Conceptualization, Data
curation.
Declaration of competing interest
The authors declare the following nancial interests/personal re-
lationships which may be considered as potential competing interests:
Sarah Peller and Nancy Press are employees at Child Aid. The article
presents the results of an evaluation of the work of Child Aid.
Acknowledgements
No further acknowledgements.
References
Bean, R. M. (2014). The power of coaching: Improving early grade reading instruction in
developing countries. Global Reading Network. USAID.
Berkman, N. D., Dewalt, D. A., & Pignone, M. P. (2004). Literacy and Health Outcomes:
Summary. AHRQ evidence report summaries. Rockville (MD): Agency for Healthcare
Research and Quality (US), 1998-2005. 87.
Blanchard, M. (2023). The relationship between socioeconomic status and literacy: How
literacy is inuenced by and inuences SES. Michigan Journal of Economics.
Cohen, D. K., & Hill, H. C. (2001). Learning policy: When state education reform works.
Binghamton, NY: Vail-Ballou.
Cornett, J., & Knight, J. (2009). Research on coaching. In J. Knight (Ed.), Coaching:
Approaches and Perspectives. Thousand Oaks, CA: Corwin Press.
Curricula Nacional Base. https://www.mineduc.gob.gt/digecur/?p=CNB.asp,
Guatemala, Central America.
Direcci´
on General de Evaluaci´
on e Investigaci´
on Educativa DIGEDUCA. (2016).
Resultados generales de evaluaci´
on educativa. Guatemala: Ministerio de Educaci´
on.
Elliott, S. (2017). Computers and the future of skill demand. Educational research and
innovation. Paris, France: OECD.
Evans, D. K., & Popova, A. (2015). What really works to improve learning in developing
countries?. Policy research working paper (p. 7203). Washington, D.C.: The World
Bank.
Goe, L. (2007). The link between teacher quality and student outcomes: A research synthesis.
Washington, D.C: National Comprehensive Center for Teacher Quality.
Graham, J., & Kelly, S. (2013). How effective are early grade reading interventions?.
Washington, D.C: The World Bank.
Hanushek, E. A., & Woessmann, L. (2007). The role of education quality for economic
growth. Washington, D.C: The World Bank. World Bank Policy Research Working
Paper No. 4122.
Hoover, W. A., & Gough, P. B. (1990). The simple view of reading. Reading and Writing, 2,
127–160.
Kim, Y. G., Lee, H., & Zulkowski, S. S. (2019). Impact of literacy interventions on reading
skills in low and middle-income Countries: A meta-analysis. Child Development, 9(2).
Knight, J. (2009). Coaching. Journal of Staff Development, 30(1). Oxford University Press.
Kremer, M., Brannen, C., & Glennerster, R. (2013). The challenge of education and
learning in the developing world. Science (New York, N.Y.), 340, 297–300, 6130.
Krishnaratne, S., White, H., & Carpenter, E. (2013). Quality education for all children?
What works in education in developing countries. New Delhi: International Initiative for
Impact Evaluation (3ie). Working Paper No. 20.
Marzano, R. J. (2007). The art and science of teaching: A comprehensive framework for
effective instruction. Alexandria, VA: Association for Supervision and Curriculum
Development.
Masino, S., & Miguel Ni˜
no-Zarazúa, M. (2016). What works to improve the quality of
student learning in developing countries? International Journal of Educational
Development, 48, 53–65.
McEwan, P. J. (2015). Improving learning in primary schools of developing countries: A
meta-analysis of randomized experiments. Review of Educational Research, 85,
353–394.
Morrow, L. M., Morrell, E., Casey, H. K., & Gambrell, L. B. (2023a). A historical look at
literacy instruction best practices from past to present. In L. M. Morrow, E. Morrell,
& H. K. Casey (Eds.), Best practices in literacy instruction: 7th edition. New York: The
Guilford Press (a).
Morrow, L. M., Dougherty, S. M., & Tracey, D. H. (2023b). Best practices in early literacy.
In L. M. Morrow, E. Morrell, & H. K. Casey (Eds.), Best practices in literacy instruction:
7th edition. New York: The Guilford Press (b).
National Reading Panel (U.S.) & National Institute of Child Health and Human
Development (U.S.). (2000). Report of the national reading panel: Teaching children to
read: An evidence-based assessment of the scientic research literature on reading and its
implications for reading instruction.
OECD Unesco. (2019). PISA 2018 results (Volume I): What students know and can do. Paris,
France: PISA OECD.
Pfost, M., Hattle, J., Dorer, T., & Artelt, C. (2014). Individual differences in reading
development: A review of 25 years of empirical research on matthew effects in
reading. The Review of Educational Research, 84(2).
Pepsen, A. (2019). Coaching in early grade reading programs: Evidence, experience and
recommendations. USAID: Global Reading Network.
Rodriguez-Segura, D. (2022). A closer look at reading comprehension: Experimental
evidence from Guatemala. International Journal of Educational Development, 93.
Smiley, A., Cao, Y., Moussa, W., Dooley, B., & Sullivan, J. (2020). Examining "best
practices" for literacy coaching and monitoring: Evidence from Northern Nigeria and
Ghana. Social Sciences & Humanities Open, 2(1).
S.L. Peller et al.
International Journal of Educational Research Open 8 (2025) 100437
9

Torrente, C., Moore, A., Liuzzi, S. H., Morgan, S., & Lapadatova, G. (2019). Guatemala
education project baseline report. Washington, DC: Mathematica Policy Research.
Unesco Institute for Statistics. (2016). The world needs almost 69 million new teachers to
reach the 2030 education goals. Fact Sheet No. 39. Paris, France: UNESCO.
Unesco Institute for Statistics. (2017). More than one-half of children and adolescents are
not learning worldwide. Fact Sheet No. 46, UIS/FS/2017/ED/46. Paris, France:
UNESCO.
United Nations. (2018). The 2030 agenda and the sustainable development goals: An
opportunity for Latin America and the Caribbean (LC/G.2681-P/Rev.3). Santiago, Chile:
United Nation.
USAID & FHI 360 (2017). Guatemala labor market assessment. New York, NY, United
States.
USAID. (2018). USAID Education Policy. Washington, D.C.
Vazquez-Lopez, V., & Huerta-Manzanilla, E. L. (2021). Factors Related with
Underperformance in Reading Prociency, the Case of the Programme for
International Student Assessment 2018. European Journal of Investigation in Health,
Psychology and Education, 11(3), 813–828.
Wenglinsky, H. (2000). How teaching matters: Bringing the classroom back into
discussions of teacher quality. Educational Testing Service, Princeton, NJ.
The World Bank. (2017). World development report 2018: Learning to realize education’s
promise. Washington, D.C: The World Bank.
S.L. Peller et al.
International Journal of Educational Research Open 8 (2025) 100437
10