ArticlePDF Available

A one-to-one programme for at-risk readers delivered by older adult volunteers

Abstract and Figures

This paper is based on a randomized controlled trial (RCT) evaluation of a reading programme delivered by older adult volunteers for at-risk early readers. Wizards of Words (WoW) was targeted at socially disadvantaged children in first and second grade experiencing delays in reading but who were not eligible for formal literacy supports. The programme was effective for phonemic awareness, word recognition, phonic knowledge and children's self-beliefs, but was not effective for reading comprehension, vocabulary, spelling or reading accuracy. The programme was most effective for those children starting with ‘below average’ reading levels and for boys. Programme intensity, school attendance and the child's experience of the programme all predicted response to intervention. Gains in phonemic awareness and phonic knowledge may be explained by the priority given in volunteer training and in programme delivery to the phonics component, and gains in word recognition may be explained by its close association with phonemic awareness and phonic knowledge, as hypothesized by the Simple View of Reading. The findings show that a reading programme delivered by older adult volunteers can have a significant impact on reading skills and self-beliefs of at-risk readers who are not eligible for other formal literacy supports.
Content may be subject to copyright.
A one-to-one programme for at-risk readers
delivered by older adult volunteers
Allyn Fives
1
*, Noreen Kearns
1
, Carmel Devaney
1
,
John Canavan
1
, Dan Russell
2
, Rena Lyons
1
, Patricia Eaton
1
and Aoife O’Brien
1
1
NUI Galway and
2
Iowa State University
This paper is based on a randomized controlled trial (RCT) evaluation of a reading programme
delivered by older adult volunteers for at-risk early readers. Wizards of Words (WoW) was targeted
at socially disadvantaged children in first and second grade experiencing delays in reading but who
were not eligible for formal literacy supports. The programme was effective for phonemic aware-
ness, word recognition, phonic knowledge and children’s self-beliefs, but was not effective for read-
ing comprehension, vocabulary, spelling or reading accuracy. The programme was most effective
for those children starting with ‘below average’ reading levels and for boys. Programme intensity,
school attendance and the child’s experience of the programme all predicted response to interven-
tion. Gains in phonemic awareness and phonic knowledge may be explained by the priority given in
volunteer training and in programme delivery to the phonics component, and gains in word recogni-
tion may be explained by its close association with phonemic awareness and phonic knowledge, as
hypothesized by the Simple View of Reading. The findings show that a reading programme
delivered by older adult volunteers can have a significant impact on reading skills and self-beliefs
of at-risk readers who are not eligible for other formal literacy supports.
Wizards of Words (WoW) is a one-to-one reading programme delivered by older
adult volunteers for socially disadvantaged children in first and second grade at-risk
of reading failure. Volunteers are aged 55 and over, and receive training and ongoing
support from two project leaders with professional backgrounds in early education.
Although inspired by Experience Corps in the US (Morrow-Howell et al., 2009b;
Lee et al., 2011) WoW is an innovative programme designed and delivered by Bar-
nardos in two cities in Ireland. WoW adopts a balanced literacy approach, combining
both whole language and phonics strategies, and the WoW logic model posits that
socially disadvantaged children at-risk of reading failure should benefit from one-to-
one support from older adult volunteers who are trained in reading instruction and
who also can serve as role models (Barnardos, 2008a).
At-risk readers are children having problems with literacy acquisition in first or sec-
ond grade (i.e., aged 6- to 8-years-old) and who are as a result more likely to experi-
ence academic failure, with problems experienced at this stage leading to less
frequent reading and further reading delays (Stanovich, 1986). The most efficient
and effective methods to address the difficulties of at-risk readers begin with early
prevention and intervention (Pullen et al., 2004) and as Elbaum et al.’s (2000)
*Corresponding author. Child and Family Research Centre, Nui Galway, Ireland.
Email: allyn.fives@nuigalway.ie
©2013 British Educational Research Association
Review of Education
Vol. 1, No. 3, October 2013, pp. 254–280
DOI: 10.1002/rev3.3016
meta-analysis shows, one-to-one supports provided in addition to regular classroom
teaching have been successful. In some approaches, including a modified version of
the ‘sound-linkage’ reading intervention (RI), one-to-one reading supports are pro-
vided by certified teachers. In an RCT evaluation of RI, the effect sizes in a wait list
control study design ranged from d =0.46 (phoneme completion), d =0.69 (BAS
reading), d =0.79 (early word reading), to d =0.94 (letter knowledge) (Hatcher
et al., 2006). Certified teachers also deliver ‘reading recovery’, which provides inten-
sive help for children who fail to make sufficient progress in reading and writing after
one year in school. The 30-minute daily lessons are designed by classroom teachers
according to the needs of each child. One evaluation of the programme reported
effect sizes ranging from d =0.36 to d =0.72 on measures of basic literacy (Huck &
Pinnell, 1986), although critics have questioned the practice of excluding approxi-
mately 35% of participants from evaluation studies (Pullen et al., 2004). Another
source of paid one-to-one support are paraprofessionals (or teaching assistants). In
the Howard Street Tutoring Programme, the sessions lasted for an hour, were deliv-
ered once a week, and the programme lasted for a year. In an RCT study, significant
effects were found on word recognition (d =0.72), passage reading (d =0.91) and
spelling (d =0.74) (Morris et al., 1990; Wasik, 1998). In the Start Making A Reader
Today (SMART) programme, tutors were drawn from the private sector and partici-
pated as part of their paid employment in various organisations. The effect size
reported in the evaluation of this programme was d =0.45 (Baker et al., 2000).
This paper explores whether volunteer programmes can bring about the same gains
as those delivered by paraprofessional and certified teachers, but without the costs
associated with paying tutors (Pullen et al., 2004). Reports from various RCT evalua-
tions provide mixed evidence for the effectiveness of volunteer reading programmes.
In a recent meta-analysis (Slavin et al., 2011) effect sizes ranged from a high of
d=0.89 (Meier & Invernizzi, 2001) to a low of d =0.10 (Ritter, 2000). Given the
variability in their effectiveness it is important to understand what factors contribute
to the success of an innovative volunteer reading programme. Wasik’s (1998) review
suggests that success is predicated on a number of factors:
certified specialists should be in place to supervise volunteers;
volunteer tutors should receive ongoing training and feedback;
tutoring sessions should be structured in accordance with a specific lesson plan;
tutoring should be intensive and consistent; and
students should be assessed on an ongoing basis.
Some support is given to these conclusions by a meta-analysis of reading pro-
grammes, which revealed differences in effect sizes between programmes with consis-
tent tutors (d =0.85) versus inconsistent tutors (d =0.06) and between programmes
with trained tutors (d =0.59) versus untrained tutors (d =0.17) (Elbaum et al.,
2000).
Volunteering by university students has been shown to be effective. In the volunteer
reading programme evaluated by Pullen et al. (2004), the tutors were mostly educa-
tion majors. The programme was designed to provide 40 one-to-one 15-minute ses-
sions over a 12-week period (January to April) for children in first grade scoring
below the 30th percentile on an invented spelling assessment. Participants in the
A one-to-one programme for at-risk readers 255
©2013 British Educational Research Association
study were randomly assigned to an intervention group (n =25) or a control group
(n =24), and effect sizes ranged from small for sight words (d =0.12) to medium for
letter word identification (d =0.40) and large for pseudo decoding (d =0.87), Word
Attack (d =0.87) and phonological awareness (d =1.06). Another programme deliv-
ered by college students to first grade at-risk readers is America Reads (Allor &
McCathern, 2004). The programme focused on the development of early reading
skills, including phonemic awareness and phonics. Volunteers received only three
hours of training and used scripted lessons. Participants were matched and randomly
assigned to the intervention group (n =137) or a contrast group (n =106). Pro-
gramme effects were observed for non-word reading (d =0.82), phonemesegmenta-
tion fluency (d =0.70) and three sub-tests of the Woodcock Johnson-Revised test
(WJ-R): passage comprehension (d =0.53), word attack (d =0.77) and word
identification (d =0.68).
The literature also has shown that reading programmes delivered by older adult
volunteers can be effective. The programme evaluated by Rimm-Kaufman et al.
(1999) emphasized phonics in context, reading comprehension and reading for
meaning. All tutors were aged over 60 and almost half were retired teachers. The pro-
gramme was developed by two project leaders, both professional early childhood edu-
cators. Tutors attended five training sessions prior to the commencement of tutoring
and met with coordinators twice every month for ongoing training. The children were
aged from five to seven, considered at-risk for future reading problems, and were
scheduled to be withdrawn for the sessions three times per week from October to
May. Effect sizes for the programme were observed on letter identification (d =0.17)
and the Clay reading assessment (d =0.15). Although the 45-minute sessions were
longer than the 30-minute sessions delivered by WoW volunteers, the two pro-
grammes had many similarities in respect of programme outcomes and volunteer sup-
ports. However, a drawback to the RCT evaluation of this programme was its small
sample size: 21 children in the intervention group were compared with 21 children in
the matched control group.
The evaluation of Experience Corps in the US was based on a large sample size
(control n =451, intervention n =430), and found that children in the programme
made statistically significant gains on reading comprehension (Woodcock Johnson
passage comprehension subscale) and the teachers’ assessments of reading skills, and
gains on phonemic awareness approaching statistical significance (measured by the
Woodcock Johnson word attack subscale) (p =.07). However, the effect sizes were
‘small’ according to Cohen’s (1988) criteria: d =0.10, d =0.13 and d =0.16 respec-
tively (Lee et al., 2011, p. 109). In addition, there was no one Experience Corps read-
ing programme as such. Although all Experience Corps programmes must adhere to
certain requirements, including that tutors spend two to four sessions per week with
the student, and sessions last between 30 to 40 minutes, nonetheless, three different
curricula were used: Book Buddies, Reading Coaches and the Brigance Inventory of
Basic Skills. As the evaluation findings reflect the impact of three different curricula
used, it is not possible to establish what programme components explained observed
effects, which in turn limits the usefulness of the findings as a guide to practice.
Meier and Invernizzi (2001) evaluated another implementation of Book Buddies.
The programme was delivered by community volunteers aged 55 years or over and
256 A. Fives et al.
©2013 British Educational Research Association
coordinated by paid reading specialists who each supervised the reading sessions of
approximately 15 tutors. Volunteers received formal training two to three times per
year and followed lesson plans written by the coordinators. Participants were the
lowest scoring first graders and were withdrawn from class for two 45-minute sessions
per week, which focused on reading, writing and phonics. There was an effect size of
d=1.36 for letter and word reading, and d =1.33 for number of words read cor-
rectly per minute. However, the sample size for the evaluation was small, comparing
28 children who received 40 Book Buddies sessions between September and January,
with 27 children randomly allocated to a wait-list control group.
Therefore, the literature suggests that one-to-one reading programmes delivered
by older adult volunteers can be successful. However, these studies had significant
limitations. The internal and external validity of the findings can be questioned as
either effect sizes or sample sizes were small. In the one study where a large sample
was recruited, various different reading curricula were evaluated together.
Unlike Experience Corps, WoW has its own programme model, based on a bal-
anced literacy approach. On the one hand, it emphasizes phonics, and therefore pre-
cision in reading; on the other hand, it emphasizes comprehension and appreciation,
and therefore fostering a love of reading. Phonemic awareness and phonic knowledge
are an important component of a volunteer reading programme. Not only is ‘poor
phonological awareness’ one of the main obstacles to successful reading (Hatcher
et al., 2006, p. 820; see also Adams, 1990; Torgesen et al., 1997), such difficulties
are especially common for socially disadvantaged children (Philips & Lonigan, 2005).
Phonological awareness significantly improves decoding (making the connection
between the printed letters and the sounds they represent) and in addition improves
comprehension, for when ‘words become instantly recognizable and a reader has suf-
ficient fluency skills, recognition resources can be focused on the message of the pas-
sage’ (Pullen et al., 2004, p. 23; see Foorman et al., 1997; Smith et al., 1998). Also,
in evaluations of reading supports, the greatest gains have been made in programmes
that adopted a similar approach to WoW, namely combining ‘explicit teaching in
phonological awareness with highly structured reading instruction using text’
(Hatcher et al., 2006, p. 820; see Hatcher et al., 2004; Duff et al., 2012).
A further objective of the WoW programme is to improve children’s self-percep-
tions as readers. Reading self-beliefs are important, as some studies have shown that
children’s ‘perceptions of their own reading ability are relatively accurate, since those
whose attainment was lower were more likely to say that they found reading difficult
and to seek help with their reading’ (Sainsbury & Schagen, 2004, p. 378). Positive
associations have been found when focusing on self-beliefs specific to the academic
domain and self-beliefs matched to achievement measures (see Hansford & Hattie,
1982; Valentine et al., 2004; Pullman & Allik, 2008; Logan & Johnston, 2009; Fives
et al., 2013). Other studies have identified a reciprocal causal relationship between
self-beliefs and reading achievement (Marsh & O’Mara, 2008). The review by Brooks
(2002) found that self-esteem counselling by trained non-professionals, when com-
bined with a reading intervention, was ‘very effective’ in raising reading attainment
(p. 13). The evaluation of WoW investigates whether improved self-belief can be a
measured gain resulting from participation in a reading programme where self-esteem
counseling was not part of the package of measures.
A one-to-one programme for at-risk readers 257
©2013 British Educational Research Association
Although the literature suggests that reading programmes delivered by older adult
volunteers can be effective when compared with regular classroom teaching on its
own, some did not benefit all those targeted. For instance, in two studies, boys
(Rimm-Kaufman et al., 1999) and socially disadvantaged children with low reading
scores (Hatcher et al., 2006) did not experience the same gains as others. This has
important implications for programme design, as a reading programme must be tar-
geted at the appropriate population, and its difficulty level should be ‘carefully
matched to that of the learner’ (Hatcher et al., 2006, p. 820; see Ehri et al., 2001;
Savage et al., 2003; Hatcher et al., 2004). This evaluation will address which sub-
groups of the target sample benefited most from WoW. In particular, it will focus on
children’s initial reading ability levels, gender and the receipt of any additional
supports.
A further issue is the amount of time spent in the programme (programme intensity)
that is needed to bring about gains in children’s reading. For example, although Book
Buddies led to significant and large gains, the greatest gains were made by those receiv-
ing the recommended number of sessions (Meier & Invernizzi, 2001), and the effects
of Experience Corps were greater when comparing the control group with those in the
intervention group who received the recommended number of sessions (Lee et al.,
2011). This paper will examine whether programme intensity predicts outcomes for
children and also what level of programme intensity brings about the greatest gains.
The literature shows there is considerable interest in the effectiveness of reading
programmes delivered by older adult volunteers and in particular whether they can be
of benefit for at-risk readers. However, given the limitations of other evaluations of
similar programmes, important research questions remain unanswered. This study
will address whether a unique reading programme was effective, it will examine
whether WoW was equally beneficial for all those targeted and also the variables that
predicted response to intervention, and it will use process study data on programme
implementation to provide plausible reasons for observed effects.
Wizards of Words (WOW)
WoW is a reading programme delivered by trained volunteers aged 55 and over for
children having difficulty acquiring reading skills. Although delivered on a one-to-one
basis, because of restrictions on volunteer availability children may read with more
than one volunteer over the course of the school year. All involvement with the chil-
dren takes place within the school premises and during school hours. Two manuals
outlining the purpose, structure and format of WoW were developed by Barnardos
for the WoW project leaders and the volunteers respectively (Barnardos, 2008b).
WoW has the following aims:
to make improvements in the children’s reading comprehension, reading fluency,
vocabulary building and phonemic awareness;
to encourage and promote their interest in and love of reading; and
to improve their perceived competence and enjoyment of reading.
WoW is premised on the benefits of volunteers aged 55 and over reading on a one-
to-one basis with children experiencing delays in reading. Volunteers, because of their
258 A. Fives et al.
©2013 British Educational Research Association
life and work experience and because they belong to the same community, may be
able to act as role models for these children, leading to improvements in reading skills
and increased aspirations for the future.
WoW adopts a ‘balanced literacy’ approach. This combines the most effective
strategies from the ‘whole language’ and ‘phonics’ approaches. The whole language
approach places emphasis on comprehension, reading connected text in an uncon-
trived fashion, and encouraging an appreciation of reading. In contrast, in a phonics
approach, the emphasis is on accuracy and precision in reading, on learning the rules
for writing and spelling words, and in particular on phonics and phonological aware-
ness (Barnardos, 2008b). This is in line with the Irish government’s Department of
Education and Science (DES) current reading initiatives and the English Curriculum
expectations. The following four areas of reading, as identified in the Primary School
English Curriculum (PSEC) introduced in 1999, are targeted:
(1) Phonics: phonemegrapheme identification, the identification of initial, medial
and final phonemes, and phoneme blending.
(2) Vocabulary building: getting the child to see new words and understand them; also,
using prefixes to change word meanings (e.g., what happens when you add /dis/
to a worddis/able or dis/like).
(3) Reading comprehension: helping the child to understand what he/she is reading,
word-by-word, sentence-by-sentence, page-by-page and book-by-book.
(4) Reading fluency: reading quickly and accurately with expression and understand-
ing.
WoW is implemented through a three stage ‘guided reading’ approach (Barnardos,
2008b):
(1) The ‘pre-reading’ stage is designed to cue children to new words in the book, and
the conversation element is designed to help enrich vocabulary and encourage
children to express themselves in whole sentences.
(2) The ‘reading’ stage includes various different methods of reading with the child: the
childlisteningto thevolunteer read,thechildandvolunteer taking turns to readeither
flexibly or as planned, the child rereading what the volunteer has read (‘echoed’), the
childand volunteer reading together (‘choral’),and the child reading independently.
Theobjective is ultimately forthe child to readindependently.
(3) ‘Follow-up activities’ are intended to reinforce one or more of the key reading
areas. They include ‘prediction’ (e.g., asking ‘what do you think will happen
next?’), ‘questioning’ (e.g., asking ‘did the actions of the character make sense?’),
‘summarising’ (e.g., asking the child to summarise the beginning, middle and end
of the story), ‘clarifying’ (making connections between the child’s background
and the story) and ‘visualizing/imagining’ (asking the child to discuss or draw
what they would feel if this had happened to them).
The Biff,Chip and Kipper stories from the Oxford Reading Tree (ORT; http://
www.oup.com/oxed/primary/literacy/ort/) were used. The stories at levels one to four
help build the young reader’s foundation reading skills. They introduce different
characters and illustrations are matched with simple texts, core high frequency/sight
word vocabulary is repeatedly used, and the reader becomes familiar with phonically
A one-to-one programme for at-risk readers 259
©2013 British Educational Research Association
regular words. Higher level stories focus on growth, consolidation and reinforcement
of reading skills, specifically expanding vocabulary, introducing longer sentences and
longer stories that are more complex. The ORT ‘Assess and Progress’ toolkit is used
throughout the year to ensure that children are moved up (or down) the reading levels
when appropriate and that they are on the level that will provide them with the right
amount of challenge. In addition, phonics instruction is based on Floppy Phonics,
also provided by ORT. The tasks focus on phonemegrapheme identification (single
letter, more than one letter, double vowels and different spellings of double vowel
sounds), the identification of initial, medial and final phonemes, and phoneme blend-
ing. The tasks also include looking at rhyme sounds. Instruction is based around a
story, with pre-reading, reading and follow-up activities.
WoW is coordinated by two project leaders who have a professional background in
education. The eight schools delivering WoW are divided into two clusters, in Dublin
and Limerick, so that they can be managed efficiently. Project leaders read with the
children on a regular basis, they observe volunteer reading sessions, monitor the chil-
dren’s reading records, provide feedback to volunteers, record and report on each
child’s progress to school staff, and liaise with schools to ensure a good working
relationship.
Volunteers receive a two-day comprehensive induction training, including informa-
tion on Barnardos policies and procedures, and WoW-specific training such as read-
ing and instructional strategies. In addition, a wide range of supports are provided by
project leaders to volunteers: ongoing availability to discuss children’s progress;
regular review and support sessions including reflection on working style and teaching
strategies; one-to-one coaching sessions on instructional strategies as required;
ongoing training targeted to meet existing needs; group meetings between all of the
volunteers to share experiences and learning; a monthly newsletter sharing
programme updates and instructional strategies; and social occasions (Christmas and
summer luncheons).
Method
Design
In the evaluation of WoW, an outcomes study with an experimental design was
combined with a process study based primarily on qualitative data from one-to-one
interviews, focus groups, observations and documentary analysis with some quantita-
tive data from surveys and implementation records. The design of the outcomes study
was a randomized controlled trial (RCT). A randomized experiment is one ‘in which
the units are assigned to receive the treatment or an alternative condition by a random
process such as toss of a coin or a table of random numbers’ (Shadish et al., 2002, p.
12). This study was a ‘pre-testpost-test control group design’ (p. 261). There was a
pre-test assessment (T0), followed by the random allocation of children to control
and intervention groups. Children were assessed again, at the end of the school year
(T1; 8 months) and once again during the next academic year (T2; 12 or
16 months). See the appendix for the inclusion of T2 data from two different time
points.
260 A. Fives et al.
©2013 British Educational Research Association
Outcomes and research questions
The outcomes of this study were reading ability, including the domains of reading
comprehension, reading accuracy, word recognition, vocabulary and phonemic
awareness and phonic knowledge. The outcomes also included children’s reading
self-beliefs, namely their attitude to reading and perceived competence in reading and
schoolwork. The research questions for this study were as follows:
(1) Was receipt of WoW effective in creating improvements in children’s reading
ability and reading self-beliefs?
(2) Did some variables moderate the effects of the programme?
(3) Did some variables predict participants’ response to the programme?
(4) What was the relationship between programme implementation and positive out-
comes for children?
Sample
The sample was recruited from among children in first grade and second grade attend-
ing the nine schools across two cities where the WoW programme was on offer. Previ-
ous participants in the programme were not eligible for recruitment to the study, which
ruled out some second grade students who had participated when in first grade. Partici-
pants were recruited in two academic years or cohorts (20092010 and 20102011).
Teachers nominated children who they thought would benefit from WoW. Written
informed consent was sought from all parents and children. All children who had
agreed to take part were then screened and children were included in the study only if
they met the following inclusion criteria:
(1) Children’s reading level should be between the following thresholds: (a) for first
class children the lower threshold was 18 months behind the age-appropriate read-
ing level and the upper threshold was four months behind; (b) for second class chil-
dren the lower threshold was 24 months behind and the upper threshold was four
months behind, as measured on WIAT Single Word Reading (WIAT-II
UK
-T).
(2) Children must not need specialist support, that is, they: (a) have not been diag-
nosed with general or specific learning disabilities or behavioural difficulties; (b)
are not in the reading recovery programme or receiving supplementary teaching
in English with a learning support teacher.
(3) Children did not have planned/foreseeable extended absences from school.
Demographic baseline characteristics of participants are presented in Table 1.
There were no significant differences between study conditions on these variables. As
previous WoW participants were not eligible for recruitment, fewer second grade chil-
dren were available for participation in the study.
Random allocation
The CONSORT statement for reporting RCT studies recommends both describing
the method used to generate the random allocation sequence and the steps taken to
ensure the implementation of the allocation sequence was concealed (Altman et al.,
A one-to-one programme for at-risk readers 261
©2013 British Educational Research Association
2001). In this study, ‘stratified randomisation’ was used. This was achieved by
performing a separate randomisation procedure within each group of participants.
The groups were created based on the participants’ cohort (cohort 1 or cohort 2),
school (one of eight in each cohort) and class year (first or second grade within each
of the eight schools). Allocation was structured so as to ensure a close balance in
number of children assigned to each study condition within each stratum (see Altman
et al., 2001). Participants within each stratum who had been screened into the study
were placed in ascending order based on their ID number. After a ‘random start’
(based on the flip of a coin) every second child was assigned to the intervention condi-
tion. In order to conceal the implementation of the allocation sequence, only one
member of the research team assigned ID numbers to participants and then carried
out the random allocation. The implementation of the allocation sequence was con-
cealed from the programme providers and participants’ study condition was ‘blind’ to
the data collectors. The participants themselves did not know their study condition at
T0, but became aware shortly afterwards when the programme began.
Participant flow
The flow of participants is presented in Figure 1. Between T0 and T2 the study sam-
ple was reduced from 229 children to 212 children. The loss of these 17 children rep-
resents an attrition rate of 7%.
Measures
Single word reading (SWR) and spelling (Spell)
The Wechsler Individual Achievement Test Second Edition for Teachers
(WIAT-II
UK
-T) was used to measure single word reading and spelling (Wechsler,
2002; Harcourt Assessment, 2006). The single word reading sub-test assesses early
Table 1. T0 (baseline) demographics
Control
N=111
Intervention
N=118
Total
N=229
Chi-square Test
Phi value p
Cohort 1 (2009/10) 55 (24%) 58 (25%) 113 (49%) .004 .952
2 (2010/11) 56 (25%) 60 (26%) 116 (51%)
City Dublin 61 (26%) 66 (29%) 127 (55%) .010 .882
Limerick 50 (22%) 52 (23%) 102 (45%)
Gender Boys 56 (25%) 60 (26%) 116 (51%) .004 .952
Girls 55 (24%) 58 (25%) 113 (49%)
School Year 1st grade 64 (28%) 67 (29%) 131 (57%) .009 .893
2nd grade 47 (21%) 51 (22%) 98 (43%)
Age 5 years 0 (0%) 1 (.5%) 1 (.5%)
6 years 56 (24.5%) 54 (23.5) 110 (48%)
7 years 50 (22%) 53 (23%) 103 (45%)
8 years 5 (2%) 10 (4.5%) 15 (6.5%)
Notes: Figures represent numbers of participants followed by percentages of overall sample in parentheses.
Chi-square tests were not run on ‘age’ data due to high number of cells with less than five units.
262 A. Fives et al.
©2013 British Educational Research Association
reading (phonological awareness) and word recognition and decoding skills. Words
are read aloud from a word list and word reading accuracy is scored. The spelling
sub-test assesses the ability to spell dictated letters, letter blends and words. In both
sub-tests raw scores are converted to standard scores, percentiles and reading age
scores. The single word reading sub-test takes approximately 1520 minutes to
administer and the spelling sub-test takes approximately 510 minutes. The test was
standardized on a sample of 892 children between the ages of 4:016:11, derived
from a validation sample representative of the UK population of children based on
Children screened (n = 369)
Pretest (T0)
Excluded:
Did not meet inclusion criteria (n = 76)
Previous parƟcipants in programme (n = 47)
Receiving addiƟonal support services (n = 16)
Refused to parƟcipate (n = 1)
Children randomly allocated (n = 229)
Allocated to control group (n = 112)
Received allocated treatment (n = 111)
Did not receive allocated treatment (n =
1: previous parƟcipant in programme)
Allocated to intervenƟon group (n = 118)
Received allocated treatment (n = 118)
PosƩest (T1)
at 8 months, n = 111
PosƩest (T1)
at 8 months, n = 116
Follow-up (T2)
at 12 months, n = 60
at 16 months, n = 45
Follow-up (T2)
at 16 months, n = 107
Sample in analysis (n = 105)
Lost to post-test (n = 0)
Lost to follow-up (n = 6)
Missing on days of data collecƟon (n = 3)
Previous parƟcipant in programme (n = 1)
LeŌschool (n = 1)
Entered programme (n = 1)
Sample in analysis (n = 107)
Lost to post-test (n = 2)
Lost to follow-up (n = 11)
LeŌstudy (n = 2)
Entered programme before follow-up (n = 8)
Missing on days of data collecƟon (n = 1)
Figure 1. Participant Flow
A one-to-one programme for at-risk readers 263
©2013 British Educational Research Association
the 2001 census. Reliability was estimated using the split-half method, with equiva-
lent halves of each sub-test selected, representing parallel forms with approximately
equal variances. Average reliability coefficients ranged from 0.940.98.
Reading accuracy (RA) and Reading comprehension (RC)
The York Assessment of Reading for Comprehension Passage Reading Test (Snow-
ling et al., 2009) was used to measure reading accuracy (decoding) and reading com-
prehension (text comprehension: literal and inferential meaning). The test is
designed to be used with pupils aged 5 to 11 years, and takes 1015 minutes to
administer. The graded passage for reading aloud is accompanied by a set of eight
comprehension questions. Reading accuracy raw scores are calculated from the total
number of reading errors made, and reading comprehension raw scores are calculated
from the total number of comprehension scores answered correctly. Raw scores are
converted to standard scores, percentiles and reading age scores. The measure was
standardized in 2008 on 1376 students from across the UK. Cronbach’s alpha scores
for reading comprehension reliability ranged from 0.48 (level 3A) to 0.77 (beginner
level). The reliability of reading accuracy was assessed by computing the correlation
between parallel forms of the test (forms A and B), and scores ranged from 0.75
(beginner level) to 0.92 (level 5).
Vocabulary (BPVS)
The British Picture Vocabulary Scale (Dunn et al., 1997) was used to assess the chil-
dren’s receptive (hearing) vocabulary. The standard procedures for using the test
with children under eight years of age include the following instructions: The exam-
iner says: ‘I want you to look at some pictures with me. See all the pictures on this
page. I will say something then I want you to put your finger on the picture of what I
have said. Let’s try one. Put your finger on “ball”’. The test takes only between five to
eight minutes, as it is administered over the critical range of items for a particular sub-
ject, usually four or five sets of 12 items. The raw score is calculated by subtracting all
errors from the ceiling item, and raw scores are converted to standard scores, percen-
tiles and reading age scores. The test was standardized in 1995 on 2571 pupils in 152
schools in the UK. The reliability of the test, calculated using Cronbach’s alpha, ran-
ged from 0.93 to 0.96 for the age ranges included in this study.
Phonemic awareness (PA) and phonic knowledge (PK)
The research team also employed the criterion-referenced tests of phonemic aware-
ness and phonic knowledge developed by Professor Morag Stuart, University of
London, for the purposes of this study. The reliability of the test was calculated using
Cronbach’s alpha, and the test of the five phonemic awareness sub-test items and the
phonic knowledge test returned a score of 0.77.
Phonemic awareness. The maximum score on the phonemic awareness test was 48
and the test was composed of five sub-tests.
264 A. Fives et al.
©2013 British Educational Research Association
(1) Initial phoneme identification (eight test items): Pupils were shown pictures which
were named for them. They were then asked to give the first sound of the name.
Only pupils who attempted all items continued on to the final phoneme identifi-
cation and the phoneme segmentation tasks.
(2) Final phoneme identification (six test items): Pupils were again shown pictures
which were named for them. They were then asked to give the last sound of the
name. The task was discontinued after three consecutive failures.
(3) Phoneme blending real words (12 test items): Pupils were required to blend syllables
or phonemes into a word. The task was discontinued after three consecutive fail-
ures. Only pupils who attempted all items continued on to the phoneme blending
non-words task.
(4) Phoneme segmentation (12 test items): Pupils were required to segment monosyl-
labic words consisting of from two to five phonemes into their constituent pho-
nemes.
(5) Phoneme blending non-words (10 test items): Pupils were required to blend pho-
nemes into a non-word pronunciation, ranging from two to five phonemes in
length.
Phonic knowledge. The maximum score on this test was 40. The pupils were shown a
card, with 40 graphemes in total, and they were shown one line at a time with five
graphemes. The tester pointed to each grapheme in turn and asked the pupil to give
the sound of that grapheme. One point was given when the children provided the
correct sound for each letter and stopped testing when the child was clear that they
don’t know any more.
Attitude to reading and perceived competence (AC)
The researchers’ own measure of academic self-belief was used: the attitude to read-
ing and self-competence measure. Children were asked two questions about attitudes
to reading: ‘attitude to reading at home’ and ‘attitude to reading in class’. They were
asked one question about general academic self-competence: ‘schoolwork self-com-
petence’; and one question about self-competence related specifically to classroom
reading: ‘classroom reading self-competence.’ The five-point scale for attitudes ran
from ‘really like it’ to ‘really don’t like it’, and the five-point scale for self-competence
ran from ‘very good’ to ‘really not good’. Reliability analysis returned a Cronbach’s
alpha value of 0.6. Asking children to self-report in regard to their attitudes and per-
ceived competence raises a concern about reporter bias. However, results from data
collected using this measure have shown statistically significant positive associations
between attitude to reading and reading achievement (Fives et al., in press).
Procedure
In the outcomes study, tests were administered by qualified speech and language
therapists. Data were collected at each of the three data collection time points
for each of the two cohorts, and at each time point each child took part in three
A one-to-one programme for at-risk readers 265
©2013 British Educational Research Association
face-to-face sessions lasting between 20 and 30 minutes each. The data collection
procedure was as follows:
Session 1: (approximately 1520 minutes)
WIAT single word reading (SWR)
Session 2: (approximately 30 minutes)
Attitude to reading and perceived competence (AC)
York assessment of reading (RA and RC)
Session 3: (approximately 30 minutes)
Phonemic awareness (PA) and phonic knowledge (PK)
British Picture Vocabulary Scale (BPVS)
WIAT spelling (Spell)
Other data collected by the research team
Volunteers’ survey
Volunteers also completed surveys recording their views on children participating in
the WoW programme. Answers to questions about the children’s relationship with
the volunteer and the children’s experience of the programme were recorded on a
five-point Likert scale, running from ‘very positive’ to ‘very negative’. The surveys
were administered for each of the two cohorts of children in the programme and data
were collected on 111 of the 118 intervention group children.
Process study data collection
As part of the overall evaluation of WoW, programme implementation was evaluated
over a three-year period. Children, parents, school staff and Barnardos staff partici-
pated in face-to-face or telephone interviews. WoW volunteers participated in focus
groups and data were collected on the recruitment process, training and ongoing sup-
ports, the WoW manual, the reading session, their relationship with the children and
their experience of the programme overall. The WoW children’s reading records were
reviewed in detail for both cohorts of participants. Data on the number of sessions
attended, the reading areas covered in a session and the progress made (on the ORT)
were documented for each child by the volunteers. This information was used to
inform the findings on the fidelity to the WoW programme model.
Programme intensity
Data were collected from WoW project leaders on the average weekly number of ses-
sions received by each child over the full academic year, on 112 of the 118 interven-
tion group children.
School absence
Data were collected at the end of the academic year from school staff, on all 229 study
children.
266 A. Fives et al.
©2013 British Educational Research Association
Participation in reading recovery or receiving supplementary teaching in English with a
learning support teacher
This information was gathered from teachers who were surveyed as part of the evalua-
tion. Data were provided on 213 of the 229 study children.
Gender
This information was recorded on parental consent forms for all participants.
Age
These data were collected on all participants from parents on the parental consent
forms, from children during assessments and checked with school records at the end
of the year.
Data were collected from all participants.
Results
Treatment fidelity
Participants in the study were randomly assigned to either the intervention group or
the control group. Individuals within the same classrooms were allocated to the differ-
ent study conditions and allocation was ‘blocked’ to ensure an equal number in each
study condition in each classroom, numbers permitting. Children in both groups
received regular classroom teaching. When children in the intervention group were
withdrawn from the classroom to receive the one-to-one WoW sessions, their class-
mates in the control group remained in the classroom where various reading related
activities were conducted including both the teacher reading to the class and also
‘library time’ where each child was allowed to choose what to read from among the
books in the classroom.
Analysis of WoW programme fidelity was based on observations of the reading ses-
sions, documentary analysis, focus groups with volunteers and interviews with project
leaders and Barnardos staff. Observations suggested that reading sessions were highly
structured, adhering to the model of pre-reading, reading and follow-up activities.
Volunteers engaged with the children in a positive and supportive manner, providing
encouragement and praise for the children’s efforts.
According to the study protocol, children in WoW were to receive three one-to-one
sessions per week for the full academic year. Records completed by volunteers and
supervised by project leaders were reviewed as part of the evaluation. Although the
programme was delivered across the academic year, programme intensity varied. The
weekly average number of WoW sessions ranged from a low of 0.33 per week to a high
of 2.63 per week, and a mean of 1.81. The analysis of programme effects will examine
whether programme intensity predicted participants’ response to the intervention and
what level of programme intensity lead to the greatest gains. Process study data
suggest lower programme intensity may have resulted from school closure, extra-
A one-to-one programme for at-risk readers 267
©2013 British Educational Research Association
curricular activities (e.g., outings, sports days) or volunteer absenteeism. Industrial
action and severe weather led to school closures for the first cohort, where
programme intensity was lower. Lower programme intensity was also a function of
school willingness, as some did not want to release the children more than twice a
week.
Information was provided by class teachers three months in to the academic
year (November) on 213 of the 229 participating children about additional
support services. The data show 39 (18%) children were in the reading recovery
programme or receiving supplementary teaching in English with a learning support
teacher. The children were withdrawn from the class in all instances, and in all
bar three the supports were received more often than once per week. Although
control group children were more likely to be nominated for these supports the
difference between those receiving these supports in the control group (58%) and
the intervention group (42%) was not statistically significant, phi =.10, p =.14.
There was a statistically significant difference among girls who were nominated to
receive these additional supports, between those in the control group (68%) and
the intervention group (32%), phi =.19, p =.05. There was also a statistically sig-
nificant difference among children in the second cohort nominated to receive these
supports, between those in the control group (71%) and those in the intervention
group (29%). The analysis of programme effects will examine whether receipt of
additional supports predicted participants’ response to the intervention or moder-
ated programme effects.
The phonics component of programme delivery and volunteer training was
amended half way through the academic year of the first cohort of participants.
A decision was made to introduce systematic phonics as part of the programme and
dedicate one of the three weekly sessions to phonics. This innovation was introduced
because the Barnardos team recognised that phonics was a key building block for fur-
ther reading. They also believed that extra training in this area would bring benefits to
volunteers in terms of their confidence to deliver phonics sessions. Although changes
were initiated half way through the academic year of the first cohort, the full benefit
of these changes were experienced by the second cohort. The remaining two weekly
sessions were to cover the areas of comprehension, fluency and vocabulary building.
However, as we have seen, programme intensity was lower than required, as children
did not receive three sessions per week on average, and as priority was given to the
phonics session the other programme components will have been under represented.
To determine whether the increased priority given to phonics in both tutor training
and programme delivery in cohort 2 affected outcomes, the analysis will examine
whether cohort moderated programme effects. The analysis will also examine
whether the programme was more effective at improving some phonics skills than
others (i.e., phoneme identification, phoneme segmentation, phoneme blending or
phonic knowledge).
Pre-test data
Children’s reading scores at T0 are presented in Table 2. Children in the control
group recorded higher mean scores on six of the eight measures. None of the differ-
268 A. Fives et al.
©2013 British Educational Research Association
ences were statistically significant, although the difference between groups for single
word reading (SWR) was approaching statistical significance. This indicates an
imbalance between the two study conditions prior to the receipt of their respective
treatments. However, selection bias was avoided as the allocation of participants to
study condition was random and staff were blind as to treatment condition (see
Altman, 1991; Fives et al., 2013).
Programme effects
To analyse the effects of the programme, in the first instance a repeated measures ver-
sion of a multivariate analysis of covariance (MANCOVA) was conducted for each of
the reading measures. Scores at T1 and at T2 were employed as dependent variables,
controlling for T0 scores on the dependent variable. This was followed by an analysis
of covariance (ANCOVA), where the differences at T1 between the two groups were
analysed while controlling for the effects of additional variables: T0 scores on the
dependent variable, city, gender, cohort, school and class year. The ANCOVA analy-
sis was repeated with scores at T2 as the dependent variable.
On the measures of single word reading (SWR) and phonemic awareness (PA) the
changes made by the intervention children were significantly more positive than the
changes made by the control children. There was a statistically significant difference
on the measures of single word reading (SWR), F (2, 206) =3.79, p =.02, d =0.38,
and phonemic awareness (PA), F (2, 203) =3.53, p =.03, d =0.37. Further univari-
ate analysis of scores for phonemic awareness (PA) showed that at T2 there was a sta-
tistically significant difference between the control group and the intervention group,
and the effect size was between small and medium, F (1, 202) =6.36, p =.01,
d=0.36, and there was no significant difference at T1, F (1, 215) =0.23, p =.63,
d=0.06. Scores for single word reading (SWR) showed a small difference between
the two study conditions at T2 that was not statistically significant, F (1, 204) =1.69,
Table 2. Time 0 (baseline) means and standard deviations on outcome measures
Measure Control Intervention
Independent-
samples t-test
n M SD n M SD Cohen’s d Sig.
SWR 111 81.05 9.56 118 79.14 7.93 0.22 .10
RA 100 89.41 8.47 112 88.05 9.12 0.15 .27
RC 100 98.35 8.61 108 97.09 8.87 0.15 .30
Spell 100 82.20 9.17 116 82.16 8.53 0.01 .98
BPVS 111 95.51 10.35 118 94.60 9.77 0.09 .49
PA 110 32.53 9.16 118 31.16 8.57 0.16 .25
PK 110 27.24 4.82 117 27.26 4.96 0.00 .98
AC 99 16.76 2.48 106 16.77 2.74 0.00 .97
Notes: SWR =WIAT single word reading, RA =York reading accuracy, RC =York reading comprehension,
Spell =WIAT spelling, BPVS =British Picture Vocabulary Scale, PA =phonemic awareness, PK =phonic
knowledge, AC =attitude to reading and perceived competence. Unadjusted mean scores are calculated from
standard scores (SWR, RA, RC, Spell, BPVS) and total correct scores (PA [total =48], PK [total =40], AC
[total =20]).
A one-to-one programme for at-risk readers 269
©2013 British Educational Research Association
p=.20, d =0.18, and no significant difference when analysing data from T1 only,
F (1, 218) =.45, p =.50, d =0.09. Intervention children also made greater gains at
T1 on scores for phonic knowledge (PK) with the difference approaching statistical
significance, F (1, 215) =2.91, p =.09, d =0.23. The data also show that at T1
intervention group children made greater gains on the item assessing attitude to read-
ing in class (phi =.16, p =.07), and also on the overall measure of self-belief (AC),
F (1, 195) =2.72, p =.10, d =0.24.
A summary of the analysis of programme effects is provided in Table 3. For each of
the outcomes measured, the mean scores at T1 and T2 (adjusted for covariates) of
both the control group and the intervention group are given. This is followed by
results from analysis of programme effects at T1 and at T2 separately (ANCOVA),
and then an analysis of programme effects at T1 and T2 together (MANCOVA).
Scores on the five sub-tests of the phonemic awareness (PA) measure were also
analysed (see Table 4). The largest programme effects were observed on scores for
phoneme blending non-words at T2 (d =0.40), initial phoneme identification at T2
(d =0.35), and the combined measures of phoneme blending (real words and non-
words) at T2 (d =0.29).
Table 3. Summary of analysis of effects of the WoW programme
Adjusted means Adjusted means ANCOVA results
MANCOVA
results
Outcome Time Control Intervention
(Mean difference)
Effect size Cohen’s d
Effect size
Cohen’s d
SWR Time 1 86.35 85.66 (0.64) 0.09 0.38**
Time 2 85.91 87.54 (1.44) 0.18
RA Time 1 93.81 93.49 (0.31) 0.06 0.20
Time 2 92.16 93.17 (1.02) 0.16
RC Time 1 100.76 100.61 (0.05) 0.00 0.11
Time 2 101.91 101.40 (0.52) 0.09
SPELL Time 1 88.02 86.75 (1.27) 0.20 0.17
Time 2 84.98 85.09 (0.11) 0.00
BPVS Time 1 94.81 93.73 (1.09) 0.17 0.14
Time 2 94.79 93.89 (0.91) 0.13
PA Time 1 36.32 35.89 (0.43) 0.06 0.37**
Time 2 37.45 39.34 (1.89) 0.36**
PK Time 1 30.87 31.70 (0.63) 0.23* 0.26
Time 2 31.63 31.60 (0.03) 0.00
AC Time 1 16.76 17.26 (0.61) 0.24* 0.33
Time 2 17.01 17.48 (.34) 0.21
Notes: SWR =WIAT single word reading, RA =York reading accuracy, RC =York reading comprehension,
Spell =WIAT spelling, BPVS =British Picture Vocabulary Scale, PA =phonemic awareness, PK =phonic
knowledge, AC =attitude to reading and perceived competence. Covariates in MANCOVA are Time 0 scores.
Covariates in ANCOVA are Time 0 scores, school, cohort, class year, gender, city.
*Significant at the p <.10 level.
**Significant at the p <.05 level.
***Significant at the p <.001 level.
270 A. Fives et al.
©2013 British Educational Research Association
Moderated the effects of the programme
In testing for moderators, aggregate measures were used to reduce the risk of Type I
errors, i.e. the risk of observing significant findings due to the large number of com-
parisons. Principal components analysis revealed the presence of three components
with eigenvalues exceeding 0.7, explaining 47%, 16% and 11% of variance
respectively. Factor scores for these three latent variables were used in a simple effects
analysis (general linear model) of moderator variables.
A number of variables were found to moderate the effects of the programme. First,
the child’s reading ability level at T0 moderated the effects of the programme at T2
on factor 2 (York RC BPVS), F (1, 191) =5.154, p =.024. ‘Below average’ reading
skills were defined as T0 scores that were more than one standard deviation below the
population mean on the screening tool WIAT single word reading (standard score
<85; percentile rank <16) with the remainder of the children (standard score 85;
percentile rank 16) categorized as ‘average.’ While in the control group the greatest
gains were made by children in the average reading level, in the intervention group
the greatest gains were made by the children in the below average reading level.
Gender also moderated the effects of the programme, as the interaction between
gender and study condition had a significant effect on children’s gain scores between
T0 and T2 for factor 1 (SWR RA Spell), F (1, 192) =8.291, p =.004. Boys
gained more than girls in the intervention group and the difference was statistically
significant (p =.000) and there was no significant difference between boys and girls
in the control group.
The receipt of additional supports (reading recovery or supplementary teaching
with a learning support teacher) moderated the effects of the programme on factor 2
Table 4. Effects of the WoW programme on phonemic awareness sub-tests
Adjusted means
Adjusted
means ANCOVA results
Outcome Time Control Intervention
(Mean difference)
Effect size Cohen’s d
Initial phoneme identification Time 1 7.61 7.48 (0.12) 0.11
Time 2 7.66 7.89 (0.23) 0.35**
Final phoneme identification Time 1 5.23 5.09 (0.14) 0.11
Time 2 5.46 5.59 (0.13) 0.20
Phoneme segmentation Time 1 7.13 7.36 (0.22) 0.06
Time 2 7.75 8.27 (0.52) 0.20
Phoneme blending real words Time 1 9.80 9.60 (0.21) 0.20
Time 2 10.12 10.35 (0.23) 0.20
Phoneme blending non-words Time 1 6.51 6.14 (0.37) 0.20
Time 2 6.52 7.35 (0.83) 0.40**
Phoneme blending (combined) Time 1 16.14 15.79 (0.35) 0.09
Time 2 16.13 17.25 (1.13) 0.29*
Notes: Covariates in ANCOVA are Time 0 scores, school, cohort, class year, gender, city.
*Significant at the p <.10 level.
**Significant at the p <.05 level.
***Significant at the p <.001 level.
A one-to-one programme for at-risk readers 271
©2013 British Educational Research Association
(York RC BPVS), F (1, 181) =3.255, p =.075. Children in the WoW programme
who received additional supports performed significantly better than children in the
WoW programme who did not receive these additional supports (p =.053) and there
was no significant difference in the control group.
Whether or not the child was part of the first or second cohort of participants also
moderated the effects of the programme on factor 3 (PA PK), F (1, 207) =4.047,
p=.075. While in the intervention group the second cohort of participants made the
same gains between T0 and T2 as were made by the first cohort, in the control group
the gains made by the second cohort were significantly smaller than the gains made
by the first cohort (p =.003).
Finally, class year modified the impact of the programme on factor 1 (SWR RA
Spell), F (1, 181) =2.826, p =.095 and on factor 3 (PA PK) F (1, 207) =1.786,
p=.183. Children in first grade made significantly greater gains from the programme
than children in second grade on factor 1 (p =.053) and factor 3 (p =.002). This is
of particular importance as time 2 data were collected at two different time points.
For first class control children, time 2 data were collected in September (12 months
after time 0); for the remainder of the students (first class intervention, second class
control, second class intervention), time 2 data were collected the following January
(16 months after time 0). An important issue is whether the decision to collect fol-
low-up data on first class control children at 12 months rather than 16 months led to
an underestimation of the gains made by the first class control group and, as a conse-
quence, whether the observed programme impact is explained by the time at which
data were collected. Further analyses indicated that the impact of class year on pho-
nemic awareness, where the greatest programme impact was observed, was explained
in part by cohort (see the appendix). The programme was effective for second class
children in cohort 2 when data were collected from the control group and the inter-
vention group at the same time, and so the reported impact of the programme cannot
be explained by the collection of data from first class control group children at an ear-
lier time to all other participants.
Predicted response to the intervention
To examine whether some variables predicted programme success, multiple regres-
sion analysis was conducted on data from children in the intervention group (see
Table 5). The child’s ‘experience of the programme’ (as reported by the volunteer)
was a statistically significant predictor of scores for single word reading (SWR)
(b=.54, p =.01) and reading accuracy (RA) (b=.41, p =.06). Initial results sug-
gested a negative association between the child’s ‘relationship with the volunteer’ (as
reported by the volunteer) and scores for single word reading (SWR) (b=.39,
p=.05). However, correlation analysis showed the variables ‘relationship’ and ‘expe-
rience’ were strongly associated (r =.78, n =111, p =.00), and when ‘relationship’
was analysed separately from ‘experience’ there was no longer a negative association
between ‘relationship’ and scores for single word reading (SWR) (b=.02, p =.85).
There was no overall programme effect on scores for reading comprehension (RC),
but the results suggest that WoW led to gains when school attendance was good
(b=.36, p =.00). There was also a positive association between programme
272 A. Fives et al.
©2013 British Educational Research Association
intensity and scores for reading comprehension (RC) although the findings were not
statistically significant (b=.11, p =.19). Similarly, the results suggest that WoW
led to gains for reading accuracy (RA) when school attendance was good (b=.15,
p=.10) and programme intensity was high (b=.14, p =.11). The intervention
children with above median programme intensity made greater gains on scores for
reading accuracy (RA) than the intervention children with below median programme
intensity (mean difference =3.49, p =.11). A different picture emerges when analy-
sing outcomes where there was an overall programme effect. WoW lead to gains in
single word reading (SWR) regardless of the children’s school attendance record,
whereas the scores of children in the control group were positively associated with
school attendance (mean difference =5.57, p =.00). In addition, in the intervention
group there was no difference on scores for phonemic awareness (PA) between those
who received above median and below median programme intensity (mean difference
=.02, p =1.00).
As programme intensity varied, a further analysis examined what level of pro-
gramme intensity brought about the greatest gains (see Table 6). Children in the
Table 5. Results of regression analysis for predictors of response to intervention
SWR RA RC Spell BPVS PA PK AC
Relationship .39** .25* .01 .17 .08 .21 .22 .03
Experience .54*** .41** .10 .21 .13 .08 .23 .20
Additional supports .08 .03 .02 .19** .08 -.03 .05 .01
Programme
intensity
.11 .14 .11 .14 .00 .07 .02 .01
School absence .08 .15* .36*** .12 .01 .04 .08 .10
Time 0 scores .39*** .38*** .45*** .45*** .68*** .46*** .42** .39***
Notes: Figures represent standardised Beta values. SWR =WIAT single word reading, RA =York reading accu-
racy, RC =York reading comprehension, Spell =WIAT spelling, BPVS =British Picture Vocabulary Scale, PA =
phonemic awareness, PK =phonic knowledge, AC =attitude to reading and perceived competence. Relationship
=relationship of child with volunteer (reported by volunteer); experience =child’s experience of programme
(reported by volunteer); additional supports =receipt of reading recovery or supplementary teaching with a
learning support teacher; programme intensity =average weekly number of sessions; school absence =number
of days absent from school over one academic year; time 0 scores =baseline scores on reading measure.
*Significant at the p <.10 level.
**Significant at the p <.05 level.
***Significant at the p <.001 level.
Table 6. Programme intensity levels and gains made by children receiving WoW
Dose Gain scores on outcome measures T0T2
SWR RA RC Spell BPVS PA PK AC
Low: <1.55 p.w. 5.48 4.45 1.11 3.15 1.81 8.81 4.40 0.39
Lowmedium: 1.561.8 p.w. 6.65 1.40 5.96 1.38 0.54 5.35 3.31 0.04
Mediumhigh: 1.812.07 p.w. 11.32 8.50 4.17 7.36 0.04 7.48 5.16 1.23
High: 2.08 +p.w. 6.85 4.79 5.54 1.44 1.46 8.12 4.61 1.38
Notes: SWR =WIAT single word reading, RA =York reading accuracy, RC =York reading comprehension,
Spell =WIAT spelling, BPVS =British Picture Vocabulary Scale, PA =phonemic awareness, PK =phonic
knowledge, AC =attitude to reading and perceived competence.
A one-to-one programme for at-risk readers 273
©2013 British Educational Research Association
intervention group were divided into four equal groups based on their programme
intensity levels as follows: low (<1.55 sessions per week), lowmedium (1.561.80),
mediumhigh (1.812.07) and high (2.08 +). There were no significant differences
in baseline scores between the four groups on any of the outcome measures reported.
However, there were significant differences in gain scores between T0 and T2 for
reading accuracy (RA), F (3, 92) =2.42, p =.07, reading comprehension (RC), F (3,
89) =2.92, p =.04, and spelling (Spell), F (3, 95) =2.76, p =.05. On reading com-
prehension (RC), the scores of the low programme intensity group worsened while
the scores for all others improved. The greatest gains were made by children with
mediumhigh programme intensity on scores for reading accuracy (RA) and spelling
(Spell), and this was the case on three other measures: SWR, BPVS and PK. The sec-
ond largest gains were made by the high programme intensity groups on five mea-
sures: SWR, RA, RC, PA and PK. On phonemic awareness and phonic knowledge,
there was little difference in gains scores between the four groups except that the
smallest gains were made by the lowmedium programme intensity group.
Finally, on scores for attitude to reading and perceived competence, in the inter-
vention group the greatest gains were made by the children who received the highest
level of programme intensity. The data have shown as well that intervention children
made greater gains on the measure of attitude to reading and perceived competence
(AC). In addition, there was a positive correlation between attitude to reading in class
at T1 and scores for reading comprehension (RC) at T2 (phi =.13, p =.09) and also
phonic knowledge (PK) at T1 (phi =.18, p =.03). This suggests that the more chil-
dren liked to read in class they better they performed in terms of comprehension and
phonic knowledge.
Discussion
There is considerable interest in the effectiveness of one-to-one reading programmes
for at-risk readers delivered by older adult volunteers. However, important research
questions remain unanswered given the limitations of recent evaluations, including
the small sample sizes, the small reported effect sizes, and in one case the study exam-
ined the effectiveness of three separate curricula. This study evaluated the effective-
ness of a unique reading programme designed by Barnardos, and effect sizes in the
study ranged from d =0.38 for single word reading and d =0.37 for phonemic
awareness, to d =0.24 for self-beliefs and d =0.23 for phonic knowledge. The find-
ings from this study are that a reading programme delivered by older adult volunteers
can lead to significant gains for at-risk readers in skills that are important for further
developments in reading. The success of the programme can be explained by the pri-
ority given to the phonics component, the efforts to make the programme a positive
and enjoyable experience for children, and the supports and training provided for
volunteers.
The greatest programme gains were made by children with ‘below average’ (<16th
percentile) reading levels who were not eligible for formal literacy interventions. This
is a group of children experiencing delays in the acquisition of reading skills but who
are not in the category of the most severe need. It is important to target this group, as
children who have problems with literacy acquisition at ages 6- to 7-years-old are
274 A. Fives et al.
©2013 British Educational Research Association
more likely to experience academic failure (Stanovich, 1986). The findings from this
study are in line with other evaluations, which have shown that while volunteer read-
ing programmes can successfully reduce the number of children in at-risk situations
(Vellutino et al., 1998; Pullen et al., 2004), they do not work best for children with
the most severe needs (Morrow-Howell et al., 2009a; see Brooks, 2002).
In this study gains were made by the intervention group in phonemic awareness
and phonic knowledge as well as decoding, which, according to the Simple View
of Reading, are skills needed for the further development of reading proficiency.
The Simple View of Reading posits two processes by which children become profi-
cient readers. On the one hand, phonemic awareness and phonic knowledge are
strong predictors of word recognition (Muter et al., 1994; Hulme et al., 2002; Pul-
len et al., 2004). On the other hand, reading comprehension is underpinned by
vocabulary but also by word recognition, for in order to comprehend what they
read, children need to be able to decode words (Stuart et al., 2008). According to
the Simple View of Reading, these two processes are both necessary for successful
reading but they are not one and the same process (Lyons et al., 2013). It is there-
fore possible that in this study the children in the intervention group were exposed
to a treatment that benefitted their phonics and word recognition skills but did
not benefit to the same extent their vocabulary and reading comprehension skills.
Although the WoW programme adopted a balanced literacy approach, as it
included ‘whole language’ as well as ‘phonics’ strategies, nonetheless one session
per week was devoted to phonics and extra training and support were provided to
volunteers in the delivery of these sessions. The phonics component itself focused
more on the manipulation of phonemes than on phonemegrapheme correspon-
dence, and the observed effect sizes for the former (phonemic awareness) were lar-
ger than for the latter (phonic knowledge). The phonemic awareness measure
combined five sub-tests, and the largest programme effects were observed on
scores for phoneme blending non-words, followed by scores for initial phoneme
identification and finally the combined measures of phoneme blending (real words
and non-words).
This study has shown that programme intensity was not as high as planned. While
children in the intervention group were scheduled to receive three sessions per week,
one of which being devoted to phonics, the average weekly number of sessions was
1.8. However, those who received approximately two sessions per week (a medium
high programme intensity level) tended to make the most progress and this would
suggest that two sessions per week is the adequate programme intensity level for
WoW. In addition, programme intensity was less important as a predictor on the
measures where overall programme effects were observed (single word reading, pho-
nemic awareness and phonic knowledge), and it was more important where WoW did
not lead to overall effects (reading comprehension, reading accuracy and spelling).
We would tentatively suggest that success in the latter areas could be a matter of
implementation (i.e., programme intensity level) rather than programme design. This
hypothesis is supported by the priority given to delivery of the phonics component of
the programme. WoW children were guaranteed to receive one phonics session per
week. However, if they received fewer than the recommended three sessions per week
they will have missed sessions on vocabulary building, reading comprehension or
A one-to-one programme for at-risk readers 275
©2013 British Educational Research Association
reading fluency. In addition, when they were withdrawn for their WoW sessions they
missed classroom reading activities that also could have benefitted their comprehen-
sion and vocabulary.
While some reading programmes combine self-esteem counseling with reading
interventions (Brooks, 2002), the WoW logic model posits that improvements in
self-belief may be generated as a result of participation in the programme and subse-
quent improvements in reading ability. The analysis has shown that WoW led to
improvements in children’s self-beliefs, that the greatest gains were made by those
who took part in reading sessions most frequently, and also that attitude to reading in
class was correlated with success in reading comprehension and phonic knowledge.
Improvements in children’s self-beliefs are important because of the association
between self-belief and reading achievement (Fives et al., in press). Other studies
have observed the ‘general tendency for higher achievement to be related to more
positive attitudes’ (Sainsbury & Schagen, 2004, p. 385) or a relationship of mutual
causality between self-beliefs and academic achievement (Marsh & O’Mara, 2008).
In the Experience Corps evaluation there was a positive and significant association
between the quality of the tutoring relationship (as perceived by the volunteers) and
reading achievement (Morrow-Howell et al., 2009b). In the WoW study, it was the
child’s experience of the programme (as reported by the volunteer) rather than the
relationship with the volunteer (as reported by the volunteer) that was positively asso-
ciated with programme effects, although it should be noted that these two variables
were strongly correlated with each other. Observations of the reading sessions con-
firmed that they were enjoyable experiences for the children, as volunteers ensured
that all of the children’s reading tasks received encouragement and support. Also,
since many volunteers were not available to deliver three sessions per week, it was
necessary to ‘pair’ children with more than one volunteer, which may explain why
relationship itself was not as important an explanatory variable as the children’s
experience of the programme.
The study of programme implementation also strongly suggests that the supports
and training provided for WoW volunteers met the requirements set out by Wasik
(1998) for a successful volunteer programme. Specifically, certified specialists were
in place to supervise the volunteers, namely the two project leaders; the volunteer
tutors received ongoing training and feedback from the project leaders; the tutoring
sessions were structured in accordance with a specific lesson plan, which was deter-
mined both by the Balanced Literacy Approach and by the reading level-appropriate
materials from the Oxford Reading Tree; and students were assessed on an ongoing
basis, and these assessment results were used to move children ‘up’ or ‘down’ on
the Oxford Reading Tree. Wasik states in addition that tutoring should be intensive
and consistent. The WoW programme was not delivered three times per week as
planned. However, the programme was delivered on average twice a week, and the
greatest gains were observed by children who received this level of programme
intensity.
WoW did not benefit all participants in the same way. We have already noted that
WoW was more beneficial for children below the 16th percentile. In addition, WoW
was more beneficial for boys. Other studies have concluded that boys may benefit
from such support because they may be more affected by parenting stress than girls
276 A. Fives et al.
©2013 British Educational Research Association
and also one-to-one volunteer support may help counteract challenges faced at
home (Ladd, 1996). The classroom environment also may be more suited to girls
reading at ‘average’ levels, if, as other studies have shown, girls already tend to have
more positive attitudes to reading, to read more, and to attain higher standards in
reading achievement (Sainsbury & Schagen, 2004; Logan & Johnston, 2009).
A related issue is the costs and benefits of removing children from the classroom,
where they are being taught by a certified teacher. The findings suggest that girls
and children reading at above the 16th percentile benefitted more from remaining in
the classroom.
Despite some limitations, this study contributes to our understanding of one-
to-one reading supports for at-risk readers provided by older adult volunteers. WoW
was successful for children reading below the 16th percentile and it was successful in
improving skills needed for further developments in reading. The success of WoW
can be explained in part by the effort to ensure the reading session was a positive and
enjoyable experience for the children, and the comprehensive and ongoing support
and training provided to volunteers by two project leaders with a professional
background in early education. Gains in phonemic awareness and phonic knowledge
may be explained by the priority given in volunteer training and in programme deliv-
ery to the phonics component, and gains in word recognition may be explained by its
close association with phonemic awareness and phonic knowledge, as hypothesized
by the Simple View of Reading. The absence of a programme effect in comprehension
or vocabulary may be explained both by programme priorities and by intervention
children missing classroom reading activities when withdrawn for WoW.
However, a number of limitations to the study should be noted. Although the study
protocol required children not to be in the reading recovery programme or in receipt
of supplementary teaching in English from a literary support teacher, a minority of
children in both groups received these supports during the course of the study.
Teaching staff made decisions on the receipt of such supports independently of the
research team or the WoW programme providers. Although control group children
were more likely to receive these supports in the second cohort, the data suggest that
receiving these supports did not influence results where an overall programme effect
was observed. A further limitation is that, although a decision was made to dedicate
one of the three weekly WoW sessions to phonics, it is not known how many of these
sessions were received. It is not possible to estimate what impact the phonics sessions
themselves had on outcomes independent of the other components of the WoW pro-
gramme (vocabulary building, reading comprehension and reading fluency). Finally,
a related limitation is that there was only one comparison group in the study. The
evaluation did not compare regular classroom teaching with a number of different
curricula with different strategies and/or content. Therefore, the study cannot con-
clude whether it was one specific component of WoW that was responsible for the
observed effects.
Acknowledgements
This paper is based on findings from the evaluation of the Wizards of Words (WoW)
reading programme. The programme is run by Barnardos, Ireland, and the Child and
A one-to-one programme for at-risk readers 277
©2013 British Educational Research Association
Family Research Centre, NUI Galway were commissioned by Barnardos to conduct
the evaluation. Funding was received from the Atlantic Philanthropies to carry out
this research.
References
Adams, M. J. (1990) Beginning to read: Thinking and learning about print(Cambridge, MA, MIT
Press).
Allor, J. & McCathren, R. (2004) The efficacy of an early literacy tutoring program implemented
by college students, Learning Disabilities Research and Practice, 19(2), 161181.
Altman, D. G. (1991) Randomization: Essential for reducing bias, British Medical Journal, 302
(6791), 14811482.
Altman, D. G., Moher, D., Schulz, K. F., Egger, M., Davidoff, F., Elbourne, D., Gotsche, P. &
Lang, T. (2001) The CONSORT statement: Revised recommendations for improving the
quality of reports of parallel group randomized trials, Annals of Internal Medicine, 134(8),
657662.
Baker, S., Gersten, R. & Keating, T. (2000) When less may be more: A two-year longitudinal evalu-
ation of a volunteer tutoring program requiring minimal training, Reading Research Quarterly,
35(4), 494519.
Barnardos, ., (2008a), Invitation to submit a proposal to evaluate a reading improvement service imple-
mented by Barnardos among first and second class primary school students in Dublin and Limerick
(Dublin, Barnardos).
Barnardos, (2008b) Wizards of Words’ manual for volunteers(Dublin, Barnardos).
Brooks, G.(2002) What works for children with literacy difficulties? The effectiveness of interven-
tion schemes (London, Department for Education and Skills). Available online at: http://www.
dcsf.gov.uk/research/data/uploadfiles/RR380.pdf (accessed March 2010).
Cohen, J.(1988) Statistical power analysis for the behavioural sciences(New Jersey, Lawrence Erlbaum
Associates).
Duff, F. J., Hayiou-Thomas, M. E. & Hulme, C. (2012) Evaluating the effectiveness of a phonolog-
ically based reading intervention for struggling readers with varying language profiles, Reading
and Writing: An Interdisciplinary Journal, 25(3), 621640.
Dunn, L., Dunn, L., Whetton, C. & Burley, J.(1997) The British Picture Vocabulary Scale (London,
GL Assessment).
Ehri, L. C., Nunes, S. R., Willows, D. M., Schuster, B. V., Yaghoub-Zadeh, Z. & Shanahan, T.
(2001) Phonemic awareness instruction helps children learn to read: Evidence from the
national reading panel’s meta-analysis, Reading Research Quarterly, 36(3), 250287.
Elbaum, B., Vaughan, S., Hughes, M. T. & Moody, S. W. (2000) How effective are one-to-one
tutoring programmes in reading for elementary students at risk for reading failure?
A meta-analysis of the intervention research, Journal of Educational Psychology, 92(4), 605619.
Fives, A., Russell, D., Kearns, N., Lyons, R., Eaton, P., Canavan, J., Devaney, C. & O’Brien,
A. (2013) The role of random allocation in Randomized Controlled Trials: Distinguishing
selection bias from baseline imbalance, Journal of MultiDisciplinary Evaluation, 9(20),
3342.
Fives, A., Russell, D., Kearns, N., Lyons, R., Eaton, P., Canavan, J., Devaney, C. & O’Brien, A.(in
press) The association between academic self-beliefs and reading achievement for children
experiencing delays in reading, The Journal of Research in Reading.
Foorman, B., Francis, D., Beeler, T., Winikates, D. & Fletcher, J.(1997) Early interventions for
children with reading problems: Study designs and preliminary findings, Learning Disabilities:
A Multi-disciplinary Journal, 8(1), 6371.
Hansford, B. C. & Hattie, J. A. (1982) The relationship between self and achievement/performance
measures, Review of Educational Research, 52(1), 123142.
Harcourt Assessment (2006) Wechsler Individual Achievement Test Second Edition for Teachers
(WIAT-II
UK
-T)(London, Harcourt).
278 A. Fives et al.
©2013 British Educational Research Association
Hatcher, P. J., Hulme, C., Miles, J., Carroll, J., Hatcher, J., Gibbs, S., Smith, G., Bowyer-Crane,
C. & Snowling, M. (2006) Efficacy of small group reading intervention for beginning readers
with reading-delay: A randomized controlled trial, Journal of Child Psychology and Psychiatry,47
(8), 820827.
Hatcher, P. J., Hulme, C. & Snowling, M.(2004) Explicit phoneme training combined with phonic
reading instruction helps young children at risk of reading failure, Journal of Child Psychology
and Psychiatry, 45(2), 338358.
Huck, C. S. & Pinnell, G. S.(1986) The Reading Recovery Project in Columbus, Ohio: Pilot year,
19845(Columbus, Ohio State University).
Hulme, C., Hatcher, P. E., Nation, K., Brown, A., Adams, J. & Stuart, G. (2002) Phoneme aware-
ness is a better predictor of early reading skills than onset-rime awareness, Journal of Experimen-
tal Child Psychology, 82(1), 228.
Ladd, G. W. (1996) Shifting ecologies during the five to seven year period, in: A. J. Sameroff, M.
M. Haith (Eds) The five to seven year shift (Chicago, The University of Chicago Press),
363386.
Lee, Y. S., Morrow-Howell, N., Jonson-Reid, M. & McCary, S. (2011) The effect of the Experi-
ence Corps
â
Program on student reading outcomes, Education and Urban Society, 44(1),
97118.
Logan, S. & Johnston, R. (2009) Gender differences in reading ability and attitudes: Examining
where these differences lie, Journal of Research in Reading, 32(2), 199214.
Lyons, R., Fives, A., Kearns, N., Canavan, J., Devaney, C. & Eaton, P. (2013) Exploring the utility
of the Simple View of Reading in Irish children attending schools in areas designated as socially
disadvantaged, Journal of Education and Training Studies, 1(1), 194203.
Marsh, H. W. & O’Mara, A. (2008) Reciprocal effects between academic self-concept, self-esteem,
achievement, and attainment over seven adolescent years: Unidimensional and multidimen-
sional perspectives of self-concept, Personality and Social Psychology Bulletin, 34(4), 542552.
Meier, J. & Invernizzi, M. (2001) Book Buddies in the Bronx: Testing a model for America Reads,
Journal of Education for Students Placed at Risk, 6(4), 319333.
Morris, D., Shaw, B. & Perney, J. (1990) Helping low readers in grades 2 and 3: An after-school
volunteer tutoring program, The Elementary School Journal, 91(2), 132150.
Morrow-Howell, N., Hong, S.-I. & Tang, F. (2009a) Who benefits from volunteering? Variations in
perceived benefits, The Gerontologist, 49(1), 91102.
Morrow-Howell, N., Jonson-Reid, M., McCrary, S., Lee, Y. S. & Spitznagel, E.(2009b) Evaluation
of Experience CorpsStudent reading outcomes(St Louis, MO, Center for Social Development,
Washington University in St Louis).
Muter, V., Hulme, C., Snowling, M. & Taylor, S. (1994) Segmentation, not rhyming, predicts early
progress in learning to read, Journal of Experimental Child Psychology, 71(1), 327.
Phillips, B. M. & Lonigan, C. J. (2005) Social correlates of emergent literacy, in: C. Hulme,
M. Snowling (Eds) The science of reading: A handbook (Malden, MA, Blackwell), 173187.
Pullen, P. C., Lane, H. B. & Monaghan, M. C. (2004) Effects of a Volunteer Tutoring Model on
the early literacy development of struggling first grade students, Reading Research and
Instruction, 43(4), 2140.
Pullmann, H. & Allik, J. (2008) Relations of academic and general self-esteem to school achieve-
ment, Personality and Individual Differences, 45(6), 559564.
Rimm-Kaufman, S. E., Kagan, J. & Byers, H. (1999) The effectiveness of adult volunteer tutoring
on reading among ‘at risk’ first grade children, Reading Research and Instruction,38(2),143152.
Ritter, G. W.(2000) The academic impact of volunteer tutoring in urban public elementary schools: Results of
an experimental design evaluation. Unpublished doctoral dissertation, University of Pennsylvania.
Sainsbury, M. & Schagen, I. (2004) Attitudes to reading at ages nine and eleven, Journal of Research
in Reading, 27(4), 373386.
Savage, R., Carless, S. & Stuart, M. (2003) The effects of rime- and phoneme-based teaching
delivered by learning support assistants, Journal of Research in Reading, 26(3), 211233.
Shadish, W. R., Cook, T. D. & Campbell, D. T.(2002) Experimental and quasi-experimental designs
for generalised causal inferences(Boston, MA, Houghton Mifflin Company).
A one-to-one programme for at-risk readers 279
©2013 British Educational Research Association
Slavin, R. E., Lake, C., Davis, S. & Madden, N. A. (2011) Effective programs for struggling
readers: A best-evidence synthesis, Educational Research Review, 6(1), 126.
Smith, S., Simmons, D. & Kame’enui, E.(1998) Phonological awareness: Curricular and instruc-
tional implications for diverse learners, in: D. C. Simmons, E. J. Kame’enui(Eds) What reading
research tells us about children with diverse reading needs: Bases and basics (Mahwah, NJ, Lawrence
Erblaum), 183218.
Snowling, M., Stothard, S., Clarke, P., Bowyer-Crane, C., Harrington, A., Truelove, E. & Hulme,
C. (2009) York Assessment of Reading for Comprehension(University of York, Centre for Reading
and Language).
Stanovich, K. E. (1986) Matthew effects in reading: Some consequences of individual differences
in the acquisition of literacy, Reading Research Quarterly, 21(4), 360407.
Stuart, M., Staintorp, R. & Snowling, M. (2008) Literacy as a complex activity: Deconstructing the
simple view of reading, Literacy, 42(2), 5966.
Torgesen, J. K., Wagner, R. K., Rashotte, C. A., Alexander, A. W. & Conway, T. (1997) Preventive
and remedial interventions for children with severe reading disabilities, Learning Disabilities,8
(1), 5161.
Valentine, J. C., DuBois, D. L. & Cooper, H. (2004) The relation between self-beliefs and aca-
demic achievement: A meta-analytic review, Educational Psychologist, 39(2), 111133.
Vellutino, F. R., Scanlon, D. M. & Tanzman, M. S. (1998) The case for early intervention in spe-
cific learning disability, Journal of School Psychology, 36(4), 367397.
Wasik, B. A. (1998) Volunteer tutoring programs in reading: A review, Reading Research Quarterly,
33(3), 266291.
Wechsler, D.(2002) Wechsler individual achievement test(San Antonio, TX, Psychological Corp).
Appendix. Timing of time 2 data collection
The results of an ANCOVA analysis of scores on phonemic awareness that controlled
for scores on the dependent variable from time 0 (i.e., before the intervention began)
indicated that the interaction between study condition, cohort and class was statisti-
cally significant, F (7, 194) =3.10, p <.001, and that class year interacted with study
condition only for cohort 1. For cohort 1, a statistically significant impact of the pro-
gramme was observed only for the first class (mean difference =1.41). By contrast, in
cohort 2 a statistically significant impact of the programme was observed for both the
first class (mean difference =3.93, Cohen’s d =0.62) and the second class (mean dif-
ference =3.18, Cohen’s d =0.66). Therefore, the programme was effective for sec-
ond class children in cohort 2 when data were collected from the control group and
the intervention group at the same time, and so the reported impact of the pro-
gramme cannot be explained by the collection of data from first class control group
children at an earlier time to all other participants.
©2013 British Educational Research Association
280 A. Fives et al.
... We are not aware of previous studies examining the effects of tutoring on student-reported attitudes in kindergarten. In two randomized field experiments of first grade students, Morgan, Fuchs, Compton, Cordray, and Fuchs (2008) found no improvements of motivation after students had been tutored whereas the tutoring program studied in Fives et al. (2013) improved students' attitudes to reading and perceived competence. In addition to being sparse, the evidence is therefore also conflicting. ...
... The results indicated that students experiencing reading failure had significantly lower motivation than students experiencing reading success, that tutoring improved decoding skills, but there were no improvements in motivation for tutored students. Fives et al. (2013) found beneficial effects of a first-grade literacy tutoring program delivered one-toone by older adult volunteers on a researcher-developed composite measure of attitudes to reading and perceived competence. They also found beneficial effects on a similar measure of decoding as ours, and on phonemic awareness and phonic knowledge. ...
... One reason may be that the tutored students in Morgan et al. (2008) were selected because they already had experienced reading failure, which our participants had not to the same extent as we implemented our program earlier (also for the students tutored in first grade). Fives et al. (2013) excluded students who already received specialist support. It may be easier to improve attitudes related to self-efficacy if students have not already experienced reading failure. ...
Article
This study examined a literacy program that targeted students most at risk of reading difficulties in kindergarten and first grade of 12 Swedish schools. The program used multi-sensory learning methods that focused on phonological awareness and phonics, and was delivered during 10 weeks over 30–35 sessions by teachers in a one-to-one or one-to-two setting. In total, 161 students aged 6–7 years were randomly assigned to a treatment group or a waiting list control group. The treatment group showed large and statistically significant improvements compared to the control group on the two pre-registered primary outcome measures: a standardized test of decoding (Hedges’ g = 1.07) and a standardized test of letter knowledge (g = 1.03). The improvements were also significantly larger on measures of phonological awareness (g = 0.56) and self-efficacy (g = 0.57), but not on measures of enjoyment and motivation. The program appears cost-effective relative to other tutoring programs.
... It is further hypothesized that this relationship and experience will bring about subsequent gains in both academic self-beliefs and reading achievement (Barnardos,8 A. Fives 2008a). Previously published findings show Wizards of Words had an impact on children's reading and academic self-beliefs at the end of the intervention period (T1) that was approaching statistical significance, and a statistically significant impact on children's reading at the follow-up (T2) time point (Fives, Kearns, Devaney, Canavan, Russell, Lyons, Eaton, & O'Brien, 2013b). ...
... At each time point, each child took part in three face-to-face sessions lasting between 20 and 30 minutes each. Data were collected at three time points for children recruited in first and second grade as follows (for a discussion of the T2 data collection in September and January, see Fives et al., 2013b). ...
... In a previous publication, the program was shown to be effective also for children aged 7-8 (i.e. children recruited at the start of second grade) on individual measures of single word reading, phonemic awareness, phonic knowledge, and academic self-beliefs (Fives et al., 2013b). The findings show that Wizards of Words was more successful for at-risk early readers than regular classroom teaching on its own. ...
Article
Full-text available
The current article examines what factors explained the success of a reading program delivered by older adult volunteers to at-risk early readers. The article also examines the direction of the relationship over time (both direct and mediated) between reading achievement, frequency of reading at home, and academic selfbeliefs. Two hundred and twenty nine socially disadvantaged children recruited at the start of first and second grade at risk of reading failure participated in the randomized controlled trial study over an 18-month period. The results from structural equation modeling show the combined instructional and emotional support was a proximal outcome, whose distal outcomes included gains in reading achievement. There was no evidence that either academic self-beliefs or frequency of reading at home mediated program impact on reading achievement. In addition, academic self-beliefs became more accurate and stable as children advanced through school. Reading achievement mediated the impact of earlier reading achievement on both academic self-beliefs and frequency of reading at home for children entering second and third grade. In contrast, for the youngest children (at the start of first grade), academic self-beliefs were negatively associated with subsequent academic self-beliefs and frequency of reading at home.
... Ten of the 16 studies deployed individuals who were specially recruited and trained (three of which were 1:1 computer-based interventions overseen by a trainer/s). Recruited individuals varied considerably and included masters students (Van der Kooy-Hofland, Bus, & Roskos, 2012), older adult volunteers (Fives et al., 2013;Fives, 2016) and higher-grade peers, student teachers and volunteers (Regtvoort, Zijlstra, & Leij, 2013). Four of the 16 interventions were delivered by school staff or TAs. ...
... Studies classified as reading interventions (RI) aimed to evaluate the efficacy of a specific reading programme (e.g. Wizards of Words in Fives et al., 2013, andFives, 2016) or an approach to reading (e.g. reading with phonology, reading alone, or phonology alone in Hatcher et al., 1994). ...
Article
This paper presents findings from a systematic literature review that explored the delivery arrangements of school-based reading interventions for struggling readers aged 5–8 years. Databases were searched for published and unpublished studies from 1970 to 2017, which employed a randomised controlled trial or quasi-experimental design. Sixty-four texts that met the inclusion criteria were categorised according to targeted tier based on the response-to-intervention model. Findings showed that tier 2/3 interventions targeting pupils who did not respond to whole class teaching (tier 1) were almost exclusively delivered in pull-out sessions by people other than the classroom teacher. At present, certain delivery arrangements appear to be used more than others in reading interventions conducted with young pupils in schools. This could to some extent reflect practices already used in primary schools and suggests that pupils who are identified for tier 2/3 support might have less access to their teacher's time and expertise.
... It will then be possible to explore further whether the positive self-beliefs of girls in the below-average reading category at age nine were " unduly disparate " or in contrast whether they functioned as a motivator for future literacy acquisition by age 13. The findings show that, although girls did not score significantly higher than boys for reading achievement, nonetheless, as other studies have reported (Fives et al., 2013;Fives et al., 2014;Kelley & Decker, 2009) girls had more positive self-beliefs than boys. The findings also show the interaction between AR and reading achievement was significantly different for boys and girls, as the positive association was stronger among girls than boys. ...
Article
Full-text available
This study explored the association between reading self-belief and reading achievement among a representative sample of nine year old children in the Republic of Ireland. Results from analysis of variance and simple effects analysis showed a positive linear association between reading achievement and attitude to reading. The association was observed in the full sample, was strongest among girls and among children in third grade, and did not differ significantly between social classes or between children who had attained different levels of reading achievement. However, there was evidence of inflated attitude to reading among children in the below average reading sub-sample.
... Our sample of participants were children at risk of reading failure, that is, children having problems with literacy acquisition in the first and second grades who were as a result more likely to experience academic failure (Stanovich, 1986). Although early prevention and intervention are recommended to address such difficulties (Pullen, Lane, & Monaghan, 2004), some interventions explicitly target self-beliefs along with reading achievement (Fives et al., 2013). The rationale for such an approach is based on the positive associations with reading achievement found when focusing on self-beliefs specific to the academic domain and self-beliefs matched to achievement measures (Hansford & Hattie, 1982;Logan & Johnston, 2009;Pullmann & Allik, 2008;Valentine et al., 2004). ...
Article
Full-text available
This paper investigates whether children's academic self-beliefs are associated with reading achievement and whether the relationship is modified by gender and/or age. Data were collected from children at risk of reading failure, that is, emergent readers (6- to 8-year-olds) in socioeconomically disadvantaged areas reading at a level below the population mean. The authors' own measure of attitude to reading and perceived competence was used. The study found a significant positive association between attitude to reading in class and vocabulary and phonemic awareness and a significant negative association between perceived competence at reading in class and single-word reading and spelling. Girls' attitude to reading and perceived competence were more positively associated with reading achievement, and this was most evident in the first grade. Perceived competence was inflated among those with the poorest reading and also among boys, in association with reading-related skills found most challenging by children in this sample.
Article
This meta‐analysis included experimental or quasi‐experimental intervention studies conducted between 1980 and 2020 that aimed to improve reading outcomes for Grade K‐5 students with or at risk for dyslexia (i.e., students with or at risk for word reading difficulties, defined as scoring at or below norm‐referenced screening or mean baseline performance thresholds articulated in our inclusion criteria). In all, 53 studies reported in 52 publications met inclusion criteria (m = 351; total student N = 6,053). We employed robust variance estimation to address dependent effect sizes arising from multiple outcomes and comparisons within studies. Results indicated a statistically significant main effect of instruction on norm‐referenced reading outcomes (g = 0.33; p < .001). Because there was significant heterogeneity in effect sizes across studies (p < .01), we used meta‐regression to identify the degree to which student characteristics (i.e., grade level), intervention characteristics (i.e., dosage, instructional components, multisensory nature, instructional group size), reading outcome domain (i.e., phonological awareness, word reading/spelling, passage reading, or reading comprehension), or research methods (i.e., sample size, study design) influenced intervention effects. Dosage and reading outcome domain were the only variables that significantly moderated intervention effects (p = .040 and p = .024, respectively), with higher dosage studies associated with larger effects (b = 0.002) and reading comprehension outcomes associated with smaller effects than word reading/spelling outcomes (b = −0.080). This meta‐analysis included experimental or quasi‐experimental intervention studies conducted between 1980 and 2020 that aimed to improve reading outcomes for Grade K‐5 students with or at risk for dyslexia (i.e., students with or at risk for word reading difficulties, defined as scoring at or below norm‐referenced screening or mean baseline performance thresholds articulated in our inclusion criteria).
Article
Full-text available
Background Low levels of numeracy and literacy skills are associated with a range of negative outcomes later in life, such as reduced earnings and health. Obtaining information about effective interventions for children with or at risk of academic difficulties is therefore important. Objectives The main objective was to assess the effectiveness of interventions targeting students with or at risk of academic difficulties in kindergarten to Grade 6. Search Methods We searched electronic databases from 1980 to July 2018. We searched multiple international electronic databases (in total 15), seven national repositories, and performed a search of the grey literature using governmental sites, academic clearinghouses and repositories for reports and working papers, and trial registries (10 sources). We hand searched recent volumes of six journals and contacted international experts. Lastly, we used included studies and 23 previously published reviews for citation tracking. Selection Criteria Studies had to meet the following criteria to be included: • Population: The population eligible for the review included students attending regular schools in kindergarten to Grade 6, who were having academic difficulties, or were at risk of such difficulties. • Intervention: We included interventions that sought to improve academic skills, were conducted in schools during the regular school year, and were targeted (selected or indicated). • Comparison: Included studies used an intervention‐control group design or a comparison group design. We included randomised controlled trials (RCT); quasi‐randomised controlled trials (QRCT); and quasi‐experimental studies (QES). • Outcomes: Included studies used standardised tests in reading or mathematics. • Setting: Studies carried out in regular schools in an OECD country were included. Data Collection and Analysis Descriptive and numerical characteristics of included studies were coded by members of the review team. A review author independently checked coding. We used an extended version of the Cochrane Risk of Bias tool to assess risk of bias. We used random‐effects meta‐analysis and robust‐variance estimation procedures to synthesise effect sizes. We conducted separate meta‐analyses for tests performed within three months of the end of interventions (short‐term effects) and longer follow‐up periods. For short‐term effects, we performed subgroup and moderator analyses focused on instructional methods and content domains. We assessed sensitivity of the results to effect size measurement, outliers, clustered assignment of treatment, risk of bias, missing moderator information, control group progression, and publication bias. Results We found in total 24,414 potentially relevant records, screened 4247 of them in full text, and included 607 studies that met the inclusion criteria. We included 205 studies of a wide range of intervention types in at least one meta‐analysis (202 intervention‐control studies and 3 comparison designs). The reasons for excluding studies from the analysis were that they had too high risk of bias (257), compared two alternative interventions (104 studies), lacked necessary information (24 studies), or used overlapping samples (17 studies). The total number of student observations in the analysed studies was 226,745. There were 93% RCTs among the 327 interventions we included in the meta‐analysis of intervention‐control contrasts and 86% were from the United States. The target group consisted of, on average, 45% girls, 65% minority students, and 69% low‐income students. The mean Grade was 2.4. Most studies included in the meta‐analysis had a moderate to high risk of bias. The overall average effect sizes (ES) for short‐term and follow‐up outcomes were positive and statistically significant (ES = 0.30, 95% confidence interval [CI] = [0.25, 0.34] and ES = 0.27, 95% CI = [0.17, 0.36]), respectively). The effect sizes correspond to around one third to one half of the achievement gap between fourth Grade students with high and low socioeconomic status in the United States and to a 58% chance that a randomly selected score of an intervention group student is greater than the score of a randomly selected control group student. All measures indicated substantial heterogeneity across short‐term effect sizes. Follow‐up outcomes pertain almost exclusively to studies examining small‐group instruction by adults and effects on reading measures. The follow‐up effect sizes were considerably less heterogeneous than the short‐term effect sizes, although there was still statistically significant heterogeneity. Two instructional methods, peer‐assisted instruction and small‐group instruction by adults, had large and statistically significant average effect sizes that were robust across specifications in the subgroup analysis of short‐term effects (ES around 0.35–0.45). In meta‐regressions that adjusted for methods, content domains, and other study characteristics, they had significantly larger effect sizes than computer‐assisted instruction, coaching of personnel, incentives, and progress monitoring. Peer‐assisted instruction also had significantly larger effect sizes than medium‐group instruction. Besides peer‐assisted instruction and small‐group instruction, no other methods were consistently significant across the analyses that tried to isolate the association between a specific method and effect sizes. However, most analyses showed statistically significant heterogeneity also within categories of instructional methods. We found little evidence that effect sizes were larger in some content domains than others. Fractions had significantly higher associations with effect sizes than all other math domains, but there were only six studies of interventions targeting fractions. We found no evidence of adverse effects in the sense that no method or domain had robustly negative associations with effect sizes. The meta‐regressions revealed few other significant moderators. Interventions in higher Grades tend to have somewhat lower effect sizes, whereas there were no significant differences between QES and RCTs, general tests and tests of subdomains, and math tests and reading tests. Authors’ Conclusions Our results indicate that interventions targeting students with or at risk of academic difficulties from kindergarten to Grade 6 have on average positive and statistically significant short‐term and follow‐up effects on standardised tests in reading and mathematics. Peer‐assisted instruction and small‐group instruction are likely to be effective components of such interventions. We believe the relatively large effect sizes together with the substantial unexplained heterogeneity imply that schools can reduce the achievement gap between students with or at risk of academic difficulties and not‐at‐risk students by implementing targeted interventions, and that more research into the design of effective interventions is needed.
Article
In this article, we review research on the outcomes of diverse reading programs on the achievement of struggling readers in elementary schools. Sixty‐five studies of 51 different programs met rigorous standards. Eighty‐three percent were randomized experiments and 17% quasi‐experiments. Outcomes were positive for one‐to‐one tutoring and were positive but not as large for one‐to‐small group tutoring. There were no differences in outcomes between teachers and teaching assistants as tutors. Whole‐class approaches (mostly cooperative learning) and whole‐class/whole‐school approaches incorporating tutoring for struggling readers obtained outcomes for struggling readers as large as those found for all forms of tutoring, on average, and benefited many more students. Technology‐supported adaptive instruction did not have statistically significant positive outcomes for struggling readers, however. In agreement with findings of previous reviews, in this synthesis, we found that substantial impacts can be obtained for struggling readers, with interventions aligned within a Response to Intervention network.
Article
Intergenerational Programs (IP) are increasingly present in all stages of formal education, mainly through intergenerational tutoring and mentoring IP. Improving reading skills has been one of the traditional purposes of IP. However, this study looks at something less frequent: a pilot IP around reading implemented as part of an intergenerational primary school model. After a multisite quasi-experimental control group design including 6–7 years old students (n = 184) from three experimental and three control primary schools in Spain, and a group of community dwelling seniors (n = 41) acting as voluntary mentors, no significant impact on children’s reading competence is identified. Reasons for this apparent failure are discussed, as well as recommendations for further studies on reading IP in intergenerational schools.
Chapter
Full-text available
If ethical principles can come into conflict, for example, in the debates about research ethics, in what way can philosophical analysis help resolve such conflicts? Through an examination of the work of Beauchamp and Childress, John Rawls, and Bernard Williams, Fives identifies a dual role for philosophy in research ethics. Through abstract and general theoretical reflection, which requires a significant degree of disengagement, we can examine such issues as whether moral conflicts arise in our ethical evaluation of research protocols. Through practical reasoning, which requires ongoing, direct involvement, we can pursue agreement on public matters, including cases where we are faced with moral dilemmas. This chapter combines theoretical analysis with a close examination of the moral dilemmas arising in one case study.
Article
Full-text available
Evidence suggests that children living in areas designated as socially disadvantaged may be at risk for literacy difficulties. The Simple View of Reading (Gough & Tunmer, 1986; Hoover &Gough, 1986) is a theoretical model that sets out to describe the processes that children need to develop to become proficient readers, that is word recognition or decoding processes and language comprehension processes. The overall aim of this study was to investigate the utility of the Simple View of Reading in a cohort of Irish children attending schools designated as socially disadvantaged. The objectives were twofold. The first objective was to investigate predictors of reading accuracy, spelling and reading comprehension skills in this sample. The second objective was to investigate whether or not these predictors varied with age. The sample comprised fifty-nine children attending two schools. Inclusion criteria were: children between the ages of 5-8 years; a sample of children within three ranges of reading abilities as rated by their teachers (teachers were asked to rate the children’s reading ability as above average, average or below average); and a minimum score of 80 on the British Picture Vocabulary Scales (BPVS) (Dunn et al., 1997) to be included. Reading accuracy (single word reading and passage reading), reading comprehension, and spelling were measured using three standardised tests i.e., Weschler Individual Attainment Test 11UK, Teachers Version (WIAT, 11UK-T) (Weschler, 2006), the York Assessment of Reading for Comprehension (YARC) (Snowling et al., 2009), and the Single Word Spelling Test (SWST), (Sacre & Masterson, 2000). Potential predictors of literacy skills, such as phonemic awareness and phonic knowledge skills, were measured using criterion-referenced tests. Vocabulary was also a potential predictor and was measured using the BPVS. There was mixed evidence for the Simple View of Reading. While phonic knowledge, phonemic knowledge and non-word reading were predictors of reading, other factors such as vocabulary were also predictors. Although vocabulary was a predictor of reading comprehension for the total sample, the picture was less clear when the effect of age was explored. To conclude, there was mixed evidence to support the Simple View of Reading. Nonetheless, the findings were encouraging in that children were acquiring literacy skills.
Article
Full-text available
This paper investigates whether children's academic self-beliefs are associated with reading achievement and whether the relationship is modified by gender and/or age. Data were collected from children at risk of reading failure, that is, emergent readers (6- to 8-year-olds) in socioeconomically disadvantaged areas reading at a level below the population mean. The authors' own measure of attitude to reading and perceived competence was used. The study found a significant positive association between attitude to reading in class and vocabulary and phonemic awareness and a significant negative association between perceived competence at reading in class and single-word reading and spelling. Girls' attitude to reading and perceived competence were more positively associated with reading achievement, and this was most evident in the first grade. Perceived competence was inflated among those with the poorest reading and also among boys, in association with reading-related skills found most challenging by children in this sample.
Article
As a nation, we are struggling to identify means of preventing early academic performance problems, particularly in reading and mathematics. In various forms, academically-focused tutoring programs for young children are being promoted widely as promising strategies for improving academic performance. There is a body of evidence showing the benefits of tutoring provided by certified, paid professionals. However, similar evidence does not exist for tutoring programs employing adult volunteers or college students as tutors. ^ This dissertation presents the results of an experimental-design study measuring the impacts on student outcomes of a major university-public school partnership that provides volunteer tutoring services for young students in inner-city elementary schools. The two primary hypotheses are: (1) Tutoring will lead to significantly improved academic outcomes as measured by academic grades and standardized test scores; and (2) Tutoring will lead to significant benefits in terms of student self-perception of academic ability, student motivation, and school attendance. ^ The study sample is comprised of 385 elementary school students identified by their teachers as in need of tutoring services. From this pool of "eligible" students, approximately half (51 percent) were randomly selected to work with college-age tutors. The remaining students served as a study control group. Regression analysis was used to analyze the effectiveness of the tutoring and the impact of factors other than tutoring on several outcomes of interest related to academic performance, self-confidence, and motivation. ^ There are two key findings from this study. First of all, the review of the research literature underscores how little we know about what, if anything, "works" in the way of volunteer tutoring to improve student outcomes. Second, the experimental-design study revealed no evidence that participation in the tutoring program during the 1998-99 academic year resulted in any measurable academic benefits for the tutees. The findings suggest that this type of intervention may not improve measurable academic outcomes in the short run. This study should serve to caution policymakers who believe that increasing volunteerism in schools will result in important academic benefits for low-income children in urban areas.
Article
This study evaluated the effectiveness of an intervention for reading-delayed children in Year-1 classes. METHODS: A sample (N = 77) of children drawn from 14 schools representing those with the weakest reading skills were randomly allocated to one of two groups. A 20-week intervention group received the intervention for two consecutive 10-week periods, while a 10-week intervention group only received the intervention for the second 10 weeks of the study. The programme was delivered in daily 20-minute sessions that alternated between small group (N = 3) and individual teaching. The programme combined phoneme awareness training, word and text reading, and phonological linkage exercises. RESULTS: The children receiving the intervention during the first 10-week period made significantly more progress on measures of letter knowledge, single word reading, and phoneme awareness than children not receiving the intervention. However, the children who only received the intervention during the second 10-week period made rapid progress and appeared to catch up with the children who had been given the more prolonged intervention. Failure to respond to the intervention was predicted by poor initial literacy skills and being in receipt of free school meals. CONCLUSION: A reading intervention programme delivered on a daily basis by trained teaching assistants is an effective intervention for children who show reading delays at the end of their first year in school. However, around one-quarter of the children did not respond to this intervention and these children would appear to need more intensive or more prolonged help to improve their reading skills.
Article
To comprehend the results of a randomized controlled trial (RCT), readers must understand its design, conduct, analysis, and interpretation. That goal can be achieved only through complete transparency from authors. Despite several decades of educational efforts, the reporting of RCTs needs improvement. Investigators and editors developed the original CONSORT (Consolidated Standards of Reporting Trials) statement to help authors improve reporting by using a checklist and flow diagram. The revised CONSORT statement presented in this article incorporates new evidence and addresses some criticisms of the original statement.The checklist items pertain to the content of the Title, Abstract, Introduction, Methods, Results, and Comment. The revised checklist includes 22 items selected because empirical evidence indicates that not reporting the information is associated with biased estimates of treatment effect or because the information is essential to judge the reliability or relevance of the findings. We intended the flow diagram to depict the passage of participants through an RCT. The revised flow diagram depicts information from 4 stages of a trial (enrollment, intervention allocation, follow-up, and analysis). The diagram explicitly includes the number of participants, according to each intervention group, included in the primary data analysis. Inclusion of these numbers allows the reader to judge whether the authors have performed an intention-to-treat analysis.In sum, the CONSORT statement is intended to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.
Article
There has been extensive debate among scholars and practitioners concerning whether self-beliefs influence academic achievement. To address this question, findings of longitudinal studies investigating the relation between self-beliefs and achievement were synthesized using meta-analysis. Estimated effects are consistent with a small, favorable influence of positive self-beliefs on academic achievement, with an average standardized path or regression coefficient of .08 for self-beliefs as a predictor of later achievement, controlling for initial levels of achievement. Stronger effects of self-beliefs are evident when assessing self-beliefs specific to the academic domain and when measures of self-beliefs and achievement are matched by domain (e.g., same subject area). Under these conditions, the relation of self-beliefs to later achievement meets or exceeds Cohen's (1988) definition of a small effect size.