ArticlePDF Available

Effects of the TutorBright tutoring programme on the reading and mathematics skills of children in foster care: a randomised controlled trial

Authors:

Abstract and Figures

We evaluated the effects of TutorBright tutoring on the reading and mathematics skills of children in family foster care, examined several potential moderators of the impact of tutoring, and explored possible ‘spill-over’ effects on the children’s executive functioning and behavioural difficulties and on their caregivers’ level of involvement in schoolwork in the home. The sample consisted of 70 children in care in Ontario, Canada. At the pre-test, the children were aged 5–16 years (M = 10.41, SD = 2.94) and enrolled in school grades 1–11 (M = 5.53, SD = 2.90). Thirty-four children were randomly assigned to tutoring and 36 to a waiting list control condition. Seven subtests from the Woodcock-Johnson III (WJ-III) achievement test served as outcome measures. The tutored children made statistically greater gains than those in the control group on the WJ-III subtests of Reading Fluency, Reading Comprehension, and Mathematics Calculation, but not on Word Reading, Spelling, Math Fluency, or Applied Math Problems. Age, executive functioning, caregiver controlling involvement in schoolwork, and self-reported post-traumatic stress disorder symptoms were found to moderate the effectiveness of tutoring. There were no spill-over effects of tutoring. The implications of the results for improving foster children’s reading and mathematics skills were discussed.
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=core20
Oxford Review of Education
ISSN: 0305-4985 (Print) 1465-3915 (Online) Journal homepage: https://www.tandfonline.com/loi/core20
Effects of the TutorBright tutoring programme on
the reading and mathematics skills of children in
foster care: a randomised controlled trial
Andrea J. Hickey & Robert J. Flynn
To cite this article: Andrea J. Hickey & Robert J. Flynn (2019) Effects of the TutorBright tutoring
programme on the reading and mathematics skills of children in foster care: a randomised
controlled trial, Oxford Review of Education, 45:4, 519-537, DOI: 10.1080/03054985.2019.1607724
To link to this article: https://doi.org/10.1080/03054985.2019.1607724
Published online: 26 Jul 2019.
Submit your article to this journal
Article views: 14
View Crossmark data
Eects of the TutorBright tutoring programme on the
reading and mathematics skills of children in foster care: a
randomised controlled trial
Andrea J. Hickey and Robert J. Flynn
Centre for Research on Educational and Community Services, University of Ottawa, Ottawa, Canada
ABSTRACT
We evaluated the eects of TutorBright tutoring on the reading and
mathematics skills of children in family foster care, examined several
potential moderators of the impact of tutoring, and explored pos-
sible spill-overeects on the childrens executive functioning and
behavioural diculties and on their caregiverslevel of involvement
in schoolwork in the home. The sample consisted of 70 children in
care in Ontario, Canada. At the pre-test, the children were aged
516 years (M= 10.41, SD = 2.94) and enrolled in school grades 111
(M= 5.53, SD = 2.90). Thirty-four children were randomly assigned
to tutoring and 36 to a waiting list control condition. Seven subtests
from the Woodcock-Johnson III (WJ-III) achievement test served as
outcome measures. The tutored children made statistically greater
gains than those in the control group on the WJ-III subtests of
Reading Fluency, Reading Comprehension, and Mathematics
Calculation, but not on Word Reading, Spelling, Math Fluency, or
Applied Math Problems. Age, executive functioning, caregiver con-
trolling involvement in schoolwork, and self-reported post-trau-
matic stress disorder symptoms were found to moderate the
eectiveness of tutoring. There were no spill-over eects of tutor-
ing. The implications of the results for improving foster childrens
reading and mathematics skills were discussed.
KEYWORDS
Children in care; foster care;
tutoring; academic skills;
mathematics; reading
Introduction
Many children in care have higher rates of learning disabilities, grade retention, and
below-average performance in mathematics and reading, compared with children in the
general population (McGuire & Jackson, 2018;OHiggins, Sebba, & Gardner, 2014; Sebba
et al., 2015; Tessier, OHiggins, & Flynn, 2018; Trout, Hagaman, Casey, Reid, & Epstein,
2008). They also often have cognitive and language decits and poor problem-solving or
reasoning skills (Kavanaugh, Dupont-Frechette, Jerskey, & Holler, 2016). Approximately
3050% of children in care are eligible for special education (Lightfoot, Hill, & LaLiberte,
2011), compared to 10% of children in the general population (Zetlin, Weinberg, & Shea,
2006). Without mastery of reading or mathematics, children in care are at an increased
risk of academic failure or school dropouts (Berlin, Vinnerljung, & Hjern, 2011; Trout
et al., 2008; Viner & Taylor, 2005). Based on national longitudinal register data in Sweden,
CONTACT Andrea J. Hickey ahick059@uottawa.ca
OXFORD REVIEW OF EDUCATION
2019, VOL. 45, NO. 4, 519537
https://doi.org/10.1080/03054985.2019.1607724
© 2019 Informa UK Limited, trading as Taylor & Francis Group
Forsman, Brännström, Vinnerljung, and Hjern (2016) found that among young adults
who had previously been in care, poor school performance at the end of obligatory
schooling had a causal and severe negative impact, including economic hardship, illicit
drug use, and poor mental health. Brännström, Vinnerljung, Forsman, and Almquist
(2017) recently extended their analyses in Sweden, demonstrating that the serious
negative psychosocial consequences of poor educational performance in childhood
persist until well into middle age (i.e. 3955 years).
Poor educational achievement is among the strongest risk factors for the future well-
being of children in care. As the Swedish research has shown, the links between poor
academic achievement and unfavourable psychosocial outcomes remain strong, even
after controlling for the socioeconomic status of the family when the child was still very
young, the length of time in care, and age at rst placement (Brännström et al., 2017;
Forsman et al., 2016).
Several meta-analyses have been conducted on interventions for at-risk students.
Lauer et al. (2006) found that although out-of-school-time programmes for low-
achieving students had only a small overall eect, one-to-one tutoring to improve
reading skills had a moderately large average eect (Hedges g= .50). Dietrichson,
Bøg, Filges, and Klint Jørgensen (2017), in a meta-analysis of approximately 100 aca-
demic interventions for children of low socioeconomic status, found that, on average,
the most eective academic interventions were, in order, one-to-one or small-group
tutoring (ES = .36), progress monitoring (in which teachers or students received infor-
mation about development; ES = .32), and cooperative or peer-assisted learning
(ES = .22). Slavin, Lake, Davis, and Madden (2009) found that one-to-one tutoring was
more eective than small-group tutoring or computer-assisted instruction programmes
for struggling readers in grades K-5. Finally, Pellegrini, Lake, Inns, and Slavin (2018)
recently completed a best-evidence synthesis of eective programmes in elementary
mathematics. In their review of 78 high-quality evaluations (83% randomised, 17% quasi-
experimental) of 61 programmes in grades K-5, they found that tutoring programmes
were found to have especially positive outcomes. One-to-one and one-to-small group
programmes had equal impacts, as did teachers and paraprofessionals as tutors. Overall,
tutoring (particularly in a one-to-one format) seems a promising means of improving the
academic skills of children who need help in reading or mathematics.
At present, there are relatively few evaluated interventions aimed at remediating the
educational challenges of young people in care. In a scoping review, Forsman and
Vinnerljung (2012) identied 11 studies that were either randomised controlled trials,
quasi-experiments, or pre-test-post-test studies. Overall, nine of the 11 evaluations
reported positive outcomes, with four out of ve tutoring evaluations yielding positive
results. Evans, Brown, Rees, and Smith (2017) reviewed the controlled research 12
projects, including 15 randomised controlled trials on academic interventions for
young people in care under the age of 19 years. Of the 12 projects, nine had demon-
strated some evidence of eectiveness. Three had employed academic tutoring, either
one-on-one (Flynn, Marquis, Paquet, Peeke, & Aubry, 2012; Marquis, 2013) or small
group-based (Harper & Schmidt, 2016). Evans et al. (2017) called for the continued use
of randomised trials but with greater attention to methodological quality, in order to
generate more convincing evidence about which academic interventions work well and
for whom.
520 A. J. HICKEY AND R. J. FLYNN
To our knowledge, only two randomised controlled trials of tutoring have been published
that have had positive results with children in care. Flynn et al. (2012) assessed the eective-
ness of one-to-one, direct-instruction tutoring (Maloney, 1998), delivered by foster carers, on
the reading and mathematics skills of 64 foster children, aged 6 to 13 years and in primary-
school grades 27. Thirty were in the tutoring group and 34 in a waiting list control group. The
tutored children made signicantly greater gains than those in the control condition on
sentence comprehension (Hedgesg= 0.38, p< .05), reading composite (g= 0.29, p< .10), and
mathematics computation (g= 0.46, p< .01), but not on word reading or spelling. Harper and
Schmidt (2016) also evaluated Maloneys(1998) version of direct-instruction tutoring, using
a small-group format and university students as tutors. The foster children were 613 years
old, in primary grades 18, and mainly of Indigenous origin. Forty-ve were randomly
assigned to the tutoring group and 46 to a waiting list control group. The tutoring group
made signicant gains on word reading (Hedgesg= 0.40, p< .001), spelling (g= 0.25, p< .02),
and mathematics computation (g= 0.34, p< .04), but not on sentence comprehension.
Overall, as Evans et al. (2017) have advocated, more high-quality research is needed.
The primary objective of the present study was to evaluate the impact of an individua-
lised, home-based tutoring programme, known as TutorBright, on the reading and
mathematics skills of children in family foster care. We hypothesised that the children
who had received academic tutoring would improve signicantly more on their mathe-
matics and reading skills than those in a waiting list control group. We also examined
whether there was a dose-response relationship between the amount of exposure to
tutoring and gains in reading and mathematics.
The second objective involved exploring whether selected covariates child age,
gender, executive functioning, behavioural diculties, post-traumatic stress disorder
(PTSD) symptoms, or caregiver involvement in academic activities in the home mod-
erated the impact of tutoring on reading and mathematics skills. The third objective
consisted of determining whether tutoring produced positive spillover eectson the
childrens executive functioning or behavioural diculties or on their caregiversinvol-
vement in academic activities in the home.
Method
Research context
The present study was a randomised eectiveness trial, conducted under real-world
conditions, rather than an ecacy trial, carried out under near-ideal, laboratory condi-
tions. The rst author (AJH) undertook the study as part of her doctoral thesis research in
clinical psychology (Hickey, 2018), of which the second author (RJF) was the research
supervisor. The study was approved by the Social Sciences and Humanities Research
Ethics Board of the University of Ottawa. This trial was not registered.
Recruitment and participants
Child welfare workers nominated foster children who had met several eligibility
criteria: enrolled in grades 111, uent in English (the language of the TutorBright
materials), currently living in a foster-family setting (including kinship-care and
OXFORD REVIEW OF EDUCATION 521
adoption-probation), and judged likely to remain in care for the duration of the study.
Foster children were excluded if they were living in a group home, were rated by their
child welfare workers as either strong students (and thus not likely to need tutoring),
or else were either intellectually disabled or very behaviourally disturbed (and thus not
likely to complete the programme). Seventy-ve children and their caregivers assented
or consented to participate and were enrolled in the study and randomised to either
TutorBright tutoring or a waiting list control condition.
At the pre-test, the 75 foster children were aged 516 years (M= 10.59, SD = 2.98;
see Table 1)andingrades111 in school (M= 5.68, SD = 2.95). The 70 foster parents
(64 females, 6 males) had been providing care to their respective children in care for
approximately 2 years (M=2.05years,SD = 1.03). Thirty-seven children were
randomly assigned to the TutorBright (experimental) group (20 males, 17
females; Mage = 10.32 years; range = 615 years) and 38 to the waiting list control
group (19 males, 19 females; Mage = 10.50 years; range = 516 years). Attrition was
low: 3 in the TutorBright group, 2 in the waiting list control group. (See CONSORT
diagram in Figure 1.)
Research design and random assignment
In our pre-test/post-test, waiting list-controlled, two-group design, random assignment to
conditions took place at the pre-test, immediately after the foster parents and foster
children had signed their consent or assent forms and had completed their initial assess-
ments. The children were assessed at two time points: Time 1 (pre-test) and Time 2 (post-
test, approximately 10 months after the pre-test). Unfortunately, given time constraints, no
longer term follow-up was conducted. The assessor was masked to the childs condition. At
Times 1 and 2, the children in both groups were assessed on their mathematics and reading
skills with selected subtests from the Woodcock-Johnson III (Woodcock, McGrew, & Mather,
2001). The foster parents were asked to complete a questionnaire package (which included
all of the measures noted below) while their child in care was being assessed. The ques-
tionnaire package took foster parents approximately 45 minutes to complete.
Experimental and waiting list control conditions
TutorBright tutoring programme
TutorBright was developed in 2007 in Toronto, Canada, by Mr. Sonny Verma. He stated
that the programme is based on Direct Instruction, which uses a well-organised instruc-
tional methodology and clearly structured teaching materials (Adams & Engelmann,
Table 1. Number of participants in each age group
x experimental group.
Age Total Number of Participants
waiting list Experimental
57 years 5 7
810 years 12 12
1113 years 13 7
1416 years 6 8
522 A. J. HICKEY AND R. J. FLYNN
1996). TutorBright had not been evaluated prior to the present research. However, the
eectiveness of Direct Instruction tutoring has been demonstrated both for children in
the general population (Flores & Ganz, 2009) and for children in care (Flynn et al., 2012;
Marquis, 2013). TutorBright aims to accelerate studentslearning in reading, language,
and mathematics and to create condence via mentoring relationships (S. Verma,
personal communication, 7 June 2017).
TutorBright uses one-on-one, in-home tutoring, with detailed instructors manuals
and customised student workbooks. The reading and mathematics curricula each have
10 progressive levels, with students moving to the next level after showing mastery of
the previous level. TutorBright also incorporates homework help, in which the tutor
provides aid in any academic subject of need (e.g. mathematics, reading, science,
geography, etc.). Requirements for TutorBright tutors include an undergraduate or
masters degree (completed or in progress), experience with teaching or mentoring,
strong communication skills, and a positive attitude. Each tutor-child match is based on
the academic needs of the child, geographic location, and mutual interests. Besides
facilitating tutoring, the one-to-one relationship with the tutor is thought to help the
child develop healthy relationships with other adults.
Throughout the current study, the TutorBright programme provided trained TutorBright
tutors, ongoing consultation to the research team, and regular monitoring of student pro-
gress. After being matched with a child, the tutor conducted home visits to assess the childs
reading, mathematics, and writing and determine the curriculum levels to use with the child.
The tutor then met individually with the child in the childs home for two one-hour sessions
51 potential participants
declined to participate due to:
behavioural difficulties,
unstable foster home, or not
wanting to be randomised to the
control group
TutorBright tutoring group
Eligible sample (N =126)
Randomisation & pre-test
(N= 75)
Waiting list control group
N= 37
Attrition: n= 3 (1 moved, 1
refused to continue to
participate, 1 withdrew due
to mental health difficulties)
N = 38
Attrition: n= 2 (1 moved; 1
refused to return for the
final assessment)
Data analysis
Analysed (n= 36) Analysed (n= 34)
Figure 1. CONSORT diagram (Schulz, Altman, & Moher, 2010).
OXFORD REVIEW OF EDUCATION 523
per week, on designated days of the week (e.g. every Monday and Wednesday, days available
to the families and tutors), for up to 50 hours of tutoring. The tutoring was aligned with the
school year, from October until the end of June or early July. Except for a two-week break for
Christmas holidays and a one-week break for March break (all of the children took this time o
from tutoring), the children did not miss more than two weeks of tutoring due to illness or
scheduling conicts. Any missed sessions were added onto the end of the tutoring period to
help ensure each child met the 50-hour goal. On average, the children in the present study
received a total of 47 tutoring sessions (48.66 hours of tutoring; SD = 4.06), over the course of
nine months. More specically, 32.4% of the sample completed 2346 sessions, 32.3%
completed 4749 sessions, and 35.3% completed 5055 sessions. During the study, they
received an average (M) of 14.10 hours of tutoring in reading (SD = 4.48), 13.97 hours in
mathematics (SD = 8.11), 10.94 hours in homework help (SD = 9.78), and 8.13 hours in other
tutoring (e.g. relationship-building, managing behaviour; SD = 3.80).
Waiting list control condition
Children in the waiting list control condition were asked to continue their schooling as
usualand not seek additional tutoring or academic support during the school year. They
were oered the tutoring intervention at the end of the school year. The participants
academic skills in the TutorBright and waiting list groups were assessed with the WJ-III
on two occasions, before the study started and at the end of the school year. To ensure
an equivalent amount of time between the pre-test and post-test in the two groups,
a waiting list child was randomly selected to be post-tested whenever a participant in
theTutorBright group reached 50 sessions of tutoring or came to the end of the
school year, whichever came rst.
Assessment of implementation delity and tutoring dosage
The delity of implementation by the tutors of each major TutorBright component, in
reading, mathematics, and homework help, was evaluated weekly by the rst author and
project coordinator (AJH). Another person, an experienced TutorBright coordinator,
oversaw the selection, training, deployment, and delity of implementation of the
TutorBright model, while making no research-related decisions. The assessment of
delity was based on the weekly performance data sent by the tutors to both coordi-
nators. These weekly data included the lessons covered and the average number of
minutes spent overall on tutoring, on reading and mathematics tutoring (based on the
TutorBright lessons), on homework help, and on o-task activities (e.g. relationship
building). The children in the tutoring condition completed an average of 47.56 sessions
(SD = 4.23, range = 3556) over a nine-month period and were judged by the project
and TutorBright coordinators as having received a high level of implementation.
Measures
Woodcock-Johnson III tests of achievement third edition (WJ-III; Woodcock et al., 2001)
The WJ-III is a norm-referenced, standardised series of tests that assess basic reading and
mathematics skills in individuals who are 2 to 90+ years of age or in grades K though
graduate school. The following subtests were administered in the current study: Letter-
Word Identication, Reading Fluency, Story Recall, Understanding Directions,
524 A. J. HICKEY AND R. J. FLYNN
Calculation, Math Fluency, Spelling, Passage Comprehension, Applied Problems, Story
Recall-Delayed, Picture Vocabulary, and Oral Comprehension. These subtests were
selected because they enable the calculation of an intra-achievementdiscrepancy
score. That is, an Oral Language score (i.e. a measure of oral language development,
derived from the Understanding Directions, Picture Vocabulary, and Oral
Comprehension subtests) can be used to predict an individuals level of reading and
mathematics achievement. According to Wilkinson and Robertson (2001), a signicant
discrepancy between a persons oral language ability and academic performance may
be indicative of a specic reading or mathematics learning disability.
A Reading Composite (Broad Reading) score is obtained by combining the Word
Reading, Reading Fluency, and Sentence Comprehension standard scores.
A Mathematics Composite (Broad Mathematics) score is obtained by combining the
Calculation, Math Fluency, and Applied Problems subtests.
Comprehensive executive function inventory parent version (CEFI; Naglieri &
Goldstein, 2012)
The CEFI is a norm-referenced, standardised, 100-item measure of executive functioning in
children ages 5 to 18 years. The CEFI was completed by the child or youthsfosterparent.
We used the CEFI total standardised score (M=100,SD = 15), with a higher score indicating
greater executive functioning (Naglieri & Goldstein, 2012). For the alpha coecients for this
sample for the CEFI and other instruments except the WJ-III, see Table 1.
Strengths and diculties questionnaire (SDQ; Goodman, 1997)
The SDQ uses parent or caregiver ratings to assess mental health problems in children
and youth aged 417 (Goodman & Goodman, 2009) on a 20-item Total Behavioural
Diculties scale, on which a higher score indicates a greater level of behavioural
problems.
Trauma symptom checklist for children alternate form (TSCC-A; Briere, 1996)
The TSCC (Briere, 1996) is a self-report, 44-item instrument that assesses a broad range
of traumatic symptoms in children and adolescents, ages 817. Because of the reading
level, the TSCC-A was used with children aged 10 and older.
Trauma symptom checklist for young children (TSCYC; Briere, 2001)
The TSCYC is a parent-report, 90-item instrument that assesses traumatic symptoms in
young children, aged 312 years. The caregivers of children aged 59 years completed
the instrument, with elevated T-scores indicating greater symptoms of posttraumatic
stress (Briere et al., 2001).
Parental support for learning scale (PSLS; Rogers, Markel, Midgett, Ryan, & Tannock,
2014)
The 48-item PSLS assesses the extent of caregiver support for educational activities in
the home. Instrumental parental involvement measures the degree of warmth, patience,
independence (i.e. more eective involvement) regarding the childs schoolwork, while
controlling parental involvement assesses the use of commands, punishment, nagging,
and disapproval (i.e. less eective involvement).
OXFORD REVIEW OF EDUCATION 525
Child welfare worker background information form
With this questionnaire, created by the research team, the child welfare worker provided
background information on the child (e.g. age of entry into care, maltreatment history, etc.).
Foster parent questionnaire
Also developed by the research team and administered at the pre-test and post-test, this
instrument gathered background information on the caregiver (e.g. number of children
in the home, length of time as a foster parent, etc., and supplementary information on
the child (medication, Individualized Education Plans, etc.).
Data analysis
The data were screened for missing data, univariate outliers, skewness, and kurtosis
(Tabachnick & Fidell, 2007; see Table 2). With a missing-data rate of less than 5% on most
variables, we used the expectation maximization (EM) algorithm in SPSS to address
missing data at the item level (Schafer & Graham, 2002).
To answer our main and exploratory research questions, we used ANCOVA via multiple
regression, the most powerful approach for analysing pre-test/post-test control-group
designs (Gliner, Morgan, & Harmon, 2003). We employed one-tailed tests to evaluate our
main hypothesis because of its clearly directional nature. That is, we expected the tutored
children to have higher adjusted mean scores at the post-test than the control children.
This choice aorded us greater statistical power, given the likelihood at the outset of the
study that we would end up with a relatively small research sample, even with data-
collection over two school years. Our experience in a previous tutoring RCT (Flynn et al.,
2012), in which we had also used one-tailed tests, had taught us to be conservative in
predicting our ultimate sample size. We also used correlational analyses to determine
whether a dose-response relationship existed between the amount of tutoring received
(measured in terms of the number of tutoring sessions completed or the total number of
tutoring hours and minutes received) and gains in WJ-III reading and reading.
We also used ANCOVA via multiple regression to answer our exploratory questions
regarding possible moderators of the impact of tutoring and potential spillover eects
from tutoring on the educationally relevant domains of child executive functioning or
behavioural diculties and caregiver involvement in schoolwork in the home. Given the
exploratory nature of these questions, we used two-tailed rather than one-tailed statistical
tests.
Eect size index
We used Hedgesgto assess the size of the eect of TutorBright on the foster childrens
reading and mathematics skills, as recommended by the What Works Clearinghouse
(WWC), 2008. We also adopted the WWC criterion according to which an eect size of
0.25 or greater should be considered substantively important, even if not statistically
signicant because of small sample size. A Hedgesg of 0.25 or larger reects a 10-
percentile or greater dierence between the means of the experimental and control
groups in a normal distribution.
526 A. J. HICKEY AND R. J. FLYNN
Table 2. Means (or percentages), standard deviations, Cronbachs alphas, theoretical range, and skewness for study variables at the pre-test, after attrition, for
the sample as a whole (combined TutorBright and waiting list control groups).
Pre-test (N= 70) Post-test (N= 70)
Variable Mean (or %) SD Cronbachs alpha
Theoretical
Range Skew Mean (or %) SD Cronbachs alpha
Theoretical
Range Skew
Outcome
Broad Reading 82.33 19.43 50150 1.00 84.43 18.50 50150 1.25
Letter-Word Identication 87.66 17.54 50150 .91 87.20 17.35 50150 1.07
Passage Comprehension 81.63 17.15 50150 1.16 84.43 14.20 50150 1.89
Reading Fluency 84.22 18.03 50150 .25 88.04 17.52 50150 .29
Spelling 84.27 19.10 50150 .58 84.73 19.61 50150 .52
Broad Mathematics 74.01 18.97 50150 .59 75.67 16.81 50150 1.25
Calculation 73.34 22.10 50150 .28 74.24 19.02 50150 .43
Applied Problems 83.16 15.28 50150 .97 85.20 13.88 50150 .53
Math Fluency 74.10 13.15 50150 .05 75.67 13.10 50150 .19
Covariates
Executive Functioning (CEFI total score) 81.89 13.12 0.92 50150 .05 83.63 13.05 50150 .61
Behaviour (SDQ total behavioural diculties) 16.19 6.59 0.81 040 .11 15.54 7.20 040 .09
PSLS caregiver involvement
Instrumental involvement
Controlling involvement
32.30
10.77
6.41
18.97
0.88
0.65
044
032
.06
-.17
31.86
12.09
5.38
4.90
044
032
.44
-.13
Symptoms of PTSD
TSCC (n= 42)
TSCYC (n= 28)
48.45
60.43
11.01
13.42
.97
.93
.72
.63
–– –
Age 10.41 2.94 516 .07 11.27 2.89 516 .12
Gender (male: female %) 54.3:45.7 –– –54.3:45.7 –– – –
Skewed variables were reected and transformed by means of square root transformations. However, the transformed data unexpectedly yielded fewer statistically signicant results, such
that the analyses conducted with the untransformed data were retained.
OXFORD REVIEW OF EDUCATION 527
Equivalence of groups at pre-test
Independent-samples t-tests and chi square analyses showed that there were no statis-
tically signicant pre-test dierences (p< .05) between the tutoring and control groups
on gender, age, school grade, total behavioural diculties, trauma symptoms, or WJ-III
subtests, indicating that randomisation had been eective (see Table 3). However, there
were signicant group dierences on the childs pre-care experience of abandonment,
with a higher rate in the control group (χ
2
[1, N= 70] = 5.90, p< .05), and on controlling
caregiver involvement in the childs schoolwork, with a higher mean score for caregivers
in the control group (t[68] = 2.46, p< .05).
Attrition
From the pre-test to the post-test a total of ve children dropped out of the study, for an
overall attrition rate of 6.67%. More specically, from the tutoring group, three children
(8.11%) dropped out, dueto mental health problems, a change in caregiver and subsequent
placement move, and a change in school programme that no longer included mathematics
and rendered tutoring less relevant. From the waiting list control group, two children
(5.26%) withdrew due, respectively, to reunication with the biological parents (with non-
return for the post assessment) and to refusal to return for the post-test.
Table 3. Sample means (SDs) or percentages at pre-test, after attrition.
Waiting list control group
(n= 36)
TutorBright tutoring group
(n= 34)
Pre-test Pre-test
Sex (M:F) 19:17 19:15
Age 10.50 (2.91) 10.32 (3.01)
School grade 5.69 (3.03) 5.35 (2.80)
Age of rst placement 5.68 (3.91) 5.24 (3.69)
Number of previous placements 1.85 (2.13) 1.94 (1.69)
Number of unplanned school changes 2.70 (2.18) 1.96 (1.62)
Does the child have an IEP at school Yes: 24 (66.7%) Yes: 22 (64.7%)
Long term health conditions:
ADHD
Learning Disability
Developmental Disability
Autism Spectrum Disorder
13 (36.1%)
8 (22.2%)
6 (16.7%)
3 (8.3%)
14 (41.2%)
13 (38.2%)
3(8.8%)
3(8.8%)
Reason for entry into care:
Neglect
Sexual Abuse
Domestic Violence
Emotional Harm
Abandonment*
Problem Behaviour
Other: Parental Mental Illness
30 (42.9%)
4 (5.7%)
15 (21.4%)
21 (30.0%)
10 (14.3%)
7 (10.0%)
0 (0.0%)
22 (25.3%)
6 (8.6%)
10 (14.3%)
22 (31.4%)
2 (2.9%)
3 (4.3%)
2 (2.9%)
Trauma Symptoms (PTSD T score):
TSCYC
TSCC
60.00 (11.75)
46.95 (11.49)
60.86 (15.36)
50.10 (10.49)
SDQ Total Diculties 15.50 (6.56) 16.91 (6.65)
CEFI Total Standard Score84.56 (11.47) 79.06 (14.29)
PSLS
Instrumental
Controlling*
30.94 (5.18)
11.92 (4.11)
33.74 (7.31)
9.56 (3.89)
*signicant dierence between groups, p< .05
trend, p< .10
528 A. J. HICKEY AND R. J. FLYNN
Results
Primary research question: eects of tutoring
Table 4 displays the means and SDs of the pre-test scores and the adjusted post-test
scores for the TutorBright (experimental) and waiting list control groups, on the seven
WJ-III subtests and overall Broad Mathematics and Broad Reading composite scores.
Table 4 also displays the t values, signicance level, eect size (Hedges g), 95% con-
dence intervals, and improvement index for each of the ANCOVAs.
Discrepancy analyses
ANCOVA via multiple regression was used to assess whether TutorBright tutoring reduced the
discrepancy between the childrens actual and predicted Broad Mathematics or Broad Reading
score, with a decrease suggesting an increase in skills (Woodcock et al., 2001). A pre-test-post-
test dierence score was rst calculated by subtracting the predicted Broad Reading and Broad
Mathematics scores (based on each participants Oral Language score) from the actual Broad
Reading and Broad Mathematics scores. The dierence between the adjusted post-test group
means was non-signicant for both Broad Reading (t[68] = .62, p= .54, 1-tailed) and Broad
Mathematics (t[65] = .58, p= .56, 1-tailed).
Results for primary research question 2: dose-response relationship
The calculation of Pearson correlations between the tutoring dosagemeasures and the
WJ-III outcome measures in the TutorBright group of 34 children in care revealed
a positive relationship, at the level of a trend, between Letter-Word Identication and
the total number of tutoring sessions received (r[34] = .30, p= .08). There were also
statistically signicant and negative relationships, at the level of a trend, between the
number of minutes spent on other activities during the tutoring sessions, namely,
behavioural management or relationship-building, and Broad Reading (r[34] = .30,
p= .09), Reading Fluency (r[34] = .32, p= .06), and Math Fluency (r[34] = .32, p= .07).
Results for exploratory research question 1: moderators of tutoring
Given the exploratory nature of these research questions, two-tailed tests were used in
these regression-based ANCOVAs.
Age
Age negatively moderated the eect of tutoring, at the level of a trend, on Broad
Mathematics (t[68] = 1.80, p= .08, 2-tailed), with the younger children benetting
relatively more than the older children.
Gender
Gender did not moderate the eect of tutoring (all ps > .20) but did have a signicant
main eect, at the level of a trend, on Spelling (t[68] = 1.75, p= .09, 2-tailed). The boys
made relatively greater gains than the girls.
OXFORD REVIEW OF EDUCATION 529
Table 4. Means and standard deviations on the pre-test and adjusted post-test scores and the ANCOVA results for the primary research question (t score, eect
size (Hedges g), 95% condence interval (C.I.) and improvement index) for the 7 WJ-III subtests and overall broad mathematics and broad reading scores.
Wait-list control group
(n= 36)
TutorBright tutoring group
(n= 34)
WJ-III sub-test Pre-test Adjusted post-test Pre-test Adjusted post-test tHedges g95% C.I. Improvement Index
Letter word identication 87.42 (16.78) 86.80 (15.60) 87.91 (18.57) 87.61 (17.27) t[68] = 0.25 0.05 0.42, +0.52 2% (i.e. 52.0
th
vs 50.0
th
percentile)
Reading uency 85.18 (16.08) 86.48 (13.88) 83.29 (19.95) 89.35 (17.21) t[68] = 2.21** 0.16 0.31, +0.63 6.4% (56.4
th
vs 50.0
th
percentile)
Passage comprehension 80.72 (18.80) 82.11 (13.22) 82.59 (15.44) 86.94 (10.86) t[68] = 2.03** 0.34 0.13, +0.81 13.3% (63.3
rd
vs 50.0
th
percentile)
Spelling 85.58 (19.92) 86.66 (18.97) 82.88 (18.38) 82.67 (17.50) N.A. 0.20 0.67-, +0.278.0% (42.0
nd
vs 50.0
th
percentile)
Broad Reading 82.19 (19.72) 83.17 (17.71) 82.47 (19.42) 85.77 (17.44) t[68] = 1.95* 0.14 0.33, +0.61 5.6% (55.6
th
vs 50.0
th
percentile)
Calculation 71.86 (21.63) 70.85 (15.73) 74.91 (22.81) 77.81 (16.58) t[68] = 2.03** 0.39 0.08, +0.86 15.2% (65.2
th
vs 50.0
th
percentile)
Math uency 72.58 (12.72) 73.96 (11.07) 75.71 (13.59) 77.46 (11.82) t[68] = .51 0.27 0.31, +0.85 10.6% (60.6
th
vs 50.0
th
percentile)
Applied problems 81.19 (14.79) 83.84 (10.03) 85.24 (15.74) 86.71 (10.67) t[68] = .06 0.21 0.26, +0.68 8.4% (58.4
th
vs 50.0
th
percentile)
(Table 4 continued)
Broad Mathematics 72.11 (18.81) 73.09 (15.00) 76.03 (19.20) 78.42 (15.30) t[68] = 1.29 0.32 0.15, +0.79 12.7% (62.7
th
vs 50.0
th
percentile)
N.A. = Not applicable, as we used only the positive tail in these 1-tailed hypothesis tests.
**p< .05, 1-tailed test. * p< .06, 1-tailed test.
530 A. J. HICKEY AND R. J. FLYNN
Executive functioning
On Letter Word Identication, children with higher CEFI scores performed relatively
worse than their lower-scoring counterparts (t[68] = 2.29, p= .03, 2-tailed).
Total behavioural diculties
SDQ behavioural diculties did not have a signicant moderating or main eect on any
of the WJ-III subtests (all ps > .20, 2-tailed).
Parental support for learning
Instrumental involvement had no signicant moderating or main eects on any of the
WJ-III subtests (all ps > .10, 2-tailed). Controlling involvement, however, did moderate
the impact of tutoring, at the level of a trend, on Reading Fluency (t[68] = 1.70, p= .09,
2-tailed) and on Broad Mathematics (t[68] = 1.68, p= .09, 2-tailed), with higher levels of
caregiver controlling involvement associated with poorer child schoolwork performance.
On the other hand, controlling involvement had signicant and positive eects on
Applied Problems (t[68] = 2.38, p= .02, 2-tailed), Passage Comprehension (t[68] = 3.14,
p< .01, 2-tailed), Reading Fluency (t[68] = 2.78, p= .01, 2-tailed), and Broad Reading (t
[68] = 3.02, p< .01, 2-tailed); higher levels of controlling involvement were related to
better academic performance.
PTSD symptoms
For the younger children (aged 59 years, n= 28), their total PTSD symptoms score did
not moderate the eect of TutorBright tutoring on any of the WJ-III subtests (all ps > .15,
2-tailed). However, total PTSD symptoms did have a signicant main eect on Spelling (t
[26] = 2.08, p= .05, 2-tailed), such that young children with higher total PTSD
symptoms performed more poorly.
For children aged 10 and older (n= 42), total self-reported PTSD symptoms negatively
moderated the impact of tutoring on math uency (t[40] = 2.15, p= .04, 2-tailed), such
that children with higher total PTSD symptoms performed more poorly than those with
lower total PTSD symptoms. PTSD symptoms also negatively moderated the impact of
tutoring on Mathematics Calculation, at the level of a trend, with children with greater
PTSD symptoms performing more poorly than those with lower PTSD scores (t
[40] = 1.72, p= .09, 2-tailed). A signicant main eect of total PTSD symptoms was
also found on Mathematics Calculation (t[40] = 3.32, p< .01, 2-tailed) and, at the level
of a trend, on Broad Mathematics (t[40] = 1.70, p= .09, 2-tailed), with higher levels of
PTSD associated with poorer performance on both measures.
Results for exploratory research question 2: positive spillover eectsof tutoring
A series of ANCOVAs conducted via multiple regression revealed no signicant dier-
ences between the adjusted post-test group means on any of the educationally relevant
domains examined, namely, child executive functioning, behavioural diculties, or
caregiver involvement in academic activities in the home (all ps > .10).
OXFORD REVIEW OF EDUCATION 531
Discussion
Overall, the children in care who participated in the study possessed below-average academic
skills, compared with children from the general population. At the pre-test, the children were
performing approximately 1 SD below the population mean on reading and 2 SDs below the
population mean on mathematics (see Table 4), an observation consistent with previous
ndings (Trout et al., 2008). At the post-test, we found that the TutorBright programme had
had statistically signicant and positive eects (p< .05) on Reading Fluency (g= .16), Reading
Comprehension (g= .34), and Mathematics Calculation (g= .39), marginally signicant eects
on Broad Reading (p< .06) and Applied Problems (p< .06), and statistically non-signicant but
substantively importanteects (i.e. above the WCC [2008] criterion of a Hedges gof 0.25 or
more)onMathFluency(g= .27) and Broad Mathematics (g= 0.32). At the same time,
TutorBright did not signicantly close the childrensintra-achievement gapbetween their
actual and potential achievement in mathematics and reading. Moreover, despite their gains
in mathematics and reading, the children in the tutoring group continued to perform at the
post-test at below-average levels in reading and mathematics (see Table 4).
The eects of tutoring on children in care seen in the current randomised study are
reasonably consistent with the meta-analysis by Ritter, Barnett, Denny, and Albin (2009)of
randomised evaluations of tutoring with primary-schoolagedchildreninthegeneralpopula-
tion. Ritter et al. found small to moderate eect sizes for one-to-one tutoring provided by
adult volunteers, with Hedgesgs ranging from 0.18 (Reading Comprehension) to 0.41
(Reading Words and Letters). Our statistically signicant eect on Mathematics Calculation
(g= .39) was larger than the non-signicant mean eect (g= .26) of Ritter et al. for the same
domain (which, however, was based on a small number of eect sizes).
The primary ndings from the current study are also consistent with previous aca-
demic tutoring studies carried out with children in care. With individualised direct-
instruction tutoring by foster parents, Flynn et al. (2012) found that the children in the
tutoring group had made signicant gains on sentence comprehension, reading com-
posite, and mathematics computation, but not on word reading or spelling. Similarly,
using a small-group direct-instruction tutoring programme for foster children in care of
mainly Indigenous ethnic background, Harper and Schmidt (2016) found that tutoring
produced signicant gains on word reading, spelling, and mathematics, but not on
sentence comprehension. Together, these ndings suggest that tutoring does improve
the mathematics and reading skills of children in care.
No signicant dose-response relationship was found between reading or mathe-
matics skills and the number of tutoring sessions (i.e. duration) or the number of
tutoring hours and minutes (i.e. intensity) experienced by the children, except for
a trend in the case of the number of tutoring sessions and letter-word identication.
This result is consistent with previous meta-analyses in the general population that have
suggested that the length (duration) of tutoring is seldom a signicant predictor of the
size of intervention eects (Elbaum et al., 2000; Suggate, 2010). On the other hand, it is
of potential pedagogical importance that negative relationships were found, at the level
of a trend, between the number of hours and minutes spent on the non-academic
activities of behaviour management or relationship building and the outcomes of
Reading Fluency, Math Fluency, and Broad Reading Composite. O-taskbehaviour
may thus be associated with lesser improvement in reading and mathematics.
532 A. J. HICKEY AND R. J. FLYNN
Regarding potential moderating variables, Marquis (2013), like us, found that gender
moderated the eectiveness of direct-instruction tutoring with foster children, with girls
tending to benet somewhat more than boys. In addition, we also identied two other
moderators; executive functioning and caregiver involvement. We suggest that the role
of potential moderators be investigated more systematically in future research. At the
same time, our examination of possible moderators involved a total of 72 ANCOVAs,
with a modest yield of only eight statistically signicant ndings (p< .05, 2-tailed) and 6
results at the level of a trend (i.e. p< .10, 2-tailed), which included both moderators and
main eects.
In other exploratory analyses, we found no evidence that TutorBright has a spillover
eect on child executive functioning or behavioural diculties or on caregiver involve-
ment in schoolwork in the home. These results are consistent with research by Marquis
(2013), Harper (2012), and Tideman et al. (2011), in which children in care experienced
little or no short-term improvement in mental health from tutoring. In children in the
general population, however, tutoring has been found to positively inuence non-
academic outcomes such as disruptive behaviour or social skills (Bowman-Perrott,
Burke, Zhang, & Zaini, 2014). Future tutoring research with children in care should
thus continue to look for positive spillover eects, while also including a qualitative
component in which caregivers and children are interviewed about such eects.
Overall, TutorBright emerged as relatively eective in improving the reading and
mathematics calculation skills of children in care. Also, the programme oers important
logistical advantages to local child welfare organisations in that it recruits, trains,
deploys, and supervises its tutors in the latterswork with children in the home.
Moreover, the emphasis of TutorBright on creating a positive relationship between
tutor and tutee aligns well with the stated preference of children in care for academic
interventions that have a clear relational component (Evans et al., 2017).
The positive results that we found for TutorBright emerged despite several limitations of
our study. First, although we recruited participants for two full years, we were ultimately
able to enroll and retain only 70 children and their caregivers in the research sample, 34 in
the TurotBright and 36 in the waiting list control group. These numbers correspond closely
to the criterion that Coyne and Kwakkenboss(2013) consider to be the minimum require-
ment for randomised trials, namely, 35 participants in each of the intervention and control
groups. Larger samples would obviously be desirable in future studies. Second, the con-
siderable geographic distance (412 kilometres, about 5 hours by car) between our uni-
versity research centre and the county in Ontario where the collaborating child welfare
organisation is located, as well as limited project funding, made the task of assessing
implementation delity relatively dicult. With greater nancial resources, future research
could conduct more frequent and detailed delity assessments (e.g. using video recording
with randomly selected tutored foster children). Future studies should also include
a process evaluation to assess delity as well as programme implementation and greater
information regarding the caregiver and child experience of the programme. Finally, the
lack of previous evaluations of the eectiveness of TutorBright provided us with little
programme-specic research guidance on how to proceed in the present study.
We believe that the eectiveness of TutorBright could be enhanced, in several ways.
Most important, the programme creators should strengthen the theoretical and empiri-
cal links between TutorBright and the research that has already been conducted on
OXFORD REVIEW OF EDUCATION 533
tutoring in the general population and with children in care. The Best Evidence
Encyclopedia (http://www.bestevidence.org/) contains useful research syntheses on
mathematics and reading tutoring and related interventions that could easily be con-
sulted when the next version of TutorBright is being prepared.
TutorBright could also benet from placing a more explicit emphasis on direct-
instruction tutoring. Direct instruction has been found to be eective in improving
educational outcomes in low-performance urban schools (Broman, Hewes, Overman, &
Brown, 2003) and, when combined with contingency management, enhances academic
outcomes for children at risk of school failure (Dolezal, Weber, Evavold, Wylie, &
McLaughlin, 2007). Although part of the TutorBright programme, direct instruction is
not its primary component. Instead, TutorBright emphasises homework helpto
improve in-class performance (S. Verma, personal communication, June 2017). Little
research, however, has been conducted on the eectiveness of homework help.
In addition, TutorBright could exploit the nding in the general population that parental
involvement in academics is often an important predictor of educational success (for meta-
analyses, see Hill & Tyson, 2009). Although little research on this specic topic has been
conducted to date with children in care, the latter do have a preference for interventions
delivered by caregivers (Evans et al. 2017), and Jackson (2007) has long been a strong
advocate of a much greater role for foster parents in the academic education of their foster
children. Thus, TutorBright would do well to promote the involvement of caregivers (e.g. by
having them read regularly with their foster children).
Finally, future research on TutorBright could seek to obtain teacher ratings of aca-
demic performance in the classroom, as programme components such as homework
help or relationship-building may improve the childs in-class behaviour. Future research
could also try to answer two important questions that tutoring research, even in the
general population, has yet to address adequately: are gains in reading or mathematics
maintained, and how much tutoring is needed to bring children up to the average
range? Unfortunately given that this study was part of a doctoral thesis, a longer term
follow-up to assess the staying powerof the intervention was not possible.
In conclusion, our ndings are broadly consistent with the nding of Forsman and
Vinnerljung (2012) that educational interventions for children in care, and especially
tutoring, tend to produce positive results. However, many children in care who have
received tutoring still remain at risk of poor academic functioning. More research is
needed to discover which academic interventions work best for children in care, in both
the short and long term.
Acknowledgments
The authors would like to acknowledge the families who participated in the study as well as the
collaborating staat the participating Childrens Aid Society and TutorBright.
Disclosure statement
No potential conict of interest was reported by the authors.
534 A. J. HICKEY AND R. J. FLYNN
Funding
This research was supported by the secondary authors (Robert J. Flynn) university research fund.
Notes on contributors
Andrea J. Hickey, PhD, C. Psych is a psychologist in supervised practice in Ottawa, Ontario. She
recently completed her PhD thesis under the supervision of Dr. Robert J. Flynn at the University of
Ottawa. Her thesis assessed the eectiveness of three academic interventions for children in care.
She is interested in the evaluation of interventions that try to improve academic outcomes for
children in care as well as research that examines the factors that predict academic outcomes for
children at risk of academic diculties.
Robert J. Flynn is an emeritus professor in the School of Psychology and a senior researcher at the
Centre for Research on Educational and Community Services at the University of Ottawa (Ontario,
Canada). Each year, he and his research team evaluate the service needs and psychological, social,
educational, and health outcomes of some 5,000 children, adolescents, and young adults residing
in foster, kinship, customary, or group care in Ontario. Annual feedback is provided to the province
and local Childrens Aid Societies. He and his students have also carried out several randomised
trials of academic tutoring that have shown that tutoring produces gains in reading and mathe-
matics in children in care.
References
Adams, G. L., & Engelmann, S. (1996). Research on direct instruction: 25 years beyond DISTAR. Seattle,
WA: Educational Achievement Systems.
Berlin, M., Vinnerljung, B., & Hjern, A. (2011). School performance in primary school and psycho-
social problems in young adulthood among care leavers from long term foster care. Children
and Youth Services Review,33, 24892497.
Borman, G. D., Hewes, G. M., Overman, L. T., & Brown, S. (2003). Comprehensive school reform and
achievement: A meta-analysis. Review of Educational Research,73, 125230.
Bowman-Perrott, L., Burke, M. D., Zhang, N., & Zaini, S. (2014). Direct and collateral eects of peer
tutoring on social and behavioral outcomes: A meta-analysis of single-case research. School
Psychology Review,43(3), 260285.
Brännström, L., Vinnerljung, B., Forsman, H., & Almquist, Y. B. (2017). Children placed in out-of-
home care as midlife adults: Are they still disadvantaged or have they caught up with their
peers? Child Maltreatment,22(3), 205214.
Briere, J. (1996). Trauma symptom checklist for children. Odessa, : Psychological assessment
resources.
Briere, J., Johnson, K., Bissada, A., Damon, L., Crouch, J., Gil, E., & Ernst, V. (2001). The trauma
symptom checklist for young children (TSCYC): Reliability and association with abuse exposure
in a multi-site study. Child Abuse & Neglect,25, 10011014.
Coyne, J., & Kwakkenbos, L. (2013). Triple P-positive parenting programmes: The folly of basing
social policy on underpowered awed studies. BMC Medicine,11, 11.
Dietrichson, J., Bøg, M., Filges, T., & Klint Jørgensen, A. M. (2017). Academic interventions for
elementary and middle school students with low socioeconomic status: A systematic review
and meta-analysis. Review of Educational Research,87(2), 243282.
Dolezal, D. N., Weber, K. P., Evavold, J. J., Wylie, J., & McLaughlin, T. F. (2007). The eects of a
reinforcement package for on-task and reading behavior with at-risk and middle school stu-
dents with disabilities. Child & Family Behavior Therapy,29,925.
Elbaum, B., Vaughn, S., Tejero Hughes, M., & Watson Moody, S. (2000). How eective are one-to-
one tutoring programmes in reading for elementary students at risk for reading failure? A meta-
OXFORD REVIEW OF EDUCATION 535
analysis of the intervention research. Journal of Educational Psychology,92, 605619.
doi:10.1037/0022-0663.92.4.605
Evans, R., Brown, R., Rees, G., & Smith, P. (2017). Systematic review of educational interventions for
looked-after children and young people: Recommendations for intervention development and
evaluation. British Educational Research Journal,43(1), 6894.
Flores, M. M., & Ganz, J. B. (2009). Eects of direct instruction on the reading comprehension of
students with autism and developmental disabilities. Education and Training in Developmental
Disabilities,3953.
Flynn, R. J., Marquis, R. A., Paquet, M. P., Peeke, L., & Aubry, T. D. (2012). Eects of individual direct-
instruction tutoring on foster childrens academic skills: A randomized trial. Children and Youth
Services Review,34, 11831189.
Forsman, H., Brännström, L., Vinnerljung, B., & Hjern, A. (2016). Does poor school performance
cause later psychosocial problems among children in foster care? Evidence from national
longitudinal registry data. Child Abuse & Neglect,57,6171.
Forsman, H., & Vinnerljung, B. (2012). Interventions aiming to improve school achievements of
children in out-of-home care: A scoping review. Children and Youth Services Review,34(6),
10841091.
Gliner, J. A., Morgan, G. A., & Harmon, R. J. (2003). Pretest-posttest comparison group designs:
Analysis and interpretation. Journal of the American Academy of Child & Adolescent Psychiatry,42,
500503.
Goodman, A., & Goodman, R. (2009). Strengths and diculties questionnaire as a dimensional
measure of child mental health. Journal of the American Academy of Child & Adolescent
Psychiatry,48(4), 400403.
Harper, J., & Schmidt, F. (2016). Eectiveness of a group-baed academic tutoring programme for
children in foster care: A randomized controlled trial. Children and Youth Services Review,67,
238246.
Harper, J. (2012). The eectiveness of a group-based tutorial Direct Instruction programme for long-
term fostercare children: A randomized controlled trial. School of Psychology, Lakehead
University. Retrieved from http://knowledgecommons.lakeheadu.ca:7070/handle/2453/385
Hickey, A. J. (2018). Improving academic outcomes for children in foster care through tutoring or
working memory training: Three randomized trials (Unpublished doctoral thesis). School of
Psychology, University of Ottawa. Retrieved from https://ruor.uottawa.ca/simple-rearch?query=
Hickey+and+Flynn&location=10393%2F11105&rpp=10&sort_by =score&order=desc
Hill, N. E., & Tyson, D. F. (2009). Parental involvement in middle school: A meta-analytic assessment
of the strategies that promote achievement. Developmental Psychology,45(3), 740.
Jackson, S., & Ajayi, S. (2007). Foster care and higher education. Adoption & Fostering,31(1), 6272.
Kavanaugh, B. C., Dupont-Frechette, J. A., Jerskey, B. A., & Holler, K. A. (2016). Neurocognitive
decits in children and adolescents following maltreatment: Neurodevelopmental conse-
quences and neuropsychological implications of traumatic stress. Applied Neuropsychology:
Child,115.
Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L. (2006). Out-of
-school-time programmes: A meta-analysis of eects for at-risk students. Review of Educational
Research,76(2), 275313.
Lightfoot, E., Hill, K., & LaLiberte, T. (2011). Prevalence of children with disabilities in the child
welfare system and out of home placement: An examination of administrative records. Children
and Youth Services Review,33(11), 20692075.
Maloney, M. (1998). Teach your children well: A solution to some of North Americas educational
problems. Belleville, ON: QLC Educational Services.
Marquis, R. A. (2013). The gender eects of a foster parent-delivered tutoring programme on foster
childrens academic skills and mental health: A randomized eld trial (Unpublished doctoral
thesis). School of Psychology, University of Ottawa. doi:10.20381/ruor-3093
McGuire, A., & Jackson, Y. (2018). A multilevel meta-analysis on acaadmic achievement among
maltreated youth. Clinical Child and Family Psychology Review. doi:10.1007/s10567/-018-0265-6
536 A. J. HICKEY AND R. J. FLYNN
Naglieri, & Goldstein. (2012). Comprehensive executive function inventory. Toronto: Multi Health
Systems.
OHiggins, A., Sebba, J., & Gardner, J. (2014). What are the factors associated with educational
achievement for children in kinship or foster care: A systematic review. Oxford, UK: Rees Centre for
Fostering and Education, University of Oxford.
Pellegrini, M., Lake, C., Inns, A., & Slavin, R. E. (2018). Eective programmes in elementary mathematics:
A best-evidence synthesis. Best Evidence Encyclopedia. Retrieved from http://www.bestevidence.
org/
Ritter, G. W., Barnett, J. H., Denny, G. S., & Albin, G. R. (2009). The eectiveness of volunteer tutoring
programmes for elementary and middle school students: A meta-analysis. Review of Educational
Research,79(1), 338.
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological
Methods,7, 147.
Schulz, K., Altman, D., & Moher, D. (2010). CONSORT 2010 Statement Updated guidelines for
reporting parallel group randomised trials. BMJ, 340332.
Sebba, J., Berridge, D., Luke, N., Fletcher, J., Bell, K., Strand, S., . . . OHiggins, A. (2015). The
educational progress of looked after children in England: Linking care and educational data.
Oxford,: Rees Centre for Research in Fostering and Education and University of Bristol.
Slavin, R. E., Lake, C., Davis, S., & Madden, N. A. (2009). Eective programmes for struggling readers:
A best-evidence synthesis. Best Evidence Encyclopedia, School of Education, Johns Hopkins
University.
Suggate, S. P. (2010). Why what we teach depends on when: grade and reading intervention
modality moderate eect size. Developmental Psychology,46(6), 15561579. doi:10.1037/
a0020612
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (6th ed.). Boston, MA: Pearson
Education.
Tessier, N. G., OHiggins, A., & Flynn, R. J. (2018). Neglect, educational success, and young people in
out-of-home care: Cross-sectional and longitudinal analyses. Child Abuse & Neglect,75, 115129.
Tideman, E., Vinnerljung, B., Hintze, K., & Aldenius Isaksson, A. (2011). Improving foster childrens
school achievements: Promising results from a swedish intensive study. Adoption & Fostering,35
(1), 4456.
Trout, A. L., Hagaman, J., Casey, K., Reid, R., & Epstein, M. H. (2008). The academic status of children
and youth in out-of-home care: A review of the literature. Children and Youth Services Review,30
(9), 979994.
Viner, R. M., & Taylor, B. (2005). Adult health and social outcomes of children who have been in
public care: Population-based study. Paediatrics,115, 894899.
What Works Clearinghouse ([WWC]). (2008). Procedures and standards handbook (version 2.0).
Washington, DC: US Institute of Educational Sciences.
Woodcock, R., McGrew, K., & Mather, N. (2001). Woodcock-Johnson tests of achievement manual.
Zetlin, A. G., Weinberg, L. A., & Shea, N. M. (2006). Seeing the whole picture: Views from diverse
participants on barriers to educating foster youths. Children and Schools,28, 165173.
OXFORD REVIEW OF EDUCATION 537
... Internationellt sett har även så kallade tutoringprogram med gruppvis eller enskilt stöd från pedagoger eller frivilliga vuxna fått bra resultat vid utvärderingar (Flynn m.fl. 2012;Hickey & Flynn 2019). ...
Article
Full-text available
[English] National and international studies have shown that children with experience of out-of-home care (OHC) have poor school performance compared to their peers. However, we know less about their educational careers over time. Based on a 50-year follow up of 12 000 individuals, out of which nearly 8 percent had been placed in OHC before their 13th birthday, this study mapped out their educational pathways over the life course. The analyses focused on how the OHC group’s educational ambitions, achievement, and choices from sixth grade and onwards differed from children who had come in contact with the child welfare system without being placed, and other majority population peers. The results showed that the OHC group consistently had poorer achievements, and made less ambitious choices compared to their majority population peers. However, the OHC group had similar or even better outcomes compared to the child welfare contact group. The results also showed that opportunities to resume studies in adulthood had played an important role for children in general and children in OHC in particular. [Svenska] Såväl svensk som internationell forskning visar att barn som placerats i samhällsvård har sämre skolresultat än andra barn. Däremot är kunskapen om hur deras utbildningskarriärer ser ut över en längre tid begränsad. Med hjälp av en femtioårig uppföljning av en kohort bestående av drygt 12 000 personer födda 1953, av vilka nästan 8 procent varit placerade före 13 års ålder, har vi följt deras utbildningsvägar över nästan hela livsförloppet. Analyserna fokuserade på hur de placerade barnens utbildningsambitioner, prestationer och val från årskurs 6 och framåt skilde sig från barn som haft barnavårdskontakt utan att placeras respektive barn utan barnavårdskontakt. Resultaten visade att de placerade barnen genomgående presterat sämre och gjort mindre ambitiösa utbildningsval än barnen utan barnavårdskontakt. Däremot var de placerade barnens utbildning lik och i vissa fall till och med bättre än utbildningen för de som haft barnavårdskontakt. Våra resultat visade också att möjligheterna att återuppta studier i vuxen ålder tycks ha spelat en viktig roll för alla barn, inte minst de placerade barnen.
... That would explain the differences found with literature reviews and meta-analysis by Alegre-Ansuategui et al. (2018) and Alegre et al. (2019a). The similar scores reported for tutors and tutees in both educational levels is also consistent with recent peer tutoring research (Leung, 2019b;Shin et al., 2019) as peer tutoring is expected to benefit academic achievement for most of the students independently of the roles they play (Hickey & Flynn, 2019). ...
Article
Full-text available
Peer tutoring in Mathematics has reported academic benefits across many educational levels, from Preschool to Higher Education. However, recent literature reviews and meta-analysis state that students experience higher gains in Primary or Elementary Education (ages 7–12 years) than in secondary education or middle school and high school (ages 13–18 years). This study examined the effects of peer tutoring on students’ mathematics achievement in primary and secondary education under similar settings. 89 students from first, fourth, seventh, and ninth grades participated in the study. The design of this research was quasi-experimental with pretest–posttest without control group. The statistical analysis reported significant improvements for both, Primary and Secondary Education. The comparison between these educational levels showed that there were no significant differences in the increments of the students’ marks. The global effect size reported for the experience was Cohen’s d = 0.78. The main conclusion is that Peer Tutoring in Mathematics reports similar academic benefits for both, Primary and Secondary Education. Future research must be conducted as the superiority of Peer Tutoring in Primary over Secondary Education has yet to be proved in the Mathematics subject.
... Others try to enhance literacy and numeracy skills by using books or games as gifts that are mailed directly to the foster child, such as "Letterbox Club" (Griffiths, 2012). Some provide extra-curricular study support by providing tutoring assistance, by instructing foster parents on how to enhance their ability to support children in performing their school tasks Hickey and Flynn, 2019). Some interventions enhance skills by engaging external resources such as voluntary university students, using a specific Direct Instruction method delivered in a group setting (Harper and Schmidt, 2012). ...
Article
Full-text available
Interventions aimed at improving school performance for children in foster care are few and are generally not implemented. By preventing failure in school, the prospect of reducing the risk for future poor health, substance abuse, unemployment, and other detrimental social conditions are met. This paper focuses on the change of preconditions for compulsory school performance in out-of-home care children, following an intervention called “Skolfam” that aims to improve school performance by individual assessments and school-based interventions. In this study, data were compiled from prospective repeated tests of 475 children in foster care in Sweden. Educational preconditions were analysed for compulsory school performance, such as intelligence (WISC-IV), psychosocial (SDQ) and adaptive behavior (ABAS-II), literacy (Reading Chains) and mathematical skills (Magne Mathematic Diagnoses) before and after the first 2 years of the “Skolfam” intervention. All tests were age-standardized and performed by experienced professionals. The results showed improved skills in complex aspects of literacy, mathematics, and cognitive performance, but no improvement in less complex literacy skills, adaptive behavior or mental health symptoms. In conclusion, higher-order cognitive functions can develop positively when appropriate school support is provided. Affective function, adaptive behavior, and psychosocial well-being present a more pervasive challenge for children in foster care. Implications for future research, practice in social services, and school is that further development of methods to aid future prospects for children in out-of-home care should aim to improve both cognitive higher-order executive-, and affective functions.
Article
Full-text available
Students of Office Technology and Management (OTM) in Nigerian polytechnics have consistently performed below expectations both academically and at work. This may be attributed to use of inappropriate instructional method. The need to improve this situation necessitated the research on effects of peer tutoring on students’ academic achievement in OTM in Nigerian polytechnics. Using a research question to guide the study, and a null hypothesis was tested at 0.05 level of significance. A non-randomized quasi experimental research design, adopting pre-test; post-test non-equivalent control group design was used. South West Nigeria was the region of the research work with a populace of 503 National Diploma Year II (ND II) OTM students in four federal polytechnics. Using cluster random sampling, a sample of the intact classes of 227 students from two states in the area of the study was drawn. Three OTM experts validated the “Office Technology and Management Achievement Test (OTMAT)” 100 items instrument for face and content validity. The Experts input modified some items and cancelled some which reduced 140 test items to 100 used for collecting data. Descriptive statistics of mean was used to analyze data and to respond to the research question while t-test and Analysis of Covariance were used to test the null hypothesis. Kuder-Richardson Formula 21 was adopted to establish internal consistency and reliability of the instrument which yielded a coefficient of 0.86. The research showed that Peer Tutoring Instructional Technique (PTIT) has higher positive effects on students’ academic achievement in OTM than Teacher-dominated Instructional Approach (TDIA), thus it can enhance students’ academic achievement in OTM. It was, therefore, suggested that OTM lecturers should include the use of Peer Tutoring Instructional Technique to boost learning in teaching their students.
Article
Full-text available
Maltreatment can influence normative development and negatively impact emotional, behavioral, and social functioning in youth. As a result, it is not surprising that maltreated youth, as compared to non-maltreated youth, tend to underperform academically. Research on the academic performance of maltreated youth has increased over the last decade and several review papers have been published in this area. While the conclusions of these review articles have been that maltreated youth are at greater risk for academic deficits as compared to their non-maltreated peers, there are several conflicting findings within the literature that make it difficult to determine if or to what extent maltreated youth may demonstrate academic difficulty. Using a multilevel, structural equation model meta-analysis technique, the current study sought to provide a quantitative synthesis of the literature by examining the mean difference between maltreated and non-maltreated youth on measures of academic performance. Moreover, the current study also examined group differences between academic subject and maltreatment type. A total of 72 effect sizes were extracted from 32 studies that met inclusion criteria. Results demonstrated an overall negative, medium effect size, such that maltreated youth tended to perform slightly greater than half a standard deviation below non-maltreated youth on measures of academic performance. Moderation analyses suggest deficits may be greater on measures of general academic performance, as compared to language arts measures. No differences were observed for maltreatment type. These findings highlight the need for increased focus on academic difficulties among maltreated youth.
Article
Full-text available
Socioeconomic status is a major predictor of educational achievement. This systematic review and meta-analysis seeks to identify effective academic interventions for elementary and middle school students with low socioeconomic status. Included studies have used a treatment-control group design, were performed in OECD and EU countries, and measured achievement by standardized tests in mathematics or reading. The analysis included 101 studies performed during 2000 to 2014, 76% of which were randomized controlled trials. The effect sizes (ES) of many interventions indicate that it is possible to substantially improve educational achievement for the target group. Intervention components such as tutoring (ES = 0.36), feedback and progress monitoring (ES = 0.32), and cooperative learning (ES = 0.22) have average ES that are educationally important, statistically significant, and robust. There is also substantial variation in effect sizes, within and between components, which cannot be fully explained by observable study characteristics.
Article
Full-text available
International research has consistently reported that children placed in out-of-home care (OHC) have poor outcomes in young adulthood. Yet, little is known about their outcomes in midlife. Using prospective data from a cohort of more than 14,000 Swedes born in 1953, of which nearly 9% have been placed in OHC, this study examines whether there is developmental continuity or discontinuity of disadvantage reaching into middle age in OHC children, compared to same-aged peers. Outcome profiles, here conceptualized as combinations of adverse outcomes related to education, economic hardship, unemployment, and mental health problems, were assessed in 1992-2008 (ages 39-55). Results indicate that having had experience of OHC was associated with two-fold elevated odds of ending up in the most disadvantaged outcome profile, controlling for observed confounding factors. These findings suggest that experience of OHC is a strong marker for disadvantaged outcomes also in midlife.
Article
Full-text available
Looked-after children and young people (LACYP) are educationally disadvantaged compared to the general population. A systematic review was conducted of randomised controlled trials evaluating interventions aimed at LACYP aged ≤18 years. Restrictions were not placed on delivery setting or delivery agent. Intervention outcomes were: academic skills; academic achievement and grade completion; special education status; homework completion; school attendance, suspension, and drop-out; number of school placements; teacher-student relationships; school behaviour; and academic attitudes. Fifteen studies reporting on 12 interventions met the inclusion criteria. Nine interventions demonstrated tentative impacts. However, evidence of effectiveness could not be ascertained due to variable methodological quality, as appraised by the Cochrane risk of bias tool. Theoretical and methodological recommendations are provided to enhance the development and evaluation of educational interventions.
Article
Full-text available
This meta-analysis examined the direct (primary) and collateral (secondary) effects of peer tutoring on social and behavioral outcomes for 128 participants in prekindergarten through grade 12 across 20 studies using SCR designs. The overall TauU weighted effect size across studies was 0.62 (95% CI [0.58, 0.66]), indicating that a small to moderate effect on behavioral and social outcomes can be attributed to peer tutoring. Moderator analyses indicated that cross-age tutoring, peer tutoring interventions that did not use reward contingencies , and interventions that measured direct effects yielded higher effect sizes. The direct effect of peer tutoring on behavioral and social outcomes was moderately large (ES 0.75), whereas the collateral effect was relatively small (ES 0.43). Furthermore, peer tutoring had a greater effect on improving social skills and social interactions (ES 0.69) and reducing disruptive and off-task behaviors (ES 0.60) than academic engagement (ES 0.38).
Article
Maltreated young persons in out-of-home care often have poor educational outcomes, heightening their risk of long-term psychosocial disadvantage (Forsman, Brännström, Vinnerljung, & Hjern, 2016). In their systematic reviews, Romano, Babchishin, Marquis, and Fréchette (2014) and O’Higgins, Sebba, and Gardner (in press) found evidence that neglect was more often linked with low academic achievement, whereas abuse was more likely to be associated with behavioral difficulties. In large samples of young persons in out-of-home care in Ontario, Canada, who had experienced mainly neglect, we investigated risk and protective factors as predictors of educational success. In a cross-sectional hierarchical regression analysis (N = 3659, aged 11–17 years), female gender, youth educational aspirations, caregiver educational aspirations for youth, time with current caregiver, internal developmental assets, and positive mental health were associated with better educational success. Neglect, grade retention, special educational needs, ethnic minority status, behavioral problems, and soft-drug use were associated with poorer educational outcomes. Gender significantly moderated caregiver educational aspirations and youth placement type. In a longitudinal analysis of a subsample (N = 962, aged 11–15 years at Time 1), covering three years, a large decline in educational success (d = −0.80) was observed. Female gender, internal developmental assets, and positive mental health positively predicted, and soft drug use negatively predicted, greater educational success at Time 2. These results point to factors that help or hinder educational success among young people in care and should inform new interventions or improved versions of existing ones that address educational success in the context of neglect.
Article
Children in care lag behind their peers on a number of outcome measures, including education. Interventions have been developed to help them close the gap with their peers but these have had limited success to date. One possible reason for this may stem from our lack of understanding about underlying processes and mechanisms. This paper presents the findings of a systematic review of the factors associated with educational outcomes for children in foster and kinship care. It aims to inform the literature on risk and protective factors and inform the development of future interventions. Eight major databases and websites were searched between 1990 and 2016 using a combination of mesh terms. Studies were included if they tested the statistical association between any variable and educational outcomes for school age children in foster or kinship care in high-income countries. Children in other placement types were excluded. Titles and abstracts were screened for 7135 studies identified through searches. Full texts were obtained for 298 and 39 were retained for inclusion. Over 70 factors were identified. For the purposes of the narrative synthesis, factors were categorised into spheres of influence adapted from Bronfennbrenner's (1979) ecological framework. The findings reveal significant heterogeneity. Male gender, ethnic minority status and special educational needs were consistent predictors of poor educational outcomes, while carers' and young people's aspirations appeared to predict greater success. The findings are discussed with implications for future research and practice.
Article
The consistently poor educational achievement of children in foster care has been associated with many negative outcomes including long term poor adult adjustment (e.g., higher rates of suicide, criminality, and substance abuse). The current study was undertaken to improve foster children's academic skills through academic remediation. Across this two year randomized study, 91 children in out of home foster or kinship care, between grades 1 and 8 inclusive, completed the study. One-half were randomly assigned to the 30-week direct-instruction small group tutoring condition, while the other half served as wait-list controls. A statistically significant increase in standard scores was found on reading decoding, spelling and mathematic skills for the children who received tutoring, but no differences were obtained on sentence comprehension. Significant effect sizes, in the small to moderate range, were also found in support of the tutoring condition across these three domains. The implications of these positive findings as they relate to improving educational achievement among children in foster care are discussed.
Article
Research has shown that children in foster care are a high-risk group for adverse economic, social and health related outcomes in young adulthood. Children’s poor school performance has been identified as a major risk factor for these poor later life outcomes. Aiming to support the design of effective intervention strategies, this study examines the hypothesized causal effect of foster children’s poor school performance on subsequent psychosocial problems, here conceptualized as economic hardship, illicit drug use, and mental health problems, in young adulthood. Using the potential outcomes approach, longitudinal register data on more than 7,500 Swedish foster children born 1973-1978 were analyzed by means of doubly robust treatment-effect estimators. The results show that poor school performance has a negative impact on later psychosocial problems net of observed background attributes and potential selection on unobservables, suggesting that the estimated effects allow for causal interpretations. Promotion of school performance may thus be a viable intervention path for policymakers and practitioners interested in improving foster children’s overall life chances.
Article
This study investigated effects of a Direct Instruction reading comprehension program implemented with students with autism spectrum disorders (ASD) and developmental disabilities (DD). There is little research in the area of reading comprehension for students with ASD and no research as to the effectiveness of reading comprehension Direct Instruction (DI). This study extended previous research by investigating the extent to which more complex instruction could be implemented with students with ASD and DD and its effect on their reading comprehension. A multiple probe across behaviors design was used. A functional relation between Direct Instruction and reading comprehension skills and behaviors was demonstrated across all behavioral conditions and across students. Data were also collected using curriculum-based assessments and all student demonstrated improvement. Results and their implications are discussed further.