Content uploaded by Charlotte Dignath
Author content
All content in this area was uploaded by Charlotte Dignath on Mar 26, 2018
Content may be subject to copyright.
Review
Effectiveness of learning strategy instruction on academic
performance: A meta-analysis
A.S. Donker
a,
⇑
, H. de Boer
a
, D. Kostons
a
, C.C. Dignath van Ewijk
b
, M.P.C. van der Werf
a
a
Groninger Institute for Educational Research, University of Groningen, Grote Rozenstraat 3, 9712 TG Groningen, The Netherlands
b
Department of Psychology, University of Mannheim, 68131 Mannheim, Germany
article info
Article history:
Received 10 April 2013
Revised 11 November 2013
Accepted 12 November 2013
Available online 21 November 2013
Keywords:
Self-regulated learning
Metacognition
Learning strategies
Academic performance
Meta-analysis
abstract
In this meta-analysis the results of studies on learning strategy instruction focused on
improving self-regulated learning were brought together to determine which specific strat-
egies were the most effective in increasing academic performance. The meta-analysis
included 58 studies in primary and secondary education on interventions aimed at improv-
ing cognitive, metacognitive, and management strategy skills, as well as motivational
aspects and metacognitive knowledge. A total of 95 interventions and 180 effect sizes dem-
onstrated substantial effects in the domains of writing (Hedges’ g= 1.25), science (.73),
mathematics (.66) and comprehensive reading (.36). These domains differed in terms of
which strategies were the most effective in improving academic performance. However,
metacognitive knowledge instruction appeared to be valuable in all of them. Furthermore,
it was found that the effects were higher when self-developed tests were used than in the
case of intervention-independent tests. Finally, no differential effects were observed for
students with different ability levels. To conclude, the authors have listed some implica-
tions of their analysis for the educational practice and made some suggestions for further
research.
Ó2013 Elsevier Ltd. All rights reserved.
Contents
1. Introduction . . . ........................................................................................... 2
1.1. Self-regulated learning and metacognition . . .............................................................. 2
1.2. Learning strategies . . . . . .............................................................................. 2
1.2.1. Cognitive strategies . . . . . . . . ................................................................... 3
1.2.2. Metacognitive Strategies . . . . ................................................................... 3
1.2.3. Management strategies . . . . . ................................................................... 3
1.3. Motivation and metacognitive knowledge. . . .............................................................. 3
1.3.1. Motivational aspects. . . . . . . . ................................................................... 3
1.3.2. Metacognitive knowledge . . . ................................................................... 4
1.4. Instructing learning strategies: findings from earlier meta-analyses. ........................................... 4
1.4.1. Effective strategies . . . . . . . . . ................................................................... 4
1.4.2. Student characteristics . . . . . . ................................................................... 4
1.4.3. Outcome variables and measures . . . . . . . . . . . . . . . . ................................................ 5
1.5. The current study . . . . . . .............................................................................. 5
1747-938X/$ - see front matter Ó2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.edurev.2013.11.002
⇑
Corresponding author. Tel.: +31 503639116.
E-mail address: a.s.donker-bergstra@rug.nl (A.S. Donker).
Educational Research Review 11 (2014) 1–26
Contents lists available at ScienceDirect
Educational Research Review
journal homepage: www.elsevier.com/locate/EDUREV
2. Method . . . . . . . . . . . . . . . . . ................................................................................. 5
2.1. Literature search and eligibility criteria. . . . . . . . . . . . . . . .................................................... 5
2.2. Coding . . . . . . ....................................................................................... 6
2.2.1. Learning strategies ............................................................................ 6
2.2.2. Student characteristics . . . . . . . . . . . . . . . . ......................................................... 7
2.2.3. Outcome measures ............................................................................ 7
2.3. Meta-analysis ....................................................................................... 7
2.3.1. Random and mixed effects models . . . . . . ......................................................... 7
2.3.2. Calculating the effect size and regression coefficient. . . . . . . . . . . ...................................... 7
2.3.3. Method of analysis ............................................................................ 8
3. Results. . . . . . . . . . . . . . . . . . ................................................................................. 8
3.1. Descriptives . . ....................................................................................... 8
3.2. Effective strategies . . . . . . . . . . . . ....................................................................... 8
3.3. Student characteristics . . . . . . . . . ...................................................................... 12
3.4. Outcome measures and effectiveness . . . . . . . . . . . . . . . . ................................................... 13
3.5. Publication bias . . . . . . . . . . . . . . . ...................................................................... 13
4. Conclusion and discussion . . ................................................................................ 14
4.1. Effective strategies . . . . . . . . . . . . ...................................................................... 14
4.2. Student characteristics . . . . . . . . . ...................................................................... 15
4.3. Outcome measures . . . . . . . . . . . . ...................................................................... 16
4.4. Publication bias . . . . . . . . . . . . . . . ...................................................................... 16
4.5. Practical recommendations. . . . . . ...................................................................... 16
4.6. Limitations. . . ...................................................................................... 17
Appendix A. Strategies – categories and examples . . . ............................................................. 18
Appendix B. Key characteristics of the studies included in the meta-analysis. .......................................... 19
References . . . . . . . . . . . . . . ................................................................................ 24
References: Articles included in the Meta-Analysis. ............................................................. 24
1. Introduction
1.1. Self-regulated learning and metacognition
Self-regulated learners are students who are capable of supporting their own learning processes by applying domain-
appropriate learning strategies (e.g., Boekaerts, 1997; Zimmerman, 1990, 1994). Self-regulated learning can be described
as: ‘‘an active, constructive process whereby learners set goals for their learning and then attempt to monitor, regulate
and control their cognition, motivation and behavior, guided and constrained by their goals and the contextual features
in the environment’’ (Pintrich, 2000, p. 453). In short: students who are able to self-regulate their learning are active, respon-
sible learners who act purposefully (i.e. use learning strategies) to achieve their academic goals. To this end, they need meta-
cognitive knowledge; knowledge and awareness about their own cognition (Flavell, 1976, 1979).
The term ‘metacognition’ is not new in this field, and it is sometimes used interchangeably with self-regulation
(Dinsmore, Alexander, & Loughlin, 2008). This is because self-regulation includes the regulation of cognition, which relates
to (cognitive) strategies and metacognition. Whereas metacognition is more narrowly defined and refers only to knowledge
regarding cognition (Dinsmore et al., 2008), self-regulated learning is broader in a sense, as it comprises both the knowledge
and control of not only cognition, but also of motivation.
Students have to acquire knowledge, as it is required to apply learning strategies. Furthermore, in order to become
effective self-regulated learners, they have to practice the actual application of this knowledge. However, becoming a
self-regulated learner is not an end in itself; it is a means to another end, namely to improve academic performance, as it
is demonstrated that self-regulated learners usually do well in education (e.g., Zimmerman, 1990). Research (Dignath,
Büttner, & Langfeldt, 2008; Hattie, Biggs, & Purdie, 1996) has suggested a causal relationship between strategy use and per-
formance: using the proper learning strategies improves academic performance. As not all students spontaneously master
the use of learning strategies and certainly not in the most effective way, students require additional instruction of learning
strategies.
1.2. Learning strategies
Learning strategies are defined as ‘‘processes (or sequences of processes) that, when matched to the requirements of
tasks, facilitate performance’’ (Pressley, Goodchild, Fleet, & Zajchowski, 1989, p.303). Learning strategies have been
repeatedly demonstrated to be positively correlated with academic performance (Alexander, Graham, & Harris, 1998; Hattie
et al., 1996; Weinstein, Husman, & Dierking, 2000). They structure the processing of information by facilitating particular
activities, such as the planning of learning tasks, goal setting, monitoring the progress toward these goals, making
2A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
adjustments if needed, and evaluating the learning process and the outcomes (Boekaerts, 1997). The literature has provided
a large number of strategies, ranging from very basic re-reading approaches to more complex methods of synthesizing
knowledge or drawing conceptual frameworks. These strategies can be categorized in many ways according to various taxo-
nomies and classifications (e.g. Mayer, 2008; Pressley, 2002; Weinstein & Mayer, 1986). In this study the following categories
have been defined: cognitive, metacognitive and management strategies.
1.2.1. Cognitive strategies
Cognitive strategies are used to increase the understanding of a certain domain. They refer directly to the use of the infor-
mation learned and are therefore domain- or even task-specific. Three main subcategories of cognitive strategies can be dis-
tinguished: rehearsal, elaboration and organization strategies (Pintrich, Smith, Garcia, & McKeachie, 1991). Rehearsal
strategies are used to select and encode information in a verbatim manner. Here the focus is on repeating material in order
to facilitate learning or remembering, for example when learning vocabulary or idiom. Elaboration strategies help students
store information into their long-term memory by building internal connections between the items to be learned and al-
ready existing knowledge. Summarizing and paraphrasing are examples of this type of strategy, which is mostly used in
reading. In mathematics an example of the rehearsal strategy is finding similarities between new problems and the ones
solved earlier, using comparable calculations. Lastly, organization strategies help students select appropriate information
by drawing graphs or pictures and establishing connections among the different elements to create meaningful units of
information (Weinstein et al., 2000).
1.2.2. Metacognitive Strategies
Metacognitive strategies regulate students’ cognition by activating relevant cognitive approaches. As metacognitive strat-
egies are linked to cognitive domains, they always involve a particular degree of learning content and can be considered as
higher order strategies. Three subcategories related to the three phases of the learning process can be distinguished: plan-
ning, monitoring and evaluation (Schraw & Dennison, 1994). Planning strategies are deployed at the start of a learning epi-
sode and include subprocesses such as goal setting and allocating resources. Examples of these strategies are making a plan,
deciding upon the amount of time to spend on an activity, and choosing what to do first. Monitoring strategies are used for
checking one’s comprehension. These strategies can be considered as continuous assessments of one’s learning and/or strat-
egy use. Examples include self-questioning and changing the approach to a specific learning task if necessary, for instance,
re-reading a passage if its’ meaning is not properly understood. After the learning process, evaluation strategies can be used
in the analysis of one’s performance and the effectiveness of the learning methods. In writing, for example, reviewing a text
is a strategy that might help improve the written text, while in mathematics it is important to check whether the answers
found make sense in the context of the original problem.
1.2.3. Management strategies
Management strategies are strategies to manage the aspects in the context which directly influence the learning process.
This type of strategy is related to the theoretical framework proposed by Pintrich (2000), which explicitly refers to the con-
textual features that influence learning. Management strategies can be classified into three main subcategories: manage-
ment of effort, management of peers and others (e.g., teachers) and management of the environment. Effort-management
refers to strategies which reflect the commitment to completing one’s study goals, in spite of difficulties or distractions
(Pintrich et al., 1991). It is a form of actively motivating oneself to persist in studying. The second subcategory, management
of peers (or others), includes strategies deducted from theories that reflect a socio-constructive view of learning, in which
peers work together to construct knowledge. Asking fellow students to assist in learning, working together on tasks, as well
as forms of reciprocal teaching can be very effective in enhancing one’s learning and understanding (e.g., Palincsar & Brown,
1984). Finally, management of the environment relates to strategies which help in using the environment to optimize the
possibilities for learning, e.g., by using the library or dictionaries and finding a quiet place to study.
1.3. Motivation and metacognitive knowledge
Apart from focusing on the strategies mentioned above, learning strategy trainings frequently address two related topics,
namely motivation and knowledge. These elements could be considered as a crucial condition for learning strategies if the
aim is specifically to enhance one’s self-regulated learning. As learning strategies are controllable and have to be imple-
mented consciously by an individual (Bjorklund, Dukes, & Brown, 2009), students need to have sufficient knowledge regard-
ing these strategies as well as the motivation to apply them. Therefore, in completing the overview of learning strategies for
self-regulated learning, motivational aspects and metacognition are considered as well.
1.3.1. Motivational aspects
Motivation is a multi-facetted construct which can help students engage in learning in various ways. With respect to aca-
demic performance, there are several aspects that might influence a student’s approach to a task, for instance self-efficacy
beliefs, which refer to one’s perception of one’s ability to accomplish a task and one’s confidence in one’s skills to perform
this task (Pintrich et al., 1991). Furthermore, task-value beliefs concern the extent to which students perceive academic tasks
as interesting and important. Finally, goal orientation relates to the reasons why students perform a task, which can be either
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 3
intrinsic (e.g., curiosity) or extrinsic (e.g., rewards) (Pintrich et al., 1991). All these motivational aspects play a role in stu-
dents’ decisions whether to engage in or refrain from strategy use (e.g., Garner, 1990; Hadwin & Winne, 1996).
1.3.2. Metacognitive knowledge
In summarizing conditions for effective strategy trainings, Hattie et al. (1996) pointed to the relevance of contextual as
well as strategic knowledge about learning tasks. Furthermore, Dignath et al. (2008) found metacognitive knowledge, or
reflection, to be an important element in strategy trainings. It is therefore a component commonly used in trainings, and
considered as a relevant factor in the interpretation of effects. Furthermore, it comprises declarative, procedural and condi-
tional elements: knowledge of how, when and why to use which learning strategies (Schraw & Dennison, 1994).
1.4. Instructing learning strategies: findings from earlier meta-analyses
Over the years, the number of learning strategy interventions conducted has become considerable. All of them have made
a relevant contribution to the growing insight into the effectiveness of learning strategy instruction and strategy application.
From the nineties on, important meta-analyses have been conducted in order to synthesize the findings available so far. The
first meta-analysis was conducted by Hattie et al. (1996). This analysis included 51 studies, resulting in an overall effect size
on student performance of Cohen’s d= .57 (SE = .04). The authors reported the highest effects for the direct teaching of cog-
nitive skills. These effects were mostly produced by interventions aimed at the near transfer of a specific task-related skill.
Multiple-component interventions, in which various strategies were addressed, revealed lower impacts. Regarding student
characteristics, it was reported that in general low ability students seemed unable to profit from these interventions. How-
ever, in this and other studies several issues remained unaddressed. For example, in the analysis of Hattie et al. (1996) the
trainings took place outside the regular school context and were focused on multiple independent variables aimed at
increasing various kinds of performance stretching beyond achievements in the learning of content. Furthermore, in many
analyses the mean effects were computed based on student performance, study skills and affect as a whole. And if a separate
effect for performance was specified, it was not reported whether it was statistically significant or not.
Dignath and colleagues (Dignath & Büttner, 2008; Dignath et al., 2008) conducted two follow-up meta-analyses. They
synthesized information from studies from 1992 to 2006 and simultaneously tested the effects of a number of study
characteristics on academic performance via stepwise backwards metaregression (Dignath & Büttner, 2008). They reported
overall effect sizes on performance of Cohen’s d= 0.61 (0.05) for primary schools and 0.54 (0.11) for secondary schools. For
both school types, the effect sizes were found to be higher when metacognitive reflection was included in the trainings. With
respect to the effects of other strategies on academic achievement, the results were mixed. Taking a closer look at which
learning strategies were the most effective in primary education, Dignath et al. (2008) – using ANOVA – reported the highest
effects for interventions which combined the instruction of different types of strategies. The authors argued that the train-
ings should include both metacognitive and motivational strategies. In secondary education the highest effects were indi-
cated for interventions focused on motivation and/or metacognitive reflection. Furthermore, it was observed that group
work had a negative effect in primary education but a positive impact in secondary education. Regarding school subject, con-
tradictory results were found. In primary school, interventions in mathematics were more effective than those in reading or
writing, whereas in secondary school the opposite was true. The authors did not address students’ ability, as only studies
conducted in the regular education segment were included in the literature review.
Again, additional questions arose. For example, the effects largely varied between interventions in mathematics and those
in reading and writing (which were combined in these analyses). Furthermore, instructing all types of strategies would be
overwhelming for the students, so where to focus on as a teacher? And finally, earlier meta-analyses did not clarify the nat-
ure of the tests used in the studies analyzed, for example whether they were independent assessments or tests specifically
developed for the study, a difference which might have influenced the effect sizes reported. In sum, although the previous
meta-analyses provided insight into the potential effectiveness of strategy instruction, their results also gave rise to new
questions, especially with respect to the application of the research findings in practice. Three issues are of interest here:
which specific strategies are effective, which students profit from strategy instruction and what influence do the types of
test instruments have on the effect sizes found?
1.4.1. Effective strategies
Probably the most relevant question is which specific strategies are the most effective in improving student learning. Ear-
lier meta-analyses have investigated the effectiveness of a broad spectrum of strategy trainings, including cognitive, meta-
cognitive, and motivational learning strategies (the latter we call motivational aspects). However, in planning and
implementing interventions in the curriculum, it is useful to know quite specifically which concrete strategies (included
in the various broad categories) should be taught to make students’ learning more effective. And as earlier meta-analyses
have found differences among the various subjects in this respect, another question arises, namely how the effectiveness
of strategies is influenced by the subject-domains in which these strategies are implemented.
1.4.2. Student characteristics
There has been an ongoing debate about the age at which learners are capable of self-regulating their learning. Some
researchers claim that children are not yet capable to engage in metacognitive activity because it requires a particular level
4A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
of cognition, accurate knowledge of academic tasks and learning, and the ability to monitor oneself. And, as argued by some,
these are elements which are not yet fully developed at such a young age (e.g., Paris & Newman, 1990). Others see metacog-
nitive activity and self-regulated learning initiatives in children already as early as in Kindergarten (e.g., Whitebread et al.,
2009). These different viewpoints are partly dependent on the researchers’ theoretical backgrounds and what they consider
to be metacognition and self-regulated learning. There is general consensus, however, on the view that both concepts devel-
op as children mature (e.g., Veenman & Spaans, 2005).
The debate about the prerequisites for students to engage in strategy use is not only limited to their age. Other variables,
such as background and capability, and the way in which these elements influence the effect of learning strategies, also play
a role. For example, according to Hattie et al. (1996), low achievers seem unable to benefit from most types of interventions.
However, considering that new models of self-regulated learning and strategy instruction have emerged since these authors’
meta-analysis was conducted, it seems interesting to re-assess the effects of these types of student characteristics.
1.4.3. Outcome variables and measures
Another issue not addressed thoroughly in earlier meta-analyses relates to the outcome measures used in the original
studies to determine the effectiveness of the interventions. Most studies only use self-developed measures to evaluate stu-
dent performance. This situation could cause effect size inflation, as researchers may direct student performance toward the
test used at the end of the intervention. The question then arises whether these effects would also have been found when
intervention independent tests had been used. This issue was addressed in a meta-analysis of student performance per-
formed by Haller, Child, and Walberg (1988), who reported that there was no difference between the effects found via
self-developed versus intervention independent tests, although it was not clear on which analysis this conclusion was based.
In a study on the effects of metacognitive instruction interventions on comprehensive reading, Chiu (1998) also examined
whether the type of measurement instrument matters. He found that the effects were higher when the test was nonstan-
dardized (Cohen’s d= .61) than when it was standardized (Cohen’s d= .24).
The results of the meta-analyses by Hattie et al. (1996) and Dignath et al. (2008) may have been influenced by the fact
that they did not correct for the type of test instrument used in the primary studies. Considering the common use of self-
developed tests in strategy interventions, it would therefore be interesting to research whether the training effect outcomes
indicated by these instruments actually differ from those yielded by independent achievement assessments and how this
difference might have influenced the interpretation of the results.
1.5. The current study
This article addresses a number of research questions, using a meta-analytical approach. The first one is: Which strategies
instructed are the most effective in improving academic performance? In answering this question, we include three types of
strategies: cognitive, metacognitive and management strategies (and their respective substrategies), while also considering
the effects of motivational aspects and metacognitive knowledge. In doing so we elaborate on earlier findings, taking a closer
look at these broad categories to investigate which concrete substrategies are the most effective. Furthermore, we distin-
guish among subject domains to look for differential effects. Secondly, our focus is on student characteristics, which forms
the basis for our second research question: Do the effects of the strategies instructed differ for different types of students?
Next, we address the possible influence of the measurement instruments used to evaluate the strategies’ effectiveness. We
expect the highest effect sizes for studies based on self-developed tests. To check this hypothesis, we formulate our third
research question: does the type of measurement instrument used in the interventions influence the effect sizes reported?
Finally, we assess whether publication bias affects our results. Several sources of evidence have shown that studies which
present relatively high effect sizes have in general more chance of being published than accounts of lower effect sizes
(Ahn, Ames, & Myers, 2012; Borenstein, Hedges, Higgins, & Rothstein, 2009). This bias is reflected in a meta-analysis. In
our research, however, we checked for publication bias using a statistical method. via statistical means insight can be gained
into the extent and effects of possible publication bias. This emphasis brings us to our last research question: To what degree
are results influenced by publication bias?
2. Method
In order to be able to conduct our analysis, first the relevant literature had to be located and coded. Before explaining our
methods of analysis we will describe how we selected the literature and specify the content in which we are interested.
2.1. Literature search and eligibility criteria
We conducted our literature search on the basis of a series of steps by which we eventually narrowed down the studies
found to our final sample. The first step was a literature search using the internet databases ERIC and PsychInfo. We chose a
limited time span; from January 1st 2000 to January 1st 2012. We decided to take the year 2000 as starting point, as in that
year the Handbook of Self-Regulation by Boekaerts, Pintrich, and Zeidner (2000) was published, which marked a new era of
research in this field. The search terms we used were ‘metacognit⁄’ and ‘self-reg⁄’ which had to be part of the title of the
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 5
articles. Another option would have been to enter all possible learning strategies as search terms. However, we did not know
all of the terms used for the learning strategies by all of the authors of the articles. Rather than potentially overlooking arti-
cles by choosing search terms which were too specific, we decided to enter the aforementioned broad terms because they
sufficiently reflected the educational context in which strategy instruction generally takes place. Subsequently, we used fur-
ther search criteria to narrow down our literature to the topic of learning strategies. As advanced search options we limited
ourselves to articles written in English and published in peer-reviewed journals in the field of (research in) education. This
first search resulted in a total of over 1000 articles.
In the second step of our selection procedure, this number was decreased by more than half because many studies found
were conducted outside the school context and did not include academic performance (e.g., studies on self-regulation in psy-
chiatric patients). The third step was the application of the following main selection criterion: interventions in which learn-
ing strategies are instructed or trained using outcome measures which include academic performance. So, we included only
empirical studies of learning strategies instructed with the aim of improving academic achievement. This approach meant
that we selected articles in which ‘academic achievement’ (operationalized as performance in one or more school subject-
domains) was the dependent variable. Correlational studies were excluded as these do not prove causality. After this step the
total number of studies was significantly decreased.
So far, our choices in capturing our scope of interest had been based on rather broad criteria. In our last step we therefore
used the following eligibility criteria to complete our selection procedure: the studies had to have a control group, as for
study designs without a control group it is not clear whether the results of the experimental group can be ascribed to the
intervention or to natural developments. Furthermore, studies which only provided posttest scores were only included if
it was indicated that the groups had been comparable before the actual intervention had started.
To ensure that the Cohen’s deffect size was approximately normally distributed, the samples used in the studies had to
include at least ten students per group (Hedges & Olkin 1985). Studies based on smaller groups were therefore excluded from
our meta-analysis. For studies which did not provide sufficient descriptions of the exact type(s) of intervention(s), it was not
possible to identify and code the learning strategies. So we also excluded these publications from our analysis.
Next, with regard to the school-subject domains, only the core academic subjects were included. This criterion meant that
music, arts and physical education were excluded. Although learning strategies are also applied in these domains, the re-
search conducted here is more rare compared to that focused on subjects with more cognitive content. In practice, the major-
ity of the studies focused on comprehensive reading, writing texts, mathematics and science. Studies on the remaining
subjects were clustered under the heading ‘other’ in our analysis. To be able to generalize our results, the participants in
the studies had to be primary school or secondary students up to and including the twelfth grade, in line with the American
and most European School Systems. Furthermore, they had to be representative of the complete school community. There-
fore, we also included studies containing samples of children with learning difficulties or disabilities in our sample.
Ideally, it was our objective to perform a meta-analysis of studies in which the effect of the interventions on academic
achievement had been examined via tests independent of the intervention. Such tests are for example standardized achieve-
ment tests. However, because in practice so many studies use self-developed tests for estimating intervention effects, we
decided to include them as well. Finally, the publications had to provide sufficient quantitative measures to calculate an ef-
fect size. Those which did not were excluded from the meta-analysis. After applying all of these criteria, the original sample
was reduced to 58 studies.
2.2. Coding
Following Lipsey and Wilson (2000), we developed a coding scheme containing both statistical and theoretical compo-
nents. This scheme was tested and refined until the authors reached agreement on the topics and corresponding categories.
Next, two of the authors coded five articles together and further discussed the outcomes until full agreement was obtained.
Finally, the same two authors coded ten articles independently, after which the interrater reliability was determined. This
reliability was calculated based on the percentage of agreement reached, which was 96%. Next, the remaining articles were
divided between the same two authors to be coded independently. If necessary, particular issues or concerns were addition-
ally discussed. The coding scheme was based on the following components: the learning strategies instructed, student char-
acteristics and the measurement instrument used.
2.2.1. Learning strategies
For all learning strategies, we coded whether they were (1) or were not (0) included in the intervention. In coding the
strategies, the same classification as described earlier was followed. For the sake of comparability we chose not to label
the strategies on a micro-level (e.g., activating the prior knowledge of a topic on which a text in a comprehensive reading
lesson is based) but to classify them on a higher level (in this case elaboration). Appendix A provides an overview of our cat-
egorization with examples of each type of strategy used in the studies included in our meta-analysis.
Strategies can be described on several levels. All the aforementioned categories have been broadly defined and are do-
main-independent. A strategy such as ‘planning’ can be referred to in general terms as ‘making a plan’ to structure, for in-
stance, the order in which homework is conducted and the amount of time available for each assignment. On this level the
description of strategy use applies to all domain and learning tasks. On a lower level, however, the execution of strategies
differs among domains. For example, in mathematics planning might include extracting the right numbers and calculations
6A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
needed to solve the problem, whereas in reading it might refer to predicting what the story will be about and estimating the
time needed to read the text.
Aligned to the strategies we also included whether or not the motivational aspects were part of the intervention. Finally,
as some studies were more focused on stimulating students’ metacognitive knowledge than on teaching of specific strate-
gies, a category of metacognitive knowledge was added, with as subcategories personal metacognitive knowledge (e.g., feed-
back specifically targeted at the student’s work, including metacognitive hints) and general metacognitive knowledge (e.g.,
an introduction into when, why and how to use learning strategies).
2.2.2. Student characteristics
As we wanted to be able to differentiate among student populations, we also divided the students into categories, using
the following labels: (1) regular population (students who were representative of the student population in the country in
which the study was conducted), (2) children from a low socioeconomic status and background (low SES), (3) students with
learning disabilities and special needs, and (4) gifted and high SES students (although we realized that students who are
gifted do not necessarily come from high SES families and vice versa; due to the low number of studies on either one of these
categories we decided to combine them).
2.2.3. Outcome measures
With regard to the outcome measures, we only coded performance tests used to determine academic achievement. If
multiple tests were used to determine performance we coded all of them, whereas, for example, questionnaires to measure
student motivation were not coded because we were only interested in effects on academic performance. With respect to
these outcomes, we coded the domain in which the performance was measured and the nature of the instrument; self-devel-
oped (designed by the researchers who also evaluated the performance in the intervention), or independent of the interven-
tion/standardized (e.g., tests used in comparable studies, or more general tests such as national exams).
2.3. Meta-analysis
To calculate the effect sizes, we used the software package Comprehensive Meta Analysis, developed by BioStat (see
www.meta-analysis.com). Here we had to decide which data we needed and how we wanted to analyze them. Before pre-
senting our results we will give a brief explanation of the choices made in the analysis of our data.
2.3.1. Random and mixed effects models
We ran our first analysis using a random model which assumes random effects among the studies. Because the studies
differed in terms of participants and interventions, it was likely that the effect sizes also differed (Borenstein et al., 2009). Our
goal was to estimate the mean of the distribution of these effects (i.e., the influence of the populations in the studies on the
effects of the learning strategies, of which we had a sample). This was done by assigning weights to the studies, based on the
variances within the samples (as opposed to their size). In the next step, when focusing on the different categories or sub-
groups, we used mixed effects models. This approach allows for random between-study effects and a fixed moderator effect
(Borenstein et al., 2009), that is, variability in the effect size distribution attributed to systematic between-study differences
(e.g., type of intervention), which could be modeled, and subject-level sampling error plus an additional random component
(Lipsey & Wilson, 2000).
2.3.2. Calculating the effect size and regression coefficient
The effect size used in this analysis was Hedges’ g: the corrected standardized mean difference between two groups,
based on the pooled, weighted standard deviation (Ellis, 2010, p. 13). We chose this effect size to account for the different
sample sizes in the studies included in our meta-analysis. The following formula is used to calculate Hedges’
g:g¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
½ðn1þn22=n1þn2Þd
pwhere drepresents Cohen’s d(the standardized mean difference, that is, the difference
between two means divided by the pooled standard deviation). If the primary studies only reported an overall sample size
but did not provide clear divisions into treatment groups, the total sample size was equally divided over the number of
groups included in the studies. To prevent outliers which would influence our results in an unrepresentative way, we
adjusted them by Windsorizing (Lipsey & Wilson, 2000). Outliers were recorded to the general, unweighted mean of the
effect sizes (.89) plus or minus two times the standard deviations (.87).
Many studies included multiple effect sizes. However, we only calculated those with our variables of interest, namely aca-
demic outcome variables. These outcomes were frequently assessed using multiple tests. In our descriptive data analyses
these assessments led to multiple effect sizes within one study (for example, the study of Bruce and Robinson (2001) re-
ported four effect sizes as index effects on students’ reading ability). Using the CMA software these effect sizes are averaged
to obtain one representative effect size per study.
In some studies multiple experimental groups were compared to one control group. In these cases the statistical depen-
dence of data comes into play (Lipsey & Wilson, 2000). If this dependency is not accounted for, the weight assigned to the
experimental groups would be too high. To correct for this problem, the number of students in the control group was divided
by the number of experimental groups. For example, if a study examined the effects of two experimental groups and
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 7
compared the results to a control group of 60 students, we used the same mean and standard deviation of the control group
test scores but filled in 30 as sample size. This correction resulted in a higher variance and thus a lower study weight.
2.3.3. Method of analysis
The CMA software makes it possible to perform meta-analyses and examine the influence of a single moderator on the
summary effect. This moderator can be either a variable measured on an interval scale (such as age), or a categorical variable
with multiple categories (for example ethnicity). In the case of a categorical variable, CMA executes an analysis of variance,
adapted to meta-analytical data. CMA also has the option to check for possible publication bias.
For additional analyses, we used the statistical package ‘Hierarchical Linear Modeling’ (HLM), version 6, of Raudenbush,
Bryk, and Congdon (2004) because of its option to perform meta-regressions with multiple predictors. A meta-regression is a
regression-analysis in which the predictors, or moderators, are at the level of the intervention while the dependent variable
is the effect size in the intervention. This analysis method was used to simultaneously test the effects of multiple learning
strategies on the summary effect.
In the studies of our sample the effects of an intervention on academic performance were often measured via multiple
tests. CMA automatically calculates the mean of these outcomes into one representative effect size. However, when correct-
ing for effects related to the measurement instrument we wanted to use all effect size measures separately. To this end we
applied HLM, which made it possible to correct for the effects related to the measurement instrument while regressing the
multiple predictors of the effect size.
To be able to load all effect sizes separately into HLM, we had to adjust their weights because otherwise interventions
containing multiple tests would be given a larger weight than those based on only one test. To adjust these weights, we di-
vided them by the number of tests through which the effectiveness of the intervention was measured.
3. Results
3.1. Descriptives
A total of 58 articles including 95 strategy-interventions met our eligibility criteria and were included in our analysis. The
majority of the interventions took place in the context of mathematics (n= 44), followed by (comprehensive) reading,
writing and science (n= 23, n= 16, and n= 9, respectively). In total, 180 effect sizes were coded, which indicates that many
interventions had been evaluated using multiple tests. Table 1 presents a summary of the study characteristics. A table with
the key characteristics of each individual study can be found in Appendix B.
The middle column of the table shows the frequency of the characteristics as included in our analyses. The strategies most
frequently addressed in the interventions were metacognitive approaches with a focus on ‘planning’ and ‘monitoring’. With
regard to cognitive strategies, elaboration was by far the most frequently trained substrategy. Management strategies were
addressed somewhat less while motivational aspects were referred to the least. In addition to the metacognitive strategies,
metacognitive knowledge was also addressed explicitly in about half of the trainings. This knowledge mostly dealt with the
‘when’ and ‘why’ of using learning strategies and was in some cases specifically tailored to the individual students. Allen and
Hancock (2008), for example, provided students with information on their ‘cognitive profiles’ and how they related to their
reading comprehension. Their students were given instruction on how to use their strengths to compensate for their weak-
nesses. In this case, students were individually provided with personal metacognitive knowledge.
Most studies were conducted in schools with regular students, followed by interventions for students with special needs.
The instruments used to measure the effects of the strategy instructions were for the most part self-developed and aligned to
the interventions. Nevertheless, a few studies used existing tests to measure the effectiveness of the interventions, such as
the Test of Science Knowledge (Michalsky, Mevarech, & Haibi, 2009), a domain-specific test designed by the National Science
Committee, and the Graded Word Reading Test (in the study of Wright & Jacobs, 2003), widely used in the United Kingdom.
Even in these studies, however, the measures were completed by additionally assessing the students via a self-developed
test.
The column on the right presents the mean effect sizes (Hedges’ g). These are the effects of strategy interventions in a
certain domain, aimed at a certain type of students or including a specific substrategy. To calculate these effects, all inter-
ventions that met the criterion were analyzed together. The table gives us an indication of the overall effectiveness of learn-
ing strategy interventions aimed at improving academic performance. The mean weighted effect size of Hedges’ g= .66
(SE = .05, with a confidence interval of Hedges’ gfrom .56 to .76: a significant effect) further strengthens the finding that
interventions are in general effective. In the following analyses the effectiveness of learning strategies is investigated in more
detail.
3.2. Effective strategies
We first tested the effects of each learning strategy on student performance separately in a meta-regression model with
the measurement instrument as covariate. This covariate was included to account for the differences in effect sizes caused by
the measurement instruments. Table 2 shows the results.
8A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
In presenting the regression coefficients we show the differences between the interventions which did and did not in-
clude the strategy under study; the effects presented are those found when the strategy was addressed compared to the
interventions which did not include the strategy. So, for example, interventions in which the strategy ‘planning’ was in-
cluded were compared with other learning strategy interventions, to test its effect. Planning shows a regression coefficient
of .20, which means that the effect size of interventions including planning (whether or not in combination with other strat-
egies) is .20 higher than of those focused on other strategies. The analyses show significant positive effects of the inclusion of
Table 1
Characteristics of the intervention.
Variable nInterventions Effect size
Strategies
Cognitive strategies
Rehearsal 10 1.39
Elaboration 50 .75
Organization 32 .81
Metacognitive strategies
Planning 68 .80
Monitoring 81 .71
Evaluation 54 .75
Management strategies
Effort 15 .77
Peers 21 .83
Environment 6 .59
Motivational aspects
Self-efficacy 13 .72
Task value 6 1.84
Goal orientation 6 .46
Metacognitive knowledge
General 35 .97
Personal 13 .94
Student characteristics
Regular 67 .61
Low SES and ethnic minority 7 .72
Special needs 14 .89
Gifted and high SES 7 .72
Measurement instruments
Self-developed 122 effect sizes .78
Intervention independent 50 effect sizes .45
Unknown 8 effect sizes
School subject
Reading 23 .36
Writing 16 1.25
Mathematics 44 .66
Science 9 .73
Other 3 .23
Table 2
Effect of each learning strategy on student performance: meta-regression results.
B(SE)
Cognitive strategy rehearsal .42 (.15)
⁄⁄
Cognitive strategy elaboration .14 (.09)
Cognitive strategy organization .09 (.09)
Metacognitive strategy planning .20 (.09)
⁄
Metacognitive strategy monitoring .07 (.12)
Metacognitive strategy evaluation .06 (.08)
Management strategy effort .02 (.13)
Management strategy environment .03 (.15)
Management strategy peers/others .03 (.10)
Motivational aspect self-efficacy .10 (.13)
Motivational aspect task value .94 (.21)
⁄⁄
Motivational aspect goal orientation .35 (.16)
⁄
Metacognitive knowledge (personal) .04 (.12)
Metacognitive knowledge (general) .31 (.08)
⁄⁄
Notes:
⁄⁄
p< .01;
⁄
p< .05. Reference category of the learning strategies: ‘strategy not in intervention’.
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 9
general metacognitive knowledge, the learning strategies planning and rehearsal, and the motivational aspect task value.
Table 2 further indicates that the inclusion of the motivational aspect task value has the largest positive influence on the
effectiveness of the intervention. Lastly, the coefficients indicate that the inclusion of ‘goal orientation’ has a negative influ-
ence on the intervention’s effectiveness. Again, this finding does not mean that interventions focused on goal orientation
have negative effects on student performance: goal orientation has a positive influence on performance, yet this influence
is significantly lower than the influence of any of the other strategies.
Next, the effects of the significant learning strategies on student performance were analyzed simultaneously. Again, the
measurement instrument served as a covariate in the meta-regression. Table 3 shows the results of this analysis.
The results show that the effect of the cognitive strategy ‘rehearsal’ is no longer significant, whereas the impacts of gen-
eral metacognitive knowledge, planning and task value still are. This finding indicates that interventions which include ‘gen-
eral metacognitive knowledge’, ‘planning’ or ‘task value’ enhance student performance the most effectively. The effect of the
inclusion of goal orientation in the interventions remained negative. For both motivational aspects, however, the results are
based on only a few studies which included these items, which is why this result should be interpreted with care. Together,
the significant strategies accounted for 36.1% of the variance in effect size.
In order to verify our findings, we also ran a regression analysis with backwards elimination. Although this analysis was
not suitable for taking a closer look at the different domains (because for this purpose we would have needed more inter-
ventions in the separate domains), we could use it to obtain an overall picture of the effective strategies. To this end, we
included all strategies in the model and eliminated the least significant ones until a model with only significant strategies
remained. The strategies in this final model generally matched the ones we had found in the previous analysis (general meta-
cognitive knowledge and task value). However, planning was no longer significant, whereas elaboration now belonged to the
significant strategies.
In the next step, we distinguished among the different subject domains, as they might differ in terms of the effectiveness
of the strategies used. Figs. 1–4 show all effect sizes found in the primary studies per subject domain, as displayed in forest
plots.
Table 3
Meta-regression of multiple learning strategies related to student performance.
B(SE)
Intercept .29 (.08)
⁄⁄
Measurement instrument self-developed .20 (.08)
⁄
Cognitive strategy rehearsal .01 (.16)
Metacognitive knowledge general .25 (.08)
⁄⁄
Metacognitive strategy planning and prediction .17 (.08)
⁄
Motivational aspect task value .81 (.23)
⁄⁄
Motivational aspect goal orientation .33 (.14)
⁄
Notes:
⁄⁄
p< .01;
⁄
p< .05. Reference categories of the measurement instruments and the
learning strategies: ‘intervention independent test’ and ‘strategy not in intervention’,
respectively.
Fig. 1. Forest plot for comprehensive reading studies. Forest plot of average effect size and 95%-confidence interval of each of the comprehensive reading
interventions (represented by a square) and summary effect (represented by a diamond).
10 A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
The forest plots show only four studies in the domain of comprehensive reading with effect sizes below or around zero;
all other interventions have positive and sometimes quite high effect sizes. The strategy interventions yielded the highest
effects within the domain of writing, although the differences are large here. Within the separate domains we looked again
at the effectiveness of specific substrategies. Table 4 reports the contribution of substrategies instructed in the various do-
mains to student performance. Again the measurement instrument was used as a covariate to correct for possible influences.
Fig. 2. Forest plot for writing studies. Forest plot of average effect size and 95%-confidence interval of each of the writing interventions (represented by a
square) and summary effect (represented by a diamond).
Fig. 3. Forest plot for mathematics studies. Forest plot of average effect size and 95%-confidence interval of each of the mathematics interventions
(represented by a square) and summary effect (represented by a diamond).
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 11
In reading, (general) metacognitive knowledge significantly improved student performance. However, the substrategies
elaboration and management of peers yielded lower levels of effectiveness. So with respect to interventions in the domain of
reading, the inclusion of metacognitive knowledge was beneficial, whereas a focus on elaboration and management of peers
(i.e., collaboration) appeared to be less so. Interventions based on the substrategies planning and effort management had
slightly favorable results, yet compared to other strategies their effects were non-significant. Also rehearsal, management
of the environment and self-efficacy showed (non-significant, yet) positive effects. However, these were based on only a
small number of interventions, so again this result should be interpreted with caution. Although in writing the effects were
more salient than in other domains, only evaluation or (general) metacognitive knowledge clearly proved to be more
beneficial than other strategies. Here no significant negative effects were found. In mathematics elaboration was the only
substrategy which improved student performance significantly more than other methods. It may therefore be worth incor-
porating in any intervention within this domain.
In reading and mathematics, only one type of substrategy showed significant positive results, while in science no signif-
icant effects were found at all. In the domain of writing however, two substrategies appeared more effective than other ones.
We analyzed these substrategies together to test the effectiveness of this combination. In writing the combination of
(general) metacognitive knowledge and evaluation was B= .75 (.24) and .57 (.26), respectively (p< .05), indicating that inter-
ventions which included both strategies were the most effective here.
3.3. Student characteristics
With respect to our second research question about student characteristics, we investigated whether the effectiveness of
the strategy interventions depended on the type of students. Table 5 shows the average effect sizes of the interventions for the
different categories of the ‘student characteristics’ predictor. The meta-analysis of variance revealed no significant between-
groups differences; the effect sizes show that the learning strategy interventions were highly effective for all types of students.
With the aid of meta-regression, we examined the between-groups differences more thoroughly. In this way, we could
compare two groups with each other instead of analyzing the between-groups differences as a total. Furthermore, with this
analysis the possible effects of the type of measurement instrument were controlled for as the instrument was included as a
covariate. The last column of Table 5 shows the results. Although it appears that special needs students benefited more from
the interventions, we did not find significant differences (B= .23, p= .58).
Fig. 4. Forest plot for science studies. Forest plot of average effect size and 95%-confidence interval of each of the science interventions (represented by a
square) and summary effect (represented by a diamond).
Table 4
Effectiveness of learning strategies: regression coefficients.
Main strategy Sub strategy Reading Writing Math Science
nB (SE) nB (SE) nB (SE) nB (SE)
Cognitive strategies Rehearsal 2 .08 (.21) 7 .37 (.30) 1 .22 (.33)
Elaboration 19 .48 (.15)
⁄⁄
8 .47 (.29) 18 .21 (.10)
⁄
4 .16 (.19)
Organization 11 .07 (.11) 14 .42 (.45) 4 .11 (.20) 1 .02 (.25)
Metacognitive strategies Planning 14 .15 (.11) 13 .38 (.38) 32 .08 (.12) 7 .08 (.18)
Monitoring 22 .29 (.23) 12 .46 (.32) 36 .20 (.14) 8 .07 (.38)
Evaluation 12 .05 (.11) 11 .60 (.30)
⁄
21 .03 (.11) 8 .02 (.25)
Management strategies Effort 3 .07 (.16) 7 .17 (.31) 5 .28 (.15)
Environment 3 .04 (.13) 1 .72 (.54) 2 .25 (.22)
Peers 6 .27 (.09)
⁄⁄
9 .45 (.30) 5 .16 (.17)
Motivational aspects Self-efficacy 3 .10 (.17) 3 .08 (.39) 7 .27 (.14)
Task value 0 6 .43 (.30) 0
Goal orientation 0 2 .72 (.58) 3 .21 (.19)
Metacognitive knowledge Personal 2 .17 (.13) 2 .43 (.47) 6 .16 (.15) 3 .25 (.22)
General 8 .27 (.12)
⁄
9 .78 (.26)
⁄⁄
14 .03 (.11) 3 .15 (.15)
Notes:
⁄⁄
p< .01;
⁄
p< .05. Cells are empty when there are no interventions with or without the strategy under study.
12 A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
There was no relationship between the effects of the interventions on student performance and the students’ age (in
grades). A meta-regression with grade as predictor and measurement instrument as covariate revealed a coefficient of only
B=.01 (SE = .02; p= .55) for grade.
3.4. Outcome measures and effectiveness
Our third research question concerned the instruments used to measure the outcomes of the interventions. We found an
average effect size of Hedges g.78 for the self-developed tests and of .45 for the intervention independent tests. This differ-
ence was significant (p< .01).
There were also some differences among the domains. In reading, the number of intervention-independent tests was
quite high; in 70% of the interventions a standardized measure was used. To math and writing the opposite applied: in only
10% and 11% of the interventions respectively, an independent test was used to calculate the effects. For science the amount
was slightly higher, with 29% of intervention-independent tests. Earlier we had also found lower effects for interventions in
the domain of comprehensive reading, which led us to believe that these results were related to the use of intervention-inde-
pendent tests. To test this premise, we therefore analyzed the effect of the measurement instrument for each subject domain
separately. Table 6 shows the results.
It appeared, however, that comprehensive reading was the only subject in which intervention-independent tests resulted
in a significant lower effect than self-developed tests. For writing, math and science, the difference in effect size between the
two types of tests was not significant.
3.5. Publication bias
The last element we were interested in concerned a problematic issue associated with meta-analysis: publication bias.
Several lines of evidence have shown that studies reporting on relatively high effect sizes have more chance of being pub-
lished than studies describing lower effect sizes (Borenstein et al., 2009). If this had also been the case in this field of re-
search, the bias would have been reflected in our meta-analysis. To test whether bias had indeed occurred, we applied
Duval and Tweedie’s Trim and Fill method (Borenstein et al., 2009; Duval & Tweedie, 2000). We used a random effects model
to estimate if there were any interventions missing in the meta-analysis. Following the trim and fill method, extreme effect
sizes of interventions on the right hand of a funnel are trimmed to obtain a symmetric funnel plot. In this way a new, unbi-
ased estimate of the summary effect size is calculated. Next, the funnel plot is filled again with the trimmed effect sizes of the
interventions and their counterparts on the left hand of the funnel plot (the lacking interventions). Then, a pooled estimate of
the summary effect size is calculated. Fig. 5 shows the funnel plot of the relationship between standard error and the effect
sizes of the interventions in the current meta-analysis. The interventions with a small sample size have in general a larger
standard error and appear at the bottom of the figure. The larger interventions are positioned higher up. Applying this ap-
proach to our meta-analysis, we obtained Fig. 5.
The figure shows that the interventions are quite neatly spread. According to the trim and fill method, there were no stud-
ies missing, suggesting that there was no publication bias. We conducted these analyses for all separate subject domains and
found that only in the case of science publication bias had occurred. Still, inclusion of the missing studies would have re-
sulted in effects in the same direction.
Table 5
Mean effect size for student characteristics and meta-regression.
Mean Hedges’s g (SE) Regression B (SE)
Intercept .40 (.07)
⁄⁄
Instrument self-developed .33 (.09)
⁄⁄
Average/regular .61 (.06)
⁄⁄
Low SES .72 (.18)
⁄⁄
.06 (.15)
Special needs .89 (.14)
⁄⁄
.23 (.12)
High SES/gifted .72 (.18)
⁄⁄
.16 (.17)
Notes:
⁄⁄
p< .01;
⁄
p< .05. The reference categories for measurement instrument and
characteristics are intervention dependent test and regular students, respectively.
Table 6
Effects for separate measurement instruments.
Self-developed Hedges’s g Intervention independent Hedges’s g
Total .78 .45
⁄⁄
Reading .82 .22
⁄⁄
Writing 1.31 1.07
Mathematics .61 .84
Science .63 .88
Notes:
⁄⁄
p< .01;
⁄
p< .05.
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 13
4. Conclusion and discussion
This meta-analysis addressed the question which learning strategies are the most effective in enhancing the academic
performance of students in primary and secondary education. To establish the effects of these strategies, we relied on re-
search studies which describe interventions in which learning strategies are instructed, assuming that the instruction of
these strategies would result in their adoption by the students. A search for literature published in a period of more than
a decade yielded a total of 58 studies, including 95 interventions of strategy instruction, which comprised 180 effect sizes.
The studies’ average mean effect size of Hedges’ g= .66 (S.E. = 0.05) again demonstrated that students’ academic perfor-
mance can indeed be improved by the instruction of learning strategies.
Three types of learning strategies were included in this meta-analysis: cognitive, metacognitive, and management strat-
egies, and their related motivational aspects and metacognitive knowledge. Of the interventions assessed, metacognitive
strategies were the most commonly used, with a focus on planning and monitoring. With regard to cognitive strategies, elab-
oration was by far the most frequently trained approach. Management strategies were used to a somewhat lesser extent
while motivational aspects were addressed the least. These results differ from the findings of Dignath et al. (2008), who
found many more motivational strategies (included in our category ‘motivational aspects’). This difference might be due
to a different focus; whereas we concentrated on improving academic performance through the use of learning strategies,
Dignath et al. (2008) emphasized the improvement of self-regulated learning, involving metacognition, cognition and moti-
vation. The meta-analysis of Hattie et al. (1996) particularly dealt with the thrust of the programs. The results of this analysis
are more in line with our findings, as the majority of these studies focused on study skills, whereas aspects such as motiva-
tion and attribution received much less attention.
In the interventions in our analysis, performance was almost always improved by a combination of strategies. Further-
more, certain subject domains appeared to be more suitable for strategy instruction than other course fields: in writing
the highest effects were found, regardless of the exact content of the intervention (i.e., the strategies taught). We have no
definite explanation for this finding. However, in writing education there is generally less explicit focus on (strategy) instruc-
tion, which implies that any increase in the emphasis on this approach may already have an effect. Dignath and Büttner
(2008) established the highest effect sizes in mathematics, but this finding might have been the result of the fact that they
considered reading and writing as one domain, whereas we addressed them as separate subjects because the performance
tasks associated with them require different approaches. In the domain of comprehensive reading, a mean weighted effect
size of .36 was found, a considerably lower result than the effect size reported by Chiu (1998), which was .67. However, due
to the different approaches used in analyzing the studies included in these meta-analyses, these findings cannot be com-
pared on a one to one basis.
4.1. Effective strategies
Although we identified many different strategies that contributed to performance, we were unable to present a broad
spectrum of significant findings. This does not mean that strategies are ineffective, on the contrary, all of them clearly
Fig. 5. Funnel plot of standard error by effect size for all interventions. Note: The observed interventions are represented by an open circle; imputed
interventions would have been represented by a filled circle. The open diamond at the bottom represents our mean effect size, the filled diamond represents
the mean effect size based on the total number of interventions (imputed interventions included).
14 A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
proved to be effective. A few aspects do stand out however. First of all, the metacognitive knowledge component adds to
academic performance. By including this component in the intervention, students are not only taught which strategies to
use and how to apply them (declarative knowledge) but also when and why to use them (procedural and conditional
knowledge). This type of informed strategy instruction leads to a significantly larger degree of metacognitive engagement
and is therefore more effective than a plain instruction of the application of learning strategies. This conclusion supports
earlier findings from Dignath and Büttner (2008), who observed higher results each time metacognitive reflection, as they
referred to it, was included in the training. Secondly, planning and task value have generally appeared to be effective,
which is possibly a reflection of the role played by motivation in these types of interventions. Although Hattie et al.
(1996) reported interventions with a focus on motivation to be less successful than other interventions, this finding was
based on only one study. In general, in the context of self-regulated learning, the need to engage students in strategy
use or learning by motivating them is widely acknowledged (see for example Weinstein, Acee, & Jung, 2011).
Double-checking these strategies by regression analysis with backwards elimination also revealed elaboration as a signif-
icant strategy, whereas planning no longer appeared to be significant. Although our regular regression analysis provided
more information on the strategies’ domain-specific effects, which is why it was given preference in addressing the re-
search questions, leaving out the finding that elaboration is generally also a very effective strategy would have been an
omission.
In our analysis some strategies were given a negative B-value, for example goal orientation. This means that although the
strategy is helpful, it is clearly less so compared to other strategies. However, because the results for goal orientation were
based on only a few studies, this rating should be interpreted with care. It would be interesting to see if future meta-analyses
which include a larger number of studies focused on goal orientation also yield this result. A second explanation for negative
outcomes, also when more studies were included – as was the case for elaboration or working with peers in the domain of
comprehensive reading – could be related to the complexity of the strategy. Perhaps these strategies are perceived as more
complicated, and perhaps the more complex strategies may be more difficult to train. It may require time before students
know how to use these approaches properly and before their effects on performance outcomes become really visible. In this
case, the training of complex strategies may have produced lower effects than that of less multifaceted approaches. And
although some of them were negatively valued, they may actually be helpful in the long run. In this respect, a closer inves-
tigation of the intervention effects in the longer term would provide more insight. Our first recommendation for future re-
search would therefore be to compare the short-term to the long-term outcomes, especially for the more complex or higher
order strategies.
4.2. Student characteristics
Our next question concerned the types of students which would profit from the strategy trainings. We distinguished four
categories: (1) regular/average, applying to the student population generally representative of the particular country, (2)
children from low SES backgrounds, (3) children with learning disabilities and special needs, and (4) gifted children and chil-
dren from higher SES backgrounds. Student characteristics were coded and placed into these categories. Graham, Harris, and
Mason (2005), for example, trained struggling writers (category 2), and in the study of Lubliner and Smetana (2005) over half
of the students were provided with free or lower-priced breakfast and lunch programs, a proxy for low SES. Strategy use was
effective for all groups; here no significant differences were found. In this respect our results differ from those obtained by
Hattie et al. (1996), who found that low-ability students were unable to profit from the majority of interventions and that the
medium-ability level students, comparable to our ‘regular students’ group, benefited the most, as did the underachievers.
The fact that our findings point in the opposite direction might be explained by the belief held by many researchers that
strategies in particular can be a way to fulfill the special needs of low-achieving students (see, e.g., Graham & Harris,
2003). Furthermore, the difference in findings may be the result of the improvement of the training programs over the course
of the years. However, it would be interesting to compare these programs to see if there are indeed differences in terms of
improvement, and to determine how other programs could be enhanced.
Our analyses showed that students in both primary and secondary education profit from strategy instruction and that
grade level is related to the training outcomes. Although Dignath and Büttner (2008) exclusively focused on trainings for
the regular student category, they also consider the students’ age. They established that the trainings were more effective
in primary education (overall mean effect size of .61) than in secondary education (overall mean effect size of .54).
Although the difference reported in our study was non-significant, there might be a trend that students in primary edu-
cation profit more than those in secondary education, an outcome also in line with other earlier findings (Hattie et al.,
1996). This result may be explained by the fact that younger children have not yet developed counterproductive learning
habits and are therefore better capable of learning particular tasks, including strategies, than older children, who have al-
ready been influenced more by their experiences (at school). Another possibility is that motivation plays a role and that
younger students are still more open to events, such as education interventions. We did not investigate this assumption
but it might be interesting for future research, as motivation is also associated with improved self-regulated learning. In
addition, although both age and student characteristics revealed no significant differences, it would be worthwhile to
investigate if there is any interaction between these two variables. Unfortunately, we were not able to conduct this type
of analysis.
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 15
4.3. Outcome measures
Apart from looking at the results in different time frames, it is also important to consider the ways in which they are mea-
sured, which brings us to our third research question. We expected the students’ scores to be higher for self-developed tests,
regardless of the strategies trained or the type of students. We indeed observed higher effects for self-developed tests
(Hedges’ g= .78) than for intervention independent tests (Hedges’ g= .45). We assumed that these self-developed tests in-
cluded near-transfer tasks (see also Hattie et al., 1996) as opposed to more general performance measures. Achieving well
in these tasks is considered favorable as it indicates that the use of learning strategies has improved the performance. Yet, in
this context the transferability of the results could be questioned. Of course, positive effects of self-developed tests indicate
the effectiveness of trained learning strategies. Furthermore, it could be argued that self-developed tests are necessary to
detect effects that would be overlooked if independent tests were used, as the former might be more sensitive to changes
occurring as a result of the learning strategies instructed. However, using self-developed tests might also tempt trainers
to direct the student performance toward the test used at the end of the intervention. In the case of intervention-indepen-
dent tests this chance is reduced. Whereas in our analysis the independent tests produced an average effect size of .45, a
quite substantial effect, the effects of the self-developed tests were higher. Of course, an even more promising result would
have been if the intervention independent tests had yielded these higher results. In general, independent tests provide more
information about the effects of strategies on learning or on possibilities of transfer. Furthermore, interventions can be com-
pared more easily if their effects are measured in the same way.
However, many of the investigated studies used self-developed tests, and the research based on intervention independent
tests was frequently combined with additional self-developed measures. Next, there were differences among the domains.
Especially in the interventions in writing, self-developed outcome measures were used. For example, García-Sánchez and
Fidalgo-Redondo (2006) judged the structure, coherence and overall quality of texts written by students on the basis of
an elaborated list of criteria, whereas Graham et al. (2005) used traditional holistic rating scales. Both are examples of
well-defined, extensive evaluation measures. However, standardized tests generally seemed to be lacking in this domain,
which caused differences in the rating scales. In the other domains more standardized tests were available, although these
were not always used to evaluate student performance. We did find some examples, for instance the TSK (a domain-specific
test designed by the National Science Committee of the IME (2004) to examine students’ knowledge of subjects in the science
curriculum), used in the study of Michalsky et al. (2009) or the ‘average mathematics grade’, the general measure used by
Camahalan (2006). Intervention-independent tests were the most often used in comprehensive reading. Examples are the
Oregon State Assessment used in the study of Allen and Hancock (2008) and the Graded Word Reading Test applied by
Wright and Jacobs (2003). When accounting for the measurement instruments applied, we found significantly higher results
for the self-developed tests used in the comprehensive reading domain. To find an explanation for this difference in trans-
ferability it would be interesting to further investigate the learning strategy interventions within this domain.
4.4. Publication bias
Our last question regarded possible publication bias. In meta-analysis a complete coverage of all relevant literature is by
definition unlikely, despite thorough and systematic searches. For example, non-published work is frequently left out, which
means that the results of these studies are not accounted for in the final meta-analyses. To be more specific, effects estimated
in published work tend to be somewhat higher than those gathered in unpublished studies, which means that if the latter
were also included, the meta-analyses’ outcomes would be different. However, since this pitfall has been acknowledged,
methods have been developed to identify the effects of these missing studies. We used the Trim & Fill method (Borenstein
et al., 2009; Duval & Tweedie, 2000) to see how our results might have been affected. We concluded that there was some bias
in the domain of science, whereas overall it was low. The effects we presented seem to be representative of the outcomes
which would have been obtained if more studies had been included.
4.5. Practical recommendations
Summarizing our findings, we conclude that strategy instruction is beneficial for students. Furthermore, our results have
provided us with practical suggestions for future trainings. Which strategies should be instructed depends on the context, i.e.
the subject domain, in which the interventions take place. For strategies to be effective in comprehensive reading, trainings
should incorporate metacognitive knowledge. Students should be taught when, why and how to use strategies and learn to
master a number of them so they have a flexible set of instruments at their disposal. A good example of how this objective
can be realized is having teachers model different strategies and explain their rationale to the students (e.g., Mason, 2004;
Wright & Jacobs, 2003). This kind of metacognitive knowledge can also be tailored to the students individually by providing
them with information about their personal learning style and strengths and weaknesses, and teach them how they can pur-
posefully use this knowledge to improve themselves (e.g., Allen & Hancock, 2008; Dresel & Haugwitz, 2008).
Metacognitive knowledge is also important in writing. Here trainings could particularly focus on the metacognitive strat-
egy ‘‘evaluation’’. Evaluating writing is a step away from reviewing texts, which means editing one’s own texts or those of
others to meet the criteria of an assignment or to make a story more coherent. In this way performance is improved. In many
16 A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
interventions in writing this strategy has been applied (see for instance the studies of Glaser and Burnstein (2007), Reynolds
and Perin (2009) and Torrance, Fidalgo, and García (2007)).
In mathematics, elaboration proved to be very effective. This substrategy entails attaching meaning to new information by
connecting the old to the new material. For instance, in the study on problem solving by Tajika, Nakatsu, Nozaki, Neumann,
and Maruno (2007), students were encouraged to explain to themselves why they chose particular solution steps when solv-
ing a problem. The authors referred to this approach as self-explanation by inference, which means generating new pieces of
knowledge not explicitly included in the problem-solving step approach related to the specific problem. Another method we
frequently came across was connecting new material to previously learned information by finding similarities among the
tasks which help solve new problems (e.g., Kramarski & Hirsch, 2003; Kramarski, Mevarech, & Arami, 2002).
4.6. Limitations
The interpretation of our study’s results is subject to several limitations. First of all, there was quite some fluctuation in the
degree to which the different strategies were addressed in the studies analyzed. In other words, some strategies were used in a
large number of studies while others were conducted in only a few of them. Therefore, the results should be interpreted with
caution, given that part of them is based on only a limited amount of studies. For example, metacognitive strategies were quite
common whereas other (sub)strategies were found much less frequently, making the evidence of their impact less strong.
A second limitation refers to the degree of differentiation among the findings. Our analyses included the characteristics
and grade levels of a broad range of students. Based on these data, we were able to identify effects for different types of stu-
dents. However, although we concluded that there were no significant differences among the groups, there are indications
that students with learning disabilities profit the most from learning strategy trainings. What we do not know, however, is
whether these trainings can be compared with the trainings for regular students, in other words, whether the same strate-
gies are instructed in different ways to regular and disabled students. What would be interesting is to first differentiate
among student groups and then conduct the analyses again, to see which strategies in which domain are the most effective
for which type of student. Unfortunately, the number of studies included in our analyses was too limited to attain this level
of differentiation. A more extensive study sample would be required to be able to distinguish among types of students and to
sufficiently balance the results.
The most important limitation, however, relates to the type of studies included in our analysis. Although referring to
strategies, we in fact analyzed studies of interventions in which strategies were trained. We included only these intervention
studies based on the view that in this way causality could be proven, assuming that the strategies instructed would actually
be used by the students after the instruction had ended, so that the effects estimated would reflect the effects of strategy use.
However, we cannot be entirely sure that this assumption was always met. Furthermore, there were differences among the
interventions. Some were obvious, such as the strategies addressed, but there were also other factors which might have
moderated their effectiveness, for example the trainer of the intervention (a teacher, a researcher, an assistant or a computer
program) or the intensity of the training (the number and length of the sessions). We did not consider these issues because
reliable information was not consistently available across the studies. However, these factors might have partly explained
the effects found, and if so, the strategies’ actual influence may diverge somewhat from the results reported in our study.
Therefore, it is important to keep in mind that when referring to effective learning strategies, we in fact refer to the effec-
tiveness of the interventions focused on the instruction of these strategies. We assume that these trainings are conducted
properly, that the strategies addressed are indeed the approaches used by the students after the training and that they were
instrumental in improving student performance. Of course, as we rely on the accounts of primary studies, we can never be
totally sure that all aforementioned assumptions have been met.
Moreover, one drawback of the relatively small number of studies we gathered is that it was statistically impossible to
calculate singular or additional effects of the individual strategies, as almost all studies involved more than one trained strat-
egy. To be able to draw any reliable conclusions in this respect, the input for the meta-analysis would have to be far larger
than the current set was. Our results can only claim that, for example, trainings including the substrategy ‘planning’ (either
with or without another strategy), seems to be more beneficial for performance than trainings without this strategy. Nev-
ertheless, we did manage to account for this drawback to some degree: we assumed that both groups of training (the one
including a certain strategy and the one not including this strategy) were on average comparable with respect to other strat-
egies included as we could rely on a substantial number of studies. Furthermore, the strategies that proved significant in our
study were brought together in an additional analysis to test the effects of all these strategies at once. This analysis showed
similar results, which indicates that our initial significant findings were not related to other strategies in the training.
In addition to the limitations of our own research, meta-analysis in general is associated with two problems in particular,
namely publication bias and the difficulty of deciding which studies are and which are not suitable for comparison (a balance
has to be found between the use of a large sample which includes all relevant studies and the risk of comparing apples with
oranges). As researchers we are aware of these risks and we have tried to take them into account in our study. It would, how-
ever, not be reasonable to assume that meta-analysis provides perfect solutions and definite, clear-cut answers to each and
every research problem.
Yet, in our case, what resulted from our approach is a meta-analysis in which we investigated the effectiveness of learning
strategies. More specifically, we tried to provide an answer to the questions which learning strategies improve academic
performance, and whether the results yielded vary per student group or outcome measure. What justifies our decision to
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 17
use meta-analysis, which combines the results of a number of studies carefully selected on the basis of both content-related
and methodological criteria, is that this approach has provided information and insights which could not have been obtained
by any individual study. And although some questions remain, this analysis has presented a first outline of effective strat-
egies. This information is valuable in that it contributes to both the body of knowledge of learning strategies and to the edu-
cational practice, where it can be used in the realization of effective strategies in different contexts to improve student
performance.
Appendix A. Strategies – categories and examples
Strategy Example Study
Cognitive
Rehearsal Playing flash-card games to remember new words. Bruce and Robinson (2001)
Elaboration Summarizing passages, reducing the number of words to
1
=
4
of the
original text.
Boulware-Gooden, Carreker,
Thornhill, and Malatesha
(2007)
Organization Using graphic organizers to structure writing ideas. Harris, Graham, & Mason,
2006
Metacognitive
Planning Making judgments about use of cognitive abilities and predicting
the number of pages to be read in a specific time period.
Allen and Hancock (2008)
Monitoring Children were asked if they had a clear understanding of what
they were doing, if the task made sense, if they needed to make
any changes.
Pennequin, Sorel, Nanty, and
Fontaine (2010)
Evaluating Students had to answer questions on the computer screen during
and after the solution process (e.g. ‘What is the difference
between the expression X and the expression that you found’?).
Kramarski and Gutman (2006)
Management
Effort Emphasizing the role of effort in learning, making the positive
effects of instruction concrete and visible and promoting an ‘‘I can
do’’ attitude.
Tracy, Reid, and Graham
(2009)
Peer Explicitly encouraging students to seek assistance from their
peers, parents, teachers and tutors.
Camahalan (2006)
Environment Using a worksheet with tips on how to organize a workplace,
regulate study time and breaks, deal with distractions etc.
Stoeger and Ziegler (2008)
Motivational aspects
Self-efficacy An explicit focus on statements such as ‘I feel capable of writing a
good text’. Students also learned to make positive causal
attributions (I made a big effort and I got a good result).
García-Sánchez and Fidalgo-
Redondo (2006)
Task value The first training session aimed to motivate students by focusing
on the communicative function and importance of writing.
Torrance et al. (2007)
Goal orientation One of the students’ goals was to become aware of their positive
and negative ‘‘attitudes toward mathematics’’. The children were
supported in developing a constructive and positive attitude
toward the subject.
Perels, Dignath, and Schmitz
(2009)
Metacognitive knowledge
Personal
metacognitive
knowledge
Students and teachers evaluated students’ prior knowledge with
two open-ended questions (i.e., ‘‘What can I already do well?’’ and
‘‘What do I still have trouble with?’’), thus evaluating personal
strengths and weaknesses.
Dresel and Haugwitz (2008)
General
metacognitive
knowledge
Modeling was combined with feedback about when, where and
why specific strategies were useful in conjunction with checking
procedures to establish whether or not a strategy was effective.
Wright and Jacobs (2003)
18 A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
Appendix B. Key characteristics of the studies included in the meta-analysis
Authors Subject nStudents’
grade
Student
characteristics
Metacognitive
knowledge
Cognitive and
metacognitive strategies
Motivation and
Management
strategies
Type
Instrument
(ntests)
Effect
size
Aleven and Koerdinger
(2002)
Math 24 10 Regular None Planning, monitoring None Self (1) 0,52
None Monitoring None Self (1) 0,64
Allen and Hancock (2008) Reading 113 5 Low SES General Planning, monitoring,
control
None Indep (2) 0,15
General Planning, monitoring,
control, rehearsal,
elaboration
None Indep (2) 0,24
Blank (2000) Other 125 7 Regular None Monitoring None Self (1) -0,48
Boulware-Gooden et al.
(2007)
Reading 112 3 Regular None Planning, control,
elaboration, organization
None Indep (2) 0,50
Bruce and Robinson (2001) Reading 46 5,5 Special needs None Planning, monitoring,
rehearsal, elaboration
Self-efficacy Indep (2) 0,37
Brunstein and Glaser (2011) Writing 117 4 Regular General None None Indep (1) 0,86
Camahalan (2006) Math 60 5 Special needs Personal Planning, monitoring,
control, rehearsal
Self-efficacy,
resources, peers
Both (2) 0,54
Cantrell, Almasi, and Carter
(2010)
Reading 47 6 Special needs None Monitoring, elaboration,
organisation
Resources, peers Indep (2) 0,25
Cantrell et al. (2010) Reading 47 9 Special needs None Monitoring, elaboration,
organisation
Resources, peers Indep (2) 0,09
Dejonckheere, Van de Keere,
and Tallir (2011)
Science 117 4,5 Regular General Planning, control None Self (1) 0,70
Dresel and Haugwitz (2008) Math 151 6 Regular None Control Self-efficacy Self (1) 0,23
Personal Planning, monitoring,
control
Self-efficacy Self (1) 0,77
Erktin (2004) Math 45 6 Gifted None Planning, monitoring,
control, elaboration
None Self (1) 0,87
García-Sánchez and Fidalgo-
Redondo (2006)
Writing 121 5,5 Special needs General Planning, monitoring,
control, rehearsal,
elaboration, organisation
Task value, effort,
peers
Self (3) 2,55
General Planning, monitoring,
control, rehearsal,
elaboration
Self-efficacy,
effort, resrouces,
peers
Self (3) 1,96
Glaser and Burnstein (2007) Writing 46 4 Regular None Planning, monitoring,
control, elaboration,
None Both (4) 1,70
(continued on next page)
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 19
Appendix B (continued)
Authors Subject nStudents’
grade
Student
characteristics
Metacognitive
knowledge
Cognitive and
metacognitive strategies
Motivation and
Management
strategies
Type
Instrument
(ntests)
Effect
size
organisation
None Elaboration, organisation None Both (4) 0,93
Graham et al. (2005) Writing 30 3 Special needs None Planning, monitoring,
control, rehearsal,
organisation
Task value, effort,
peers
Self (4) 0,92
General Planning, monitoring,
control, rehearsal,
organisation
Task value, effort,
peers
Self (4) 1,03
Guterman (2003) Reading 109 4 Low SES None Planning, monitoring,
control, elaboration
Effort Self (1) 0,83
Harris et al. (2006) Writing 45 2 Special needs None Planning, monitoring,
control, rehearsal,
organisation
Task value, effort,
peers
Self (4) 0,71
General Planning, monitoring,
control, rehearsal,
organisation
Task value Self (4) 1,21
Hauptman and Cohen
(2011)
Math 80 10 Regular None Planning, monitoring,
control
None Indep (1) 0,35
None Planning, monitoring,
control
None Indep (1) 0,78
Huff and Nietfeld (2009) Reading 73 5 Regular None Monitoring, elaboration None Indep (1) 0,00
General Monitoring, elaboration None Indep (1) 0,00
Jacobse and Harskamp
(2009)
Math 73 5 Regular None Planning, monitoring,
control
None Self (1) 0,76
Kaniel, Licht, and Peled
(2000)
Reading 79 5 Gifted General Monitoring, control,
elaboration
Resources Indep (1) 0,66
Kapa (2007) Math 79 8 Regular None Planning, monitoring,
control
None Self (2) 1,01
None Planning, monitoring None Self (2) 0,98
None Control None Self (2) 0,74
Kapa (2001) Math 72 8 Regular None Planning, monitoring,
control
None Self (1) 0,60
None Planning, monitoring None Self (1) 0,44
None Control None Self (1) 0,30
Kim and Pedersen (2011) Science 72 6 Regular None Planning, monitoring,
control
None Self (2) 0,41
Kramarski and Dudai (2009) Math 24 9 Regular General Monitoring None Both (2) 0,24
General Monitoring None Both (2) 0,87
20 A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
Appendix B (continued)
Authors Subject nStudents’
grade
Student
characteristics
Metacognitive
knowledge
Cognitive and
metacognitive strategies
Motivation and
Management
strategies
Type
Instrument
(ntests)
Effect
size
Kramarski and Gutman
(2006)
Math 24 9 Regular Personal Monitoring, control None Self (2) 1,17
Kramarski and Hirsch
(2003)
Math 24 8 Regular None Planning, monitoring,
elaboration
None Unknown
(1)
0,80
None Planning, monitoring,
elaboration
None Unknown
(1)
0,43
Kramarski and Mevarech
(2003)
Math 24 8 Regular General Planning, monitoring,
elaboration
None Self (2) 0,55
General Planning, monitoring,
elaboration
None Self (2) 0,34
Kramarski and Mizrachi
(2006)
Math 24 7 Regular General Planning, monitoring,
elaboration
Peers Self (2) 1,80
General Planning, monitoring,
elaboration
Peers Self (2) 1,06
Kramarski and Ritkof (2002) Math 22 9 Regular General Planning, monitoring,
elaboration
Peers Self (2) 0,71
Kramarski and Zoldan
(2008)
Math 22 9 Regular General Monitoring, control None Self (4) 0,67
General Planning, monitoring,
elaboration
None Self (4) 0,38
General Planning, monitoring,
control, elaboration
None Self (4) 0,76
Kramarski et al. (2002) Math 22 7 Regular None Planning, monitoring,
elaboration
None Self (2) 1,02
None Planning, monitoring,
elaboration
None Self (2) 1,17
Kramarski, Mevarech, and
Lieberman (2001)
Math 22 7 Regular General Planning, monitoring,
elaboration
None Self (2) 1,49
General Planning, monitoring,
elaboration
None Self (2) 0,56
Lubliner and Smetana
(2005)
Reading 77 5 Low SES General Monitoring, control,
elaboration
Peers Self (2) 0,60
Mason (2004) Reading 32 5 Special needs General Planning, monitoring,
control, elaboration,
organisation
None Self (8) 0,94
Mevarech and Kramarski
(2004)
Math 94 8 Regular None Planning, monitoring,
elaboration
None Self (1) 0,15
None Planning, monitoring,
elaboration
None Self (1) 0,62
(continued on next page)
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 21
Appendix B (continued)
Authors Subject nStudents’
grade
Student
characteristics
Metacognitive
knowledge
Cognitive and
metacognitive strategies
Motivation and
Management
strategies
Type
Instrument
(ntests)
Effect
size
Meyer, Abrami, Wade, Aslan,
and Deault (2010)
Other 296 5 Regular None Planning, monitoring,
control, organisation
Goal orientation,
peers
Indep (4) 0,08
Michalsky et al. (2009) Science 108 4 Regular Personal Planning, monitoring,
control, elaboration
None Both (2) 0,79
Personal Planning, monitoring,
control, elaboration
None Both (2) 0,62
Personal Planning, monitoring,
control, elaboration
None Both (2) 1,62
Molenaar, Chiu, and Sleegers
(2011)
Writing 110 5 Regular None Planning, monitoring,
control, elaboration,
organisation
Goal orientation,
peers
Self (1) 0,45
None Planning, monitoring,
control, elaboration,
organisation
Goal orientation,
peers
Self (1) 0,85
Mourad (2009) Writing 97 7 Special needs Both Planning, monitoring,
rehearsal, elaboration,
organisation
Self-efficacy Self (1) 2,55
Pennequin et al. (2010) Math 97 3 Regular Personal Planning, monitoring,
control, elaboration,
organisation
None Indep (1) 2,17
Perels et al. (2009) Math 220 6 Regular Both Planning, monitoring,
control
Self-efficacy, goal
orientation
Self (1) 0,45
Perels, Gurtler, and Schmitz
(2005)
Math 220 8 Gifted None Planning, control,
organisation
Self-efficacy,
effort
Self (1) 0,23
None Planning, control Self-efficacy,
effort
Self (1) 0,33
None Organisation None Self (1) 0,46
Peters and Kitsantas (2010a) Science 170 8 Regular None Monitoring, control None Both (2) 0,69
Peters and Kitsantas (2010b) Science 68 8 Regular None Monitoring, ebaloration,
organisation
None Both (2) 0,74
Reynolds and Perin (2009) Writing 68 7 Low SES General Control, elaboration,
organisation
None Self (3) 1,26
General Planning, organisatie None Self (3) 0,92
Sanz de Acedo Lizarraga,
Sanz de Acedo
Baquedano, and Oliver
(2010)
Other 68 11 Low SES General Planning, monitoring,
control, elaboration,
organisation
None Self (1) 1,22
22 A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
Appendix B (continued)
Authors Subject nStudents’
grade
Student
characteristics
Metacognitive
knowledge
Cognitive and
metacognitive strategies
Motivation and
Management
strategies
Type
Instrument
(ntests)
Effect
size
Souvingnier and
Mokhlesgerami (2006)
Reading 65 5 Regular None Planning, monitoring,
control, elaboration,
organisation
Self-efficacy,
effort
Indep (1) 0,14
65 None Planning, monitoring,
control, elaboration,
organisation
None Indep (1) -0,10
43 None Monitoring, elaboration,
organisation
None Indep (1) 0,12
40 None Planning, monitoring,
control, elaboration,
organisation
Self-efficacy,
effort
Indep (1) 0,49
198 None Planning, monitoring,
control, elaboration,
organisation
None Indep (1) 0,57
Stoeger and Ziegler (2008) Math 198 4 Regular Personal Planning, monitoring,
control
Self-efficacy, goal
oriantation effort,
resources
Self (1) 0,36
Stoeger and Ziegler (2010) Math 188 4 Regular General Planning, control Goal orientation,
effort
Self (1) 0,48
Tajika et al. (2007) Math 188 6 Regular None elaboration Effort Self (1) 1,04
Math 43 6 Regular None Monitoring None Self (1) 0,04
Talebi (2009) Reading 43 11 Gifted General Planning, monitoring None Both (2) 1,15
General Planning, monitoring None Both (2) 1,46
Teong (2003) Math 43 6 Special needs None Monitoring, control,
organisation
None Self (1) 0,60
Torrance et al. (2007) Writing 50 6 Regular General Planning, monitoring,
control, organisation
Task value, peers Self (3) 2,55
Tracy et al. (2009) Writing 50 3 Regular Personal Planning, monitoring,
organisation
Self-efficacy,
effort
Self (2) 0,34
Van Keer and Vanderlinde
(2010)
Reading 59 3 Regular None Monitoring, elaboration,
organisation
Peers Indep (1) 0,05
6 None Monitoring, elaboration,
organisation
Peers Indep (1) 0,03
Vaughn, Klingner, and
Swanson (2011)
Reading 59 7,5 Regular Personal Planning, monitoring,
control, elaboration
Peers Indep (3) -0,01
Wright and Jacobs (2003) Reading 59 3,5 Special needs Personal Planning, monitoring None Indep (3) 0,68
Zion, Michalsky, and
Mevarech (2005)
Science 53 10 Regular General Planning, monitoring,
control
None Self (2) 0,65
General Planning, monitoing,
control
None Self (2) 0,76
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 23
References
Ahn, S., Ames, A. J., & Myers, N. D. (2012). A review of meta-analyses in education: Methodological strengths and weaknesses. Review of Educational Research,
82, 436–476.
Alexander, P. A., Graham, S., & Harris, K. (1998). A perspective on strategy research: Progress and prospects. Educational Psychology Review, 10,
129–154.
Bjorklund, D. F., Dukes, C., & Brown, R. D. (2009). The development of memory strategies. In M. L. Courage & N. Cowan (Eds.), The development of memory in
infancy and childhood (pp. 145–175). New York: Psychology Press.
Boekaerts, M. (1997). Self-regulated learning: A new concept embraced by researchers, policy makers, educators, teachers, and students. Learning and
Instruction, 7, 161–186.
Boekaerts, M., Pintrich, P. R., & Zeidner, M. (2000). Handbook of self-regulation. San Diego, CA: Academic Press.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley.
Chiu, C. W. T. (1998). Synthesizing metacognitive interventions: what training characteristics can improve reading performance? Paper presented at the
Annual Meeting of the American Educational Research Association (San Diego, CA, April 13–17, 1998).
Dignath, C., & Büttner, G. (2008). Components of fostering self-regulated learning among students. A meta-analysis on intervention studies at primary and
secondary school level. Metacognition and Learning, 3, 231–264.
Dignath, C., Büttner, G., & Langfeldt, H. P. (2008). How can primary school students acquire self-regulated learning most efficiently? A meta-analysis on
interventions that aim at fostering self-regulation. Educational Research Review, 3, 101–129.
Dinsmore, D. L., Alexander, P. A., & Loughlin, S. M. (2008). Focusing the conceptual lens on metacognition, self-regulation and self-regulated learning.
Educational Psychology Review, 20, 391–409.
Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics,
56, 455–463.
Ellis, P. D. (2010). The essential guide to effect sizes: An introduction to statistical power, meta-analysis and the interpretation of research results. Cambridge
University Press.
Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231–236). Hillsdale, NJ: Erlbaum.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34, 906–911.
Garner, R. (1990). When children and adults do not use learning strategies: Toward a theory of settings. Review of Educational Research, 60, 517–529.
Graham, S., & Harris, K. (2003). Students with learning disabilities and the process of writing: A meta-analysis of SRSD studies. In H. L. Swanson, K. R. Harris,
& S. Graham (Eds.), Handbook of learning disabilities (pp. 323–344). New York: Guilford Press.
Hadwin, A. F., & Winne, P. H. (1996). Study strategies have meagre support: A review with recommendations for implementation. The Journal of Higher
Education, 67, 692–715.
Haller, E., Child, D. A., & Walberg, H. J. (1988). Can comprehension be taught? A quantitative synthesis of ‘‘metacognitive’’ studies. Educational Researcher,
17(9), 5–8.
Hattie, J., Biggs, J., & Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of Educational Research, 66,
99–136.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
Lipsey, M. W., & Wilson, D. B. (2000). Practical meta-analysis. Thousand Oaks, CA: Sage.
Mayer, R. E. (2008). Learning and instruction (2nd ed.). Upper Saddle River, NJ: Pearson Merrill Prentice Hall.
Palincsar, A., & Brown, A. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 1,
117–175.
Paris, S. G., & Newman, R. S. (1990). Developmental aspects of self-regulated learning. Educational Psychologist, 25, 87–102.
Pintrich, P. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation
(pp. 451–502). San Diego: Academic Press.
Pintrich, P., Smith, D., Garcia, T., McKeachie, W. (1991). A Manual for the Use of the Motivated Strategies for Learning Questionnaire. Technical Report 91-B-
004. The Regents of the University of Michigan.
Pressley, M. (2002). Comprehension strategies instruction: A turn-of-the-century status report. In C. C. Block & M. Pressley (Eds.), Comprehension instruction:
Research-based best practices (pp. 11–27). New York: Guilford.
Pressley, M., Goodchild, F., Fleet, J., & Zajchowski, R. (1989). The challenges of classroom strategy instruction. Elementary School Journal, 89, 301–342.
Raudenbush, S. W., Bryk, A. S., & Congdon, R. (2004). HLM 6 for Windows [Computer software]. Skokie, IL: Scientific Software International Inc.
Schraw, G., & Dennison, R. S. (1994). Assessing meta-cognitive awareness. Contemporary Educational Psychology, 19, 460–475.
Veenman, M. V. J., & Spaans, M. A. (2005). Relation between intellectual and metacognitive skills: Age and task differences. Learning and Individual
Differences, 15, 159–176.
Weinstein, C., Acee, T. W., & Jung, J. H. (2011). Self-regulation and learning strategies. New Directions for Teaching and Learning, 126, 45–53.
Weinstein, C., Husman, J., & Dierking (2000). Self-regulation interventions with a focus on learning strategies. In M. Boekaerts, P. R. Pintrich, & M. Zeidner
(Eds.), Handbook of self-regulation: Theory, research, and applications (pp. 727–747). San Diego: Academic Press.
Weinstein, C. E., & Mayer, R. E. (1986). The teaching of learning strategies. In M. Wittrock (Ed.), Handbook of research on teaching (pp. 315–327). New York,
NY: Macmillan.
Whitebread, D., Coltman, P., Pasternak, D. P., Sangster, C., Grau, V., Bingham, S., Almeqdad, Q., & Demetriou, D. (2009). The development of two observational
tools for assessing metacognition and self-regulated learning in young children. Metacognition and Learning, 4, 63–85.
Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25, 3–17.
Zimmerman, B. J. (1994). Dimensions of academic self -regulation: A conceptual framework for education. In D. H. Schunk & B. J. Zimmerman (Eds.), Self-
regulation of learning and performance: Issues and educational applications (pp. 3–21). Hillsdale, NJ: Lawrence Erlbaum Associates.
References: Articles Included in the Meta-Analysis
Aleven, V. A. W. M. M., & Koerdinger, K. R. (2002). An effective metacognitve strategy: Learning by doing and explaining with a computer-based cognitive
tutor. Cognitive Science: A Multidisciplinary Journal, 26, 147–179.
Allen, K. D., & Hancock, T. E. (2008). Reading comprehension improvement with individualized cognitive profiles and metacognition. Literacy Research and
Instruction, 47, 124–139.
Blank, L. M. (2000). A metacognitve learning cycle: A better warranty for student understanding? Science Education, 84, 486–506.
Boulware-Gooden, R., Carreker, S., Thornhill, A., & Malatesha, J. R. (2007). Instruction of metacognitive strategies enhances reading comprehension and
vocabulary achievement of third-grade students. The Reading Teacher, 61, 70–77.
Bruce, M. E., & Robinson, G. L. (2001). The clever kid’s reading program: metacognition and reciprocal teaching. Paper presented at the Annual European
Conference on Reading, Dublin.
Brunstein, J. C., & Glaser, C. (2011). Testing a path-analytical mediation model of how self-regulated writing skills improve fourth-graders’ composition
skills: A randomized controlled trial. Journal of Educational Psychology, 103, 922–938.
Camahalan, F. M. G. (2006). Effects of self-regulated learning on mathematics achievement of selected Southeast Asian children. Journal of Instructional
Psychology, 33, 194–205.
24 A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
Cantrell, S. A., Almasi, J. F., & Carter, J. C. (2010). The impact of a strategy-based intervention on the comprehension and strategy use of struggling adolescent
readers. Journal of Educational Psychology, 102, 257–280.
Dejonckheere, P., Van de Keere, K., & Tallir, I. (2011). Are fourth and fifth grade children better scientists through metacognitive learning? Electronic Journal
of Research in Educational Psychology, 9, 133–156.
Dresel, M., & Haugwitz, M. (2008). A computer-based approach to fostering motivation and self-regulated learning. The Journal of Experimental Education, 77,
3–18.
Erktin, E. (2004). Teaching thinking for mathematics through the enhancement of metacognitive skills. Research in the Schools, 11, 3–13.
García-Sánchez, J. N., & Fidalgo-Redondo, R. (2006). Effects of two types of self-regulatory instruction programs on students learning disabilities in writing
products, processes, and self-efficacy. Learning Disability Quarterly, 29, 181–211.
Glaser, C., & Burnstein, J. C. (2007). Improving fourth-grade students’ composition skills: effects of strategy instruction and self-regulation procedures.
Journal of Educational Psychology, 99, 297–310.
Graham, S., Harris, K. R., & Mason, L. (2005). Improving the writing performance, knowledge, and self-efficacy of struggling young writers: The effects of self-
regulated strategy development. Contemporary Educational Psychology, 30, 207–241.
Guterman, E. (2003). Integrating written metacognitive awareness guidance as a ‘psychological tool’ to improve student performance. Learning and
Instruction, 13, 633–651.
Harris, K. R., Graham, S., & Mason, L. H. (2006). Improving the writing, knowledge, and motivation of struggling young writers: effects of self-regulated
strategy development with and without peer support. American Educational Research Journal, 43, 295–340.
Hauptman, H., & Cohen, A. (2011). The synergetic effect of learning styles on the interaction between virtual environments and the enhancement of spatial
thinking. Computers & Education, 57, 2106–2117.
Huff, J. D., & Nietfeld, J. L. (2009). Using strategy instruction and confidence judgments to improve metacognitive monitoring. Metacognition and Learning, 4,
161–176.
Jacobse, A. E., & Harskamp, E. (2009). Student-controlled metacognitive training for solving word problems in primary school mathematics. Educational
Research and Evaluation, 15, 447–463.
Kaniel, S., Licht, P., & Peled, B. (2000). The influence of metacognitve instruction of reading and writing strategies on positive transfer. Gifted Education
International, 51, 45–63.
Kapa, E. (2001). A metacognitve support during the process of problem solving in a computerized environment. Educational Studies in Mathematics, 47,
317–336.
Kapa, E. (2007). Transfer from structured to open-ended problem solving in a computerized metacognitive environment. Learning and Instruction, 17,
688–707.
Kim, H. J., & Pedersen, S. (2011). Advancing young adolescents’ hypothesis-development performance in a computer-supported and problem-based learning
environment. Computers and Education, 57, 1780–1789.
Kramarski, B., & Dudai, V. (2009). Group-Metacognitve support for online inquiry in mathematics with differential self-questioning. Journal of Educational
Computer Research, 40, 377–404.
Kramarski, B., & Gutman, M. (2006). How can self-regulated learning be supported in mathematical e-learning environments? Journal of Computer Assisted
Learning, 22, 24–33.
Kramarski, B., & Hirsch, C. (2003). Effects of computer algebra system (CAS) with metacognitive training on mathematical reasoning. Educational Media
International, 40, 249–257.
Kramarski, B., & Mevarech, Z. R. (2003). Enhancing mathematical reasoning in classroom: the effects of cooperative learning and the metacognitve training.
American Educational Research Journal, 40, 281–310.
Kramarski, B., Mevarech, Z. R., & Arami, M. (2002). The effects of metacognitve instruction on solving mathematical authentic tasks. Educational Studies in
Mathematics, 49, 225–250.
Kramarski, B., Mevarech, Z. R., & Lieberman, A. (2001). Effects of multilevel versus unilevel metacognitve training on mathematical reasoning. Journal of
Educational Research, 94, 292–300.
Kramarski, B., & Mizrachi, N. (2006). Online discussion and self-regulated learning: Effects of instructional methods on mathematical literacy. Journal of
Educational Research, 99, 218–230.
Kramarski, B., & Ritkof, R. (2002). The effects of metacognition and email interactions on learning graphing. Journal of Computer Assisted Learning, 18, 33–43.
Kramarski, B., & Zoldan, S. (2008). Using errors as springboards for enhancing mathematical reasoning with three metacognitive approaches. Journal of
Educational Research, 102, 137–151.
Lubliner, S., & Smetana, L. (2005). The effects of comprehensive vocabulary instruction on title I students’ metacognitive word-learning skills and reading
comprehension. Journal of Literacy Research, 37, 163–200.
Mason, L. H. (2004). Explicit self-regulated strategy development versus reciprocal questioning: effects on expository reading comprehension among
struggling readers. Journal of Educational Psychology, 96, 238–296.
Mevarech, Z. R., & Kramarski, B. (2004). The effects of metacognitve training versus worked-out examples on students’ mathematical reasoning. British
Journal of Educational Psychology, 73, 449–471.
Meyer, E., Abrami, P. C., Wade, C. A., Aslan, O., & Deault, L. (2010). Improving literacy and metacognition with electronic portfolio’s: Teaching and learning
with ePEARL. Computers & Education, 55, 84–91.
Michalsky, T., Mevarech, Z. R., & Haibi, L. (2009). Elementary school children reading scientific texts: effects of metacognitive instruction. Journal of
Educational Research, 102, 363–376.
Molenaar, I., Chiu, M. M., & Sleegers, P. (2011). Scaffolding of small-groups’ metacognitive activities with an avatar. International Journal of Computer-
Supported Collaborative Learning, 6, 601–624.
Mourad, A. E. (2009). The effectiveness of a program based on self-regulated strategy development on the writing skills of writing-disabled secondary
school students. Electronic Journal of Research in Educational Psychology, 7, 5–24.
Pennequin, V., Sorel, O., Nanty, I., & Fontaine, R. (2010). Metacognition and low achievement in mathematics: The effect of training in the use of
metacognitive skills to solve mathematical word problems. Thinking and Reasoning, 16, 198–220.
Perels, F., Dignath, C., & Schmitz, B. (2009). Is it possible to improve mathematical achievement by means of self-regulation strategies? Evaluationofan
intervention in regular math classes. European Journal of Psychology of Education, 24, 17–31.
Perels, F., Gurtler, T., & Schmitz, B. (2005). Training of self-regulatory and problem-solving competence. Learning and Instruction, 15, 123–139.
Peters, E. E., & Kitsantas, A. (2010a). Self-regulation of student epistemic thinking in science: The role of metacognitive prompts. Educational Psychology, 30,
27–52.
Peters, E. E., & Kitsantas, A. (2010b). The effect of nature of science metacognitive prompts on science students’ content and nature of science knowledge,
metacognition, and self-regulatory efficacy. School Science and Mathematics, 110, 382–396.
Reynolds, G. A., & Perin, D. (2009). A comparison of text structure and self-regulated writing strategies for composing from sources by middle school
students. Reading Psychology, 30, 265–300.
Sanz de Acedo Lizarraga, M. L., Sanz de Acedo Baquedano, M. T., & Oliver, M. S. (2010). Stimulation of thinking skills in high school students. Educational
Studies, 36, 329–340.
Souvingnier, E., & Mokhlesgerami, J. (2006). Using self-regulation as a framework for implementing strategy instruction to foster reading comprehension.
Learning and Instruction, 16, 57–71.
Stoeger, H., & Ziegler, A. (2010). Do pupils with differing cognitive abilities benefit similarly from a self-regulated learning training program? Gifted
Education International, 26, 110–123.
A.S. Donker et al. / Educational Research Review 11 (2014) 1–26 25
Stoeger, H., & Ziegler, A. (2008). Evaluation of a classroom based training to improve self-regulation in time management tasks during homework activities
with fourth graders. Metacognition and Learning, 3, 207–230.
Tajika, H., Nakatsu, N., Nozaki, H., Neumann, E., & Maruno, S. (2007). Effects of self-explanation as a metacognitive strategy for solving mathematical word
problems. Japanese Psychological Research, 49, 222–233.
Talebi, S. H. (2009). An attempt towards learner autonomy in L2 (English) and L3 (Arabic) reading through cognitive and metacognitive reading strategy
instruction (CMRSI) in L2. Indian Journal of Applied Linguistics, 35(2), 101–112.
Teong, S. K. (2003). The effect of metacognitive training on mathematical word-problem solving. Journal of Computer Assisted Learning, 19, 46–55.
Torrance, M., Fidalgo, R., & García, J.-N. (2007). The teachability and effectiveness of cognitive self-regulation in sixth-grade writers. Learning and Instruction,
17, 265–285.
Tracy, B., Reid, R., & Graham, S. (2009). Teaching young students strategies for planning and drafting stories: The impact of self-regulated strategy
development. Journal of Educational Research, 102, 323–331.
Van Keer, H., & Vanderlinde, R. (2010). The impact of cross-age peer tutoring on third and sixth graders’ reading strategy awareness, reading strategy use,
and reading comprehension. Middle Grades Research Journal, 5, 33–45.
Vaughn, S., Klingner, J. K., & Swanson, E. A. (2011). Efficacy of collaborative strategic reading with middle school students. American Educational Research
Journal, 48, 938–964.
Wright, J., & Jacobs, B. (2003). Teaching phonological awareness and metacognitive strategies to children with reading difficulties: A comparison of two
instructional methods. Educational Psychology, 23, 17–47.
Zion, M., Michalsky, T., & Mevarech, Z. R. (2005). The effects of metacognitive instruction embedded within an asynchronous learning network on scientific
inquiry skills. International Journal of Science Education, 27, 957–983.
26 A.S. Donker et al. / Educational Research Review 11 (2014) 1–26
