ArticlePDF Available

Strategic Resource Use for Learning: A Self-Administered Intervention That Guides Self-Reflection on Effective Resource Use Enhances Academic Performance


Abstract and Figures

Many educational policies provide learners with more resources (e.g., new learning activities, study materials, or technologies), but less often do they address whether students are using these resources effectively. We hypothesized that making students more self-reflective about how they should approach their learning with the resources available to them would improve their class performance. We designed a novel Strategic Resource Use intervention that students could self-administer online and tested its effects in two cohorts of a college-level introductory statistics class. Before each exam, students randomly assigned to the treatment condition strategized about which academic resources they would use for studying, why each resource would be useful, and how they would use their resources. Students randomly assigned to the treatment condition reported being more self-reflective about their learning throughout the class, used their resources more effectively, and outperformed students in the control condition by an average of one third of a letter grade in the class.
Content may be subject to copyright.
Psychological Science
1 –12
© The Author(s) 2017
Reprints and permissions:
DOI: 10.1177/0956797617696456
Research Article
Educational policies encourage the provision of ample,
high-quality academic resources and support for students,
be it investing more money per student, introducing new
class activities, or improving technology in schools (e.g.,
Barkley, Cross, & Major, 2014; Cuban & Cuban, 2009;
Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013;
Hanushek, 1997). All this makes sense from an educa-
tional perspective—after all, the effectiveness of many
student-centered interventions relies on the availability of
rich learning resources (Cohen, Garcia, Apfel, & Master,
2006; Paunesku et al., 2015).
However, providing students with all these resources
hinges on the assumption that they know how to select
and use their resources wisely. Yet empirical research
suggests that many students do not tend to proactively or
strategically self-regulate their learning on their own
(Zimmerman, 2011; Zimmerman & Martinez-Pons, 1988).
Oftentimes, many of them are passive consumers of
information and lack tactical awareness when studying.
This may substantially limit what students achieve in their
classes, rather than allowing them to perform to their
How can we help people to regulate their own learn-
ing more effectively? Effectual use of metacognitive self-
regulation, which involves the proactive and tactical
696456PSSXXX10.1177/0956797617696456Chen et al.Strategic Resource Use Intervention
Corresponding Author:
Patricia Chen, Department of Psychology, 269 Jordan Hall, 450 Serra
Mall, Stanford University, Stanford, CA 94305
Strategic Resource Use for Learning:
A Self-Administered Intervention That
Guides Self-Reflection on Effective
Resource Use Enhances Academic
Patricia Chen1, Omar Chavez2, Desmond C. Ong1,3, and
Brenda Gunderson4
1Department of Psychology, Stanford University; 2Department of Statistics and Data Sciences,
University of Texas at Austin; 3Department of Computer Science, Stanford University; and 4Department
of Statistics, University of Michigan, Ann Arbor
Many educational policies provide learners with more resources (e.g., new learning activities, study materials, or
technologies), but less often do they address whether students are using these resources effectively. We hypothesized
that making students more self-reflective about how they should approach their learning with the resources available
to them would improve their class performance. We designed a novel Strategic Resource Use intervention that students
could self-administer online and tested its effects in two cohorts of a college-level introductory statistics class. Before
each exam, students randomly assigned to the treatment condition strategized about which academic resources they
would use for studying, why each resource would be useful, and how they would use their resources. Students
randomly assigned to the treatment condition reported being more self-reflective about their learning throughout the
class, used their resources more effectively, and outperformed students in the control condition by an average of one
third of a letter grade in the class.
strategic resource use, self-regulation, psychological intervention, learning, performance
Received 4/11/16; Revision accepted 2/7/17
2 Chen et al.
direction of mental processes toward one’s goals, has
been shown to predict better learning, motivation, and
academic performance among learners (Pintrich & De
Groot, 1990; Pintrich, Smith, Garcia, & McKeachie, 1993).
This mental process can be applied to students’ manage-
ment of the resources available to them, such as allocating
study time effectively, reviewing their class notes before
an exam, and seeking help when necessary (Boekaerts,
1999; Pintrich, Smith, Garcia, & McKeachie, 1991). Previous
research has found that students who manage their
resources more effectively tend to perform better in their
classes (Karabenick, 2003; Pintrich et al., 1993). Although
greater engagement in self-reported resource-management
behaviors has been associated with better academic per-
formance in the literature, an important question remains:
Would an experimental intervention that specifically
addresses strategies for resource use causally contribute
to performance?
In the literature, a number of educational interven-
tions have focused on teaching students a variety of self-
regulatory skills at the same time, including various
learning techniques, setting goals, organizing their class
material, and reflecting on their study approaches
(Bembenutty, 2013; Diamond, Barnett, Thomas, & Munro,
2007; Pape, Bell, & Yetkin-Ozdemir, 2013; Weinstein &
Acee, 2013). These are often instructor-facilitated, multi-
faceted, multisession practices that target a host of skills.
However, they do not focus specifically on improving
students’ strategic use of resources for learning. These
intervention designs also tend to be less amenable to
large-scale distribution to students in the absence of
instructor facilitation.
For these reasons, we designed a novel, self-administered
intervention targeting learners’ strategic use of their
resources for learning. Our Strategic Resource Use inter-
vention prompts students to think deliberately about how
to approach their learning effectively with the resources
available to them (e.g., their lecture notes, homework
problems, and instructors’ office hours). This involves
strategizing about how to approach their learning effec-
tively, deliberately choosing the specific resources that
would foster their mastery of the learning content, and
then planning how they would use these resources to
study the class material. Our intervention design com-
bined theory from previous academic self-regulation
interventions (e.g., Bembenutty, 2013; Pape et al., 2013;
Weinstein & Acee, 2013) with the precision and scalabil-
ity of brief social-psychological interventions (Walton,
2014; Yeager & Walton, 2011). Students were able to take
our online intervention on their own.
We tested this intervention in two randomized con-
trolled trials among college students. Our goal was to
investigate whether this key component of self-regulated
learning—strategically reflecting on how to use one’s
resources effectively for learning—causally contributes to
students’ performance and, if so, how it does. We
expected that, relative to students in the control condi-
tion, students in the intervention treatment condition
would perform better in the class by practicing greater
self-reflection about their learning, and thereby use their
resources more effectively while studying. Because we
are not aware of any evidence to suggest that these
behaviors are more or less commonly practiced among
students of different demographic backgrounds and per-
formance levels, we had no directional hypotheses about
whether this intervention would preferentially advantage
one particular group of students over another.
We conducted two randomized field experiments in a
large Midwestern public university. The participants were
undergraduate students enrolled in two separate cohorts
of a spring-semester introductory statistics class. Perfor-
mance in introductory statistics is central to many stu-
dents’ college careers: It is a prerequisite course for a
number of majors in the social sciences, natural sciences,
premedicine track, and business school. It also satisfies a
quantitative skill requirement for undergraduates. All stu-
dents had the same instructor, who was blind to individ-
ual students’ randomly assigned condition, thus con-
trolling for instructional style and content. The two exper-
iments were almost identical, but the second had minor
improvements in wording and additional survey mea-
sures to test for mechanisms.
We conducted an a priori power analysis using previ-
ous cohorts’ performance in the class to estimate the
standard deviation in the planned sample. We used a
significance level of .05 (i.e., a false-positive rate of .05)
and a power criterion of above .80 (1 – β > .80, where β
is the false-negative rate) to plot a power-analysis graph
(see Fig. S1 in the Supplemental Material available
online). We estimated that sample sizes of approximately
200, 100, and 50 would be sufficient to detect differences
between conditions of 1.5, 2.0, and 3.0 percentage points,
respectively, in students’ final course performance. The
instructor projected an enrollment of 200 students per
class. Thus, even with a conservative forecast of a 50%
participation rate, we were confident that each of our
planned studies would be adequately powered to detect
an effect size of at least 2.0 percentage points.
Participants in the two class cohorts had similar demo-
graphic backgrounds (Study 1: mean grade point aver-
age, or GPA = 3.11; 39.9% male, 57.9% female, 2.2%
gender unknown; 63.6% White, 6.4% African American,
20.2% Asian, 2.9% Hispanic; Study 2: mean GPA = 3.17;
32.9% male, 62.3% female, 0.5% other gender, 4.3% gen-
der unknown; 57.6% White, 10.1% African American,
Strategic Resource Use Intervention 3
20.2% Asian, 2.0% Hispanic). All students in the class
were given the opportunity to participate in our surveys
for homework extra credit points before and after each of
their two exams. Table S1 in the Supplemental Material
shows the breakdown of participation for each survey in
each cohort. Our main outcome measures of student per-
formance were students’ final course grades and their
performance on their two class exams.
Individual students were randomly assigned to the
intervention treatment condition or the control condition.
In Study 1, there were 84 students in the treatment group
and 87 in the control group. In Study 2, there were 95 in
the treatment group and 95 in the control group. Random
assignment occurred automatically when students started
their first online preexam survey. Each survey took about
10 to 15 min to complete. For any subsequent surveys,
students were always in the condition to which they had
initially been assigned.
We administered the preexam surveys, which con-
tained either the treatment or control messages, about 10
days before each of the class exams, and closed the sur-
veys about 7 days before the exam date. In consultation
with the course instructor, we deemed this timing to be
the most likely to affect students’ exam preparation
because it gave students enough time to study for the
exam, but it also was not so far in advance as to seem
irrelevant. The majority of students in our two studies
took a preexam survey before each of their two exams
(Study 1: 73.0%; Study 2: 69.1%; for full details of response
rates, see Table S1 in the Supplemental Material).
Postexam surveys were distributed immediately after
students received their exam grades in class and were
open for 2 to 4 days afterward. The majority of students
in each study took part in the two available postexam
surveys after each of their class exams (Study 1: 68.0%;
Study 2: 71.0%). The postexam surveys primarily mea-
sured which resources students had used to study for
their exams, how useful they had found each resource,
and the degree to which they had self-reflected on their
learning throughout the class. These postexam survey
measures were identical across all students regardless of
condition. Our pre- and postexam surveys comprised
multiple questions, including measures other than those
reported here, for the purposes of research that is not the
focus of this article.
Preexam treatment and
control messages
At the start of each preexam survey, all students were
reminded that their upcoming exam was worth 100
points. They were asked to write down their desired
grade on the upcoming exam and to answer three survey
questions about how motivated they were to get that
grade, how important it was to them to achieve that grade,
and how confident they were in achieving that grade.
After this, students in the control condition received a
regular exam reminder that their exam was coming up in
a week and that they should start preparing for it.
Students in the treatment condition received this same
exam reminder and then a brief Strategic Resource Use
exercise. In a nutshell, the Strategic Resource Use exer-
cise prompted students to deliberately consider the
upcoming exam format, which resources would facilitate
their studying, why each resource would be useful, and
how they were planning to use each resource. In the first
part of the intervention, students in the treatment condi-
tion read a message telling them that successful high
achievers use resources strategically when preparing for
exams. After considering the types of questions that they
expected to be tested on in their upcoming exam, stu-
dents then indicated which class resources they wanted
to use (from a list of 15 available) to maximize the effec-
tiveness of their learning. The checklist of class resources
included lecture notes, practice exam questions, text-
book readings, instructor office hours, peer discussions,
private tutoring, and many others (see Appendix S1 in
the Supplemental Material). We collaborated with the
course instructor to design this comprehensive class
resource checklist. When students actively choose their
learning resources while anticipating the kinds of ques-
tions that they will get on the upcoming exam, they think
strategically about which resources they should channel
their efforts toward in order to make their learning
After filling out the checklist, students in the treatment
condition then answered two open-ended response
questions. First, they described why they thought each
chosen resource would be useful for their exam prepara-
tion. This elaborative process is important because it
helps students articulate exactly why each resource will
contribute to their learning and primes them to think
about how they would make use of the resource effec-
tively. Second, students described specific, realistic, and
concrete plans for when, where, and how they would
study with the resources they had chosen. Forming such
goal-directed plans for action makes it more likely that
students will translate their resource-use intentions into
actual behavior (Gollwitzer, 1999; Gollwitzer & Brandstätter,
1997). After all, strategies are important, but they would
be no better than castles in the air if not executed. We
supplemented our instructions for these two open-ended
questions with concrete examples to guide students
through their explanations.
This Strategic Resource Use exercise guided students
to think strategically about how to approach their learn-
ing by considering which learning resources to use,
why each resource would be useful, when to schedule
4 Chen et al.
studying, where to study, and how (or what steps to take)
to study effectively. The first two of these questions con-
stituted the strategic component, whereas the latter three
formed the planning component of our Strategic Re-
source Use intervention—thus addressing both tactical
and implemental parts of learning.
Assessing causal mechanisms
The goal of the Strategic Resource Use intervention was
to have students reflect on how they would learn most
effectively with the resources available in their environ-
ment. We predicted that this strategic reflection would
make students’ resource use more effective during learn-
ing and, therefore, help them perform better in the class.
Self-reflections on how to learn effectively. At the
end of the class in Study 2, we administered an eight-item
Self-Reflection on Learning scale. This scale assessed the
extent to which students adjusted their studying to the
class, thought about how effectively they were learning,
changed the way they were studying when their ap -
proaches were ineffective, and reflected on their perfor-
mance. It included questions such as “I actively tried to
find out what was expected of me to get good grades in
this class,”As I studied for the class, I kept monitoring
whether or not the way I was studying was effective,” and
“After each exam, I thought about how my performance
in class was a result of how I had been doing things” (see
Appendix S2 in the Supplemental Material; α = .80).
These measures were adapted from the metacognitive
self-regulation subscale of the Motivated Strategies for
Learning Questionnaire from Pintrich et al. (1991).
Self-reported effectiveness of resource use. In each
postexam survey in both conditions, we asked students
to indicate which class resources they had actually used
in their exam preparation. The students chose from a
comprehensive list of resources—the same list that stu-
dents in the treatment condition saw in their preexam
surveys. Students rated how useful they had found each
resource that they had used in their exam preparation
(1 = not useful, 5 = extremely useful). These measures
were adapted from the Resource Questionnaire in Brown,
Doughty, Draper, Henderson, and McAteer (1996). We
averaged the students’ usefulness ratings across their two
exams to obtain a proxy of how effectively students were
using their resources for learning in the class.
Secondary emotional and
motivational effects
To examine other psychological processes that may also
have benefited from our intervention, we measured stu-
dents’ preexam negative affect, their perceived control
over their performance, the extent to which they had
planned their studying ahead of time, and how well they
had kept to their plans (Gollwitzer & Brandstätter, 1997;
Locke & Latham, 1990; Pham & Taylor, 1999). Although
these processes were not the primary goals of our inter-
vention, they are also plausible effects of the intervention
that are relevant both to the psychology of effective learn-
ing and to students’ performance.
On their preexam surveys, students rated the negative
affect that they were experiencing with regard to their
upcoming exams. For example, they reported how anx-
ious, nervous, fearful, and stressed they were about their
upcoming exam on a scale from 1 (not at all) to 7
(extremely). Our negative-affect measure was adapted
from Smith and Ellsworth (1987). We averaged students’
responses on the negative-affect questions to calculate a
composite score of negative affect for each exam (αs
ranged from .82 to .91). In their postexam surveys, stu-
dents’ rated the degree of control that they perceived
they had over their exam performance. For example, they
rated how much they agreed with the statements “I
believe that how well I do in this class is mostly under my
control” and “My exam grades are affected by the way I
choose to study for this course,” on a scale from 1 (strongly
disagree) to 6 (strongly agree). In our postexam surveys,
we asked students to rate how much planning they had
done ahead of time (e.g., “How much planning did you
do to prepare yourself for the Exam 2?” 1 = none, 7 = a
great deal) and how well they had followed through with
their plans (e.g., “To be honest, how well did you follow
through with your plans?” 1 = not at all, 7 = extremely well).
Students’ exam and class
performance data
At the end of the class, we obtained students’ class per-
formance data from the instructor, along with their demo-
graphic and prior performance data from the registrar.
Across our two studies, there were no statistically signifi-
cant differences between conditions in students’ prior
performance and preexam motivation levels (see Table
S2 in the Supplemental Material for descriptive statistics
for these variables). Regression analyses showed that
there were no statistically significant differences between
conditions in students’ high school GPAs (Study 1: p =
.939; Study 2: p = .393) and college GPAs before the inter-
vention (Study 1: p = .577; Study 2: p = .557). Across both
cohorts, there were also no statistically significant differ-
ences between conditions in students’ desired grades on
each of their two exams (all ps > .160), their motivation
to achieve their desired grades (all ps > .267), the per-
sonal importance of these desired grades (all ps > .181),
Strategic Resource Use Intervention 5
and their confidence in attaining their desired grades (all
ps > .161).
Treatment effects
We conducted our analyses using three main approaches:
First, we conducted an intent-to-treat analysis (Gupta,
2011; Wertz, 1995) by comparing the performance of all
students randomly assigned to a condition, regardless of
how many surveys they took. This avoided the self-
selection bias potentially introduced by analyzing only
students who completed all the surveys in either condi-
tion. Second, we compared the performance of students
in the treatment and control conditions who took a sur-
vey before each of their two exams (i.e., all treatment
preexam surveys vs. all control preexam surveys). Third,
we considered whether treatment dosage (i.e., the num-
ber of preexam surveys taken) resulted in differential
benefits among those treated.
In both studies, our intent-to-treat analyses found that
students in the treatment condition outperformed those
in the control condition on their final course grades by an
average of one third of a letter grade. In Study 1, students
in the treatment condition performed an average of 3.64
percentage points (95% confidence interval, or CI = [0.28,
7.00]) higher on their final course grades than students in
the control condition (treatment condition: M = 83.90%;
control condition: M = 80.26%), Cohen’s d = 0.33, Welch’s
two-sample1 t(162) = 2.14, p = .034. This performance
advantage was replicated in Study 2, where students in
the treatment condition scored an average of 4.21 per-
centage points (95% CI = [0.97, 7.44]) higher in the class
than did the students in the control condition (treatment
condition: M = 83.44%; control condition: M = 79.23%),
d = 0.37, t(183) = 2.56, p = .011. In both studies, perfor-
mance differences between conditions were significant
on every exam except Exam 1 in Study 1; in that case, the
difference was in the same predicted direction but not
statistically significant at the .05 level (Fig. 1).
We found the same results when we compared the
final course performances of students who received the
full intervention (i.e., a survey before each of their two
exams) against the performance of students in the con-
trol condition who received the same number of control
messages. In both studies, the average between-groups
difference in final course performance was one third of a
letter grade. Compared with students in the control con-
dition, students in the treatment condition attained final
course grades that were, on average, 3.45 percentage
points higher (95% CI = [0.26, 6.65]) in Study 1 (treatment
condition: M = 86.35%; control condition: M = 82.90%),
d = 0.38, t(127) = 2.14, p = .034, and 4.65 percentage
points higher (95% CI = [1.45, 7.85]) in Study 2 (treatment
condition: M = 85.77%; control condition: M = 81.12%),
d = 0.47, t(139) = 2.87, p = .005. Significant performance
differences were also observed on students’ exams, with
Study 1 Study 2
Exam 1 Exam 2 Course Exam 1 Exam 2
Average Grade (%)
Control Condition
Treatment Condition
Fig. 1. Students’ average performances on Exam 1, Exam 2, and the entire course, presented separately for the control and
treatment conditions. Error bars represent 95% confidence intervals of the means for each condition.
6 Chen et al.
the exception of Exam 1 in Study 1; the results for that
exam were in the predicted direction but not statistically
significant (see Fig. S1 in the Supplemental Material).
We found a treatment-dosage effect among students
who had taken the intervention. The majority of treated
students in each study took the treatment twice rather
than once (Study 1: 75.9% twice vs. 24.1% once; Study 2:
70.5% twice vs. 29.5% once). Students in the treatment
condition who took the intervention twice (rather than
once) scored significantly higher on their final course
grades in both Study 1 (mean difference = 10.16 percent-
age points, 95% CI = [5.30, 15.03]), d = 1.11, t(32) = 4.26,
p < .001, and Study 2 (mean difference = 7.90 percentage
points, 95% CI = [2.77, 13.04]), d = 0.81, t(38) = 3.12, p =
To rule out the possibility that these performance dif-
ferences were driven primarily by differences in students’
motivation, we tested whether there were significant dif-
ferences in self-reported motivation between the students
who took the treatment survey before only one exam
and the students who took the treatment survey before
each of their two exams. These motivation variables were
assessed in our preexam surveys: students’ desired grades
on each exam, their self-reported motivation to achieve
their desired grades, the personal importance of their
grades in the course, and their confidence in attaining
their desired grades. There were no statistically signifi-
cant between-groups differences in any of these motiva-
tion variables (all ps > .05). In addition, we found that the
effect of the number of treatment dosages received
remained statistically significant even when we controlled
for students’ GPA at the beginning of the class in Study 1,
b = 6.24, 95% CI = [1.85, 10.63], SE = 2.21, t(81) = 2.83,
p = .006, and in Study 2, b = 6.17, 95% CI = [2.35, 9.99],
SE = 1.92, t(90) = 3.21, p = .002. GPA is a performance
index that is often associated with students’ motivation to
learn and do well academically.
We repeated our three analytical approach es by
excluding the homework extra-credit points that students
attained for participating in our surveys. We obtained the
same between-conditions differences in students’ exam
and final course performances in all three analyses.
In summary, we concluded that students benefited
from doing the intervention exercise compared with get-
ting a regular exam reminder, and that greater exposure
to the intervention was associated with higher perfor-
mance in the class.
Treatment homogeneity
We found that the Strategic Resource Use intervention
was academically advantageous for different types of
college students across the demographic and perfor-
mance variables that we had collected (i.e., gender, race,
class standing, and preintervention performance levels).
Moderation analyses showed that there were no statisti-
cally significant differences in the treatment effect
between males and females, among students of different
racial groups, among students of different class stand-
ings, and between low- and high-performing students in
both cohorts (all interaction ps > .188). Model compari-
sons further reinforced these results: We pooled data
across both studies and compared one model specifying
all the interactions between condition and individual dif-
ference variables (gender, race, class standing, preinter-
vention GPA, and cohort) with another model without
the interactions (i.e., with only the main effects). Results
from the two models were not statistically different (p =
.369), which implied that the more parsimonious model
without interactions was sufficient to explain the data.
These results support our inference that the intervention
did not provide greater benefit to one kind of student
compared with another.
Causal process
We tested our prediction that, compared with the
business-as-usual control message, our Strategic Resource
Use intervention would affect students’ performance
through greater self-reflection on their learning and more
effective resource-use behaviors, in that order. Aggregat-
ing the available data in our two studies, we first ran
regression analyses to test for the predicted relationships
among our variables. Students who had received the
treatment reported practicing significantly more self-
reflection on their learning in class, b = 0.21, 95% CI =
[0.03, 0.38], SE = 0.09, t(163) = 2.38, p = .019. The more
students thought strategically about how to effectively
approach their learning, the more useful they found the
resources they had used for studying, b = 0.22, 95% CI =
[0.08, 0.36], SE = 0.07, t(145) = 3.12, p = .002, and this
predicted how well they performed in the class, b = 2.71,
95% CI = [0.44, 4.98], SE = 1.15, t(265) = 2.35, p = .019.
There was no direct effect of condition on students’
resource-use behaviors (p = .418).
The treatment effect was not driven by students in the
treatment condition using a greater number of resources
than students in the control condition. If anything, stu-
dents in the treatment condition used fewer learning
resources on average (treatment: M = 11.76; control: M =
13.42; difference between means = 1.66, 95% CI = [0.44,
2.88]), d = 0.33, t(261) = 2.67, p = .008. This result suggests
that the intervention made students use their resources
more effectively—by getting them to self-reflect more
about how they were approaching their learning, rather
than just getting them to use a greater number of resources.
We tested our serial mediation model, aggregating
across the data in both studies, using Mplus (Version 7.4;
Strategic Resource Use Intervention 7
Muthén & Muthén, 2015) with 10,000 bias-corrected
bootstrap resamples to estimate the indirect effect. This
bias-corrected bootstrap method is preferable to the
Sobel test because it corrects for any nonnormality in the
distributions of the variables and their product term when
computing the indirect effect (MacKinnon, Lockwood, &
Williams, 2004). Our serial mediation model is repre-
sented in Figure 2.
There was a significant indirect effect through stu-
dents’ self-reflection about their learning and the reported
effectiveness of their resource use (in that order), which
explained how our intervention affected students’ perfor-
mance, indirect effect b = 0.20, bias-corrected bootstrap
95% CI = [0.02, 0.69]. Goodness-of-fit statistics showed
that our predicted model was an excellent fit to the data,
χ2(2) = 1.02, p = .601, root-mean-square error of approxi-
mation (RMSEA) = 0.00, comparative fit index (CFI) =
1.00, standardized root-mean-square residual (SRMR) =
0.020 (Hu & Bentler, 1999).
We compared our predicted model with a more com-
plex, saturated model that included two additional path-
ways—one with students’ condition predicting their
reported resource-use effectiveness and another using
students’ self-reflections about learning to predict their
final grades. A χ2 difference test showed that the more
complex, saturated model did not do a better job of
explaining the data, Δχ2 = 1.02, Δdf = 2, p = .601. There-
fore, according to Occam’s razor and the parsimony prin-
ciple in structural equation modeling (Kelloway, 1998;
Kline, 2016), our simpler predicted model is preferable to
the more complex, saturated model. Moreover, neither of
the two additional pathways in the saturated model was
statistically significant (both ps > .290), further supporting
our rationale for excluding them. We also ruled out alter-
native models that did not fit our data well, such as a
serial mediation model with the mediators in the oppo-
site order (students’ reported resource-use effectiveness
preceding their self-reflections about learning), and a
parallel mediation model with students’ reported resource-
use effectiveness and their self-reflections about learning
as parallel mediators of the treatment effect. Goodness-
of-fit statistics for these alternative models are presented
in Table S3 in the Supplemental Material.
Although a single mediator model revealed a weak
indirect effect of students’ self-reflections about learning,
b = 0.02, 95% CI = [0.001, 1.65], our predicted serial medi-
ation model is a more theoretically accurate representa-
tion of the process that the intervention targeted. The
intervention was designed to guide students to strategize
how they could learn effectively with the resources that
they had and thereby change how effectively they used
their resources to study. Thus, of these plausible process
models, the model proposed in Figure 2 best captured
the causal process by which our Strategic Resource Use
intervention benefited students’ performance.
Exam-focused resource selection
and follow-through in the treatment
To further understand how the intervention translated
into benefits for students in the treatment condition, we
asked the following questions: What were the perfor-
mance benefits of using resources that students had stra-
tegically selected ahead of time rather than those that
they had not selected in advance but ended up using?
How much did the benefits of planning resource use
depend on actually following through with these plans?
To address these questions, we matched the resources
that every student had selected and planned to use before
their exams with their postexam resource-use responses.
We aggregated across all exams in both studies and used
mixed-effects models with exam number, individual stu-
dent, and cohort included as random effects.
Reported Resource-
Use Effectiveness
on Learning
(Control vs. Treatment) Final Class Grade
b = 0.27,
95% CI = [0.10, 0.43]
b = 0.21,
95% CI = [0.05, 0.38]
b = 3.42,
95% CI = [0.52, 6.36]
Total Effect: b = 3.94, 95% CI = [1.68, 6.28]
Direct Effect: b = 3.75, 95% CI = [3.75, 6.05]
Fig. 2. Serial mediation model showing the effect of the treatment condition on students’ final course performance, mediated by their self-reflections
on learning and reported effectiveness of their resource use. Condition is coded as follows: control = 0, treatment = 1. CI = confidence interval.
Residual error terms are not included in this figure.
8 Chen et al.
Importance of strategic selection in resource use. We
tested the contribution of strategic selection to students’
grades by comparing the degree to which treatment
group students’ exam performance was explained by the
number of resources that they had strategically selected
and used versus the number of resources they had not
selected a priori but ended up using. Both of these vari-
ables were added as fixed-effects predictors in our mixed-
effects model. Only the number of resources that students
had strategically selected in advance and used was posi-
tively related to their exam performance, b = 0.77, 95%
CI = [0.33, 1.21], SE = 0.22, t(241) = 3.48, p < .001; the
number of resources that they used but had not selected
in the intervention ahead of time did not significantly
predict their exam performance (p = .382). Thus, within
the same model, the resources that students had strategi-
cally selected through our intervention exercise predicted
students’ exam performance, but not those that they used
without such strategic forethought.
Follow-through with plans. Were students’ exam per-
formances influenced by their degree of follow-through
with their resource-use plans? Note that individual stu-
dents differed in the total number of resources that they
planned to use, as well as in the number of those planned
resources that they ended up using. We ran a mixed-
effects model predicting students’ exam performance
with three fixed-effects predictors: the total number of
resources that treatment-condition students had planned
to use, the number of those resources that they actually
used, and the interaction between these two regressors.
Note that the total number of resources that students
planned to use included the number of resources that
they had planned to use and actually did use, as well as
the number of resources that they had planned to use but
did not end up using. The interaction allowed us to
model the effect of follow-through across different num-
bers of resources that students had planned to use.
There was a significant negative interaction between
the total number of resources that treatment-condition
students had planned to use and the number of those
resources that they actually used, b = −0.13, 95% CI =
[−0.24, −0.02], SE = 0.05, t(219) = −2.35, p = .020. In addi-
tion, the number of planned resources that students actu-
ally used had a significant simple effect on their exam
performance, b = 1.82, 95% CI = [0.71, 2.94], SE = 0.56,
t(217) = 3.25, p = .001; however, there was no significant
simple effect of the total number of resources that they
had planned to use (p = .367). Our results suggest that
merely strategically planning which resources would be
useful did not, by itself, automatically boost students’
grades—improvements in performance also required put-
ting these strategic plans into practice (Gollwitzer, 1999).
In addition, the significant interaction indicates that
students’ use of resources conferred decreasing marginal
benefit as the total number of resources that students
planned to use increased (i.e., planning to use one addi-
tional resource conferred greater benefit when it was the
4th resource than when it was the 14th). In other words,
planning to use more resources conferred performance
benefits to the extent that (a) individuals followed through
on using those resources and (b) the scope of planning
stayed within reasonably practical bounds rather than
being indiscriminate.
Additional emotional and
motivational benefits
We examined additional consequences of participation in
the intervention, including its effects on students’ pre-
exam negative affect, students’ perceived control over
their exam performance, students’ self-reported prepara-
tory preexam planning, and the degree to which students
followed through with their plans. We aggregated the
data across all exams in both studies and used mixed-
effects models to test for differences on each of these
variables by condition, including exam number, individ-
ual student, and cohort as random effects. Relative to
students in the control condition, students in the treat-
ment condition experienced lower negative affect toward
their upcoming exams, b = −0.43, 95% CI = [−0.73, −0.14],
SE = 0.15, t(353) = −2.88, p = .004, and perceived greater
control over their own performance in the class (although
this effect was marginally significant), b = 0.16, 95% CI =
[−0.01, 0.33], SE = 0.09, t(347) = 1.86, p = .064. Neither
students’ subjective degree of prior planning (p = .492)
nor the degree to which they felt that they had followed
through with their plans (p = .381) significantly differed
between conditions.
Students’ open-ended responses
To understand which psychological elements of the inter-
vention predicted students’ class performance, we coded
and analyzed students’ open-ended responses about why
each resource they had chosen would be useful to them
and their exam-preparation plans. Note that this was
done only for students in the treatment condition who
had answered these open-ended questions; students in
the control condition were not exposed to these ques-
tions. Examples of students’ open-ended responses are
provided in Appendix S3 of the Supplemental Material.
Students’ explanations about why their chosen resources
would be useful were coded into five main psychological
processes that are consistent with self-regulation theory:
(a) explicit consideration of the exam format, (b) leverag-
ing multiple resources in a synergistic manner, (c) foster-
ing learning and understanding of the class material,
Strategic Resource Use Intervention 9
(d) illustrating an understanding of personal strengths and
weaknesses, and (e) recognizing that learning is a social
process (as opposed to an individual’s isolated endeavor).
Two independent coders categorized students’ open-
ended responses into these five categories (interrater κ
ranged from .88 to 1.00), and any disagreements were
resolved through discussion. Students’ plans were similarly
coded into the following three planning categories: when,
where, and how the resources were going to be used
(interrater κ ranged from .94 to 1.00). We created a mea-
sure of the extent to which students engaged in each cat-
egory of psychological processes across their two exams
(0 = the student did not mention it at all, 1 = the student
wrote about it only before one exam, and 2 = the student
wrote about it before both exams).
We regressed the final course performance of students
in the treatment condition on this measure of engagement
separately for each of these eight categories (for results,
see Table 1). Four psychological elements of the interven-
tion significantly and consistently related to students’ final
course performance across our two studies: explicitly tai-
loring one’s choice of resources to the exam questions
anticipated, focusing resource use on building better
learning and understanding of the content, planning
when to use the resources, and planning how to use their
resources to study (Table 1). For example, as students
chose their resources, those in the treatment condition
who were more engaged in reflecting on what was
expected of them on their exams tended to perform bet-
ter in the class. These results emphasize that strategic self-
reflection and planning are both important psychological
processes that are activated by the intervention and con-
tribute to the benefits learners derive from it.
General Discussion
Goal achievement is not always about having more
resources; it is also about how effectively people use
their resources. Regardless of how richly we endow stu-
dents with study materials, support, and environments
conducive to learning, many of these resources will be
wasted on students who do not thoughtfully use them in
a productive manner. Encouraging students to be strate-
gic in their use of class resources to master the class
material enables them to leverage more of their potential
during performance.
We showed that a brief, self-administered intervention
that guided students to make strategic use of their avail-
able resources had a significant impact on their grades.
Across two studies, our intervention produced a difference
Table 1. Results From Regression Analyses Testing the Extent to Which Engagement in Each
Psychological Process Was Associated With Students’ Final Course Grades
Study and psychological process b95% CI t p
Self-regulation categories
Study 1
Consideration of exam format 3.76 [1.07, 6.45] 2.78 .007
Synergistic use with other resources 1.36 [−1.57, 4.29] 0.92 .359
Learning and understanding the class material 4.98 [1.04, 8.92] 2.51 .014
Understanding personal strengths and weaknesses 1.38 [−1.55, 4.30] 0.94 .352
Learning as a social process 0.16 [−2.08, 2.41] 0.14 .886
Study 2
Consideration of exam format 3.22 [0.62, 5.83] 2.46 .016
Synergistic use with other resources 1.79 [−1.03, 4.61] 1.26 .211
Learning and understanding the class material 8.18 [4.85, 11.50] 4.88 < .001
Understanding personal strengths and weaknesses 2.44 [−1.52, 6.40] 1.22 .224
Learning as a social process −0.41 [−2.57, 1.74] −0.38 .706
Planning categories
Study 1
When 4.33 [0.79, 7.87] 2.43 .017
Where 3.16 [0.31, 6.00] 2.20 .030
How 5.67 [2.50, 8.83] 3.56 < .001
Study 2
When 4.97 [1.66, 8.29] 2.98 .004
Where −0.19 [−3.40, 3.01] −0.12 .906
How 6.71 [3.53, 9.89] 4.19 < .001
Note: CI = confidence interval. The degrees of freedom for all Welch’s two-sample t tests was 85 in Study 1 and 93 in
Study 2.
10 Chen et al.
of one third of a letter grade, on average, in a college class
that is a prerequisite to many college majors. Our inter-
vention promoted students’ performance by fostering
greater self-reflection about how best to approach their
learning in class, which directed more effective resource
use while studying. In addition to performing better,
students in the treatment condition also reaped other
psychological benefits: They experienced lower negative
affect toward their upcoming exams and perceived
greater control over their performance, relative to stu-
dents in the control condition. These secondary benefits
add to the value in offering this brief, online intervention
to students.
It was unlikely that the between-conditions perfor-
mance differences that we observed were due simply to
increased awareness of the resources available. In this
class, the nature of instruction involved multiple remind-
ers about what students should be doing each week,
including the resources they could use for learning. For
example, all students, regardless of condition, received a
“Get Things Done” list (see Appendix S4 in the Supple-
mental Material) in their e-mail inbox every week. It is
therefore likely that students in the treatment condition
benefited from greater self-reflection, and thereby more
effective resource use, rather than simply receiving more
reminders of the resources available.
Our results suggest that the process that the interven-
tion sets in motion goes beyond just planning (e.g.,
Gollwitzer & Brandstätter, 1997; Kirschenbaum, Hum-
phrey, & Malett, 1981). It triggers general self-reflection
about how effectively students are approaching their
learning, such as thinking about how productive their
learning approaches are and reflecting on how they have
been learning. This self-reflection directs learners’ efforts,
which makes their resource use more effective during
learning, rather than just strengthening the likelihood
that they will enact their resource-use intentions. Stu-
dents’ open-ended responses also showed that it was
more than planning the use of one’s resources that related
to better class performance: The self-regulatory processes
of selecting resources strategically in light of the antici-
pated exam format, and doing so in a manner that would
maximize content mastery, also significantly contributed
to students’ performance.
In our studies, the benefits of this Strategic Resource
Use intervention were not limited to students of a par-
ticular demographic background or performance level.
Although this relatively homogeneous benefit may
seem somewhat surprising in light of other interven-
tions that specifically target a particular group of students
(e.g., Cohen et al., 2006; Hulleman & Harackiewicz,
2009; Walton & Cohen, 2011), our intervention seemed
to foster a general approach to learning that many
students were generally either unaware of or not prac-
ticing optimally.
Because it changes the way that students strategize
about how to use existing resources, our intervention
may provide the greatest benefit for motivated students
in resource-rich learning environments. In learning envi-
ronments with scarce resources, it may be more pertinent
to ensure that a basic repertoire of resources is available
for learners to use, even before confronting the problem
of how effectively they are making use of what is avail-
able. But in the many learning contexts with an already-
existing assortment of resources, it is valuable for students
to self-reflect about how they should effectively use their
resources to learn, rather than doing so inefficiently.
Moreover, this intervention may confer performance ben-
efits to students to the extent that they are not already
practicing these skills effectively on their own. Its bene-
fits may not be as large for students who are already very
self-reflective about how they use their resources for
learning. Future research should continue to address
whether there are conditions under which the interven-
tion leads to more versus less benefits for different kinds
of learners.
We showed that the intervention brought about other
psychological benefits in addition to improvements in
class grades. However, the intervention may have had
other effects on students that we did not measure. For
instance, it may have influenced how much time students
spent studying. Past literature suggests that the relation-
ship between academic performance and the time stu-
dents spend studying is tenuous (Plant, Ericsson, Hill, &
Asberg, 2005; Schuman, Walsh, Olson, & Etheridge, 1985).
This relation tends to be qualified by how effectively they
spend their study time (Plant et al., 2005; Schuman et al.,
1985). Here, we focused on measuring how effectively
students reported using their resources for learning,
rather than just their sheer amount of studying. Nonethe-
less, it is plausible that the intervention may increase stu-
dents’ study duration or even the way they distribute
their study time (Cepeda, Pashler, Vul, Wixted, & Rohrer,
2006)—questions that future research could look into
testing conclusively.
In this research, we faced the challenge of assessing
students’ self-reflections about their learning and the
effectiveness of their resource use without affecting the
learning process as it was taking place. For instance, ask-
ing students to evaluate the effectiveness of their resource
use as they are studying might influence them to change
their own study practices in the moment, thereby render-
ing such measurements invalid. This also potentially
undermines the intervention itself. To avoid these com-
plications, we chose to measure students’ retrospective
self-reports about these learning behaviors. However, this
Strategic Resource Use Intervention 11
self-report approach has inherent weaknesses, too—such
as potential discrepancies between students’ reports and
their actual behaviors. Assessing students’ self-reflections
about their learning and their resource-use effectiveness
in the moment, without influencing the learning process
itself, should be a goal of future research. This may be
possible with less invasive measurement methods than
those used in the current study.
Our Strategic Resource Use intervention combines
psychologically precise design with an easily scalable
self-administration format (e.g., Paunesku et al., 2015;
Walton, 2014). We made the design self-explanatory, con-
cise, and easily accessible via the Internet so that learners
can autonomously initiate the intervention to improve
the way they approach their learning. Thus, the interven-
tion is amenable to convenient, large-scale application in
schools, and even potentially for online learners taking
massive open online courses.
Beyond education, there are many other situations in
real life in which people engage in goal pursuit ineffec-
tively, especially when they are not aware of how unpro-
ductive their strategies are or how to make the most of
the resources around them. Encouraging self-reflection in
people about how to approach their goals strategically
with the resources that are available to them can go a
long way in helping them to achieve their goals. Through
psychologically precise, self-administered interventions,
such as the Strategic Resource Use intervention, people
can be empowered to take control of their goal pursuit in
a strategic and effective manner.
Action Editor
D. Stephen Lindsay served as action editor for this article.
Author Contributions
P. Chen, O. Chavez, and B. Gunderson were involved in the
study design. P. Chen led the theoretical conceptualization and
experimental design. O. Chavez conducted the power analyses
before the studies were run. B. Gunderson provided the instruc-
tor’s perspective that informed the study design, along with the
opportunity and support to conduct the experiments in her
classroom. P. Chen conducted the statistical analyses in consul-
tation with O. Chavez and D. C. Ong, both of whom contrib-
uted to the statistical coding and analyses. P. Chen wrote the
manuscript, and all the authors provided feedback. All the
authors approved the final version of the manuscript for
We thank the following people, whose feedback and assistance
have made this project possible: Stuart A. Karabenick, Jared
Tritz, Carol S. Dweck, Gregory M. Walton, Thomas N. Robinson,
Teck Sheng Tan, Michelle S. Y. Lee, Patty Kuo, Amiram Vinokur,
and all the students who supported our research.
Declaration of Conflicting Interests
The authors declared that they had no conflicts of interest with
respect to their authorship or the publication of this article.
Supplemental Material
Additional supporting information can be found at http://
Open Practices
We support open, collaborative science conducted in a mutu-
ally respectful and responsible manner. We have posted our
data, code, and materials at These materials
are accessible by researchers and practitioners upon request.
To request access, please complete and submit the request
form at We want
to work with researchers and practitioners to improve student
learning and are happy to answer any questions about the
materials. The complete Open Practices Disclosure for this arti-
cle can be found at
1. We used Welch’s two-sample t tests for all comparisons
between conditions presented in this article.
Barkley, E. F., Cross, K. P., & Major, C. H. (2014). Collaborative
learning techniques: A handbook for college faculty. San
Francisco, CA: John Wiley & Sons.
Bembenutty, H. (2013). The triumph of homework comple-
tion through a learning academy of self-regulation. In H.
Bembenutty, T. J. Cleary, & A. Kitsantas (Eds.), Applications
of self-regulated learning across diverse disciplines (pp.
153–196). Charlotte, NC: Information Age.
Boekaerts, M. (1999). Self-regulated learning: Where we are
today. International Journal of Educational Research, 31,
Brown, M. I., Doughty, G. F., Draper, S. W., Henderson, F. P.,
& McAteer, E. (1996). Measuring learning resource use.
Computers & Education, 27, 103–113.
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D.
(2006). Distributed practice in verbal recall tasks: A review
and quantitative synthesis. Psychological Bulletin, 132,
Cohen, G. L., Garcia, J., Apfel, N., & Master, A. (2006). Reducing
the racial achievement gap: A social-psychological inter-
vention. Science, 313, 1307–1310.
Cuban, L., & Cuban, L. (2009). Oversold and underused:
Computers in the classroom. Cambridge, MA: Harvard
University Press.
Diamond, A., Barnett, W. S., Thomas, J., & Munro, S. (2007).
Preschool program improves cognitive control. Science,
318, 1387–1388.
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., &
Willingham, D. T. (2013). Improving students’ learning
with effective learning techniques: Promising directions
12 Chen et al.
from cognitive and educational psychology. Psychological
Science in the Public Interest, 14, 4–58.
Gollwitzer, P. M. (1999). Implementation intentions: Strong
effects of simple plans. American Psychologist, 54, 493–503.
Gollwitzer, P. M., & Brandstätter, V. (1997). Implementation
intentions and effective goal pursuit. Journal of Personality
and Social Psychology, 73, 186–199.
Gupta, S. K. (2011). Intention-to-treat concept: A review.
Perspectives in Clinical Research, 2, 109–112.
Hanushek, E. A. (1997). Assessing the effects of school
resources on student performance: An update. Educational
Evaluation and Policy Analysis, 19, 141–164.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes
in covariance structure analysis: Conventional criteria ver-
sus new alternatives. Structural Equation Modeling: A Multi-
disciplinary Journal, 6, 1–55.
Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting
interest and performance in high school science classes.
Science, 326, 1410–1412.
Karabenick, S. A. (2003). Seeking help in large college classes:
A person-centered approach. Contemporary Educational
Psychology, 28, 37–58.
Kelloway, E. K. (1998). Using LISERL for structural equation
modeling: A researcher’s guide. Thousand Oaks, CA: Sage.
Kirschenbaum, D. S., Humphrey, L. L., & Malett, S. D. (1981).
Specificity of planning in adult self-control: An applied
investigation. Journal of Personality and Social Psychology,
40, 941–950.
Kline, R. B. (2016). Principles and practice of structural equa-
tion modeling. New York, NY: Guilford Press.
Locke, E. A., & Latham, G. P. (1990). A theory of goal set-
ting and task performance. Englewood Cliffs, NJ: Prentice
MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004).
Confidence limits for the indirect effect: Distribution of the
product and resampling methods. Multivariate Behavioral
Research, 39, 99–128.
Muthén, L. K., & Muthén, B. O. (2015). Mplus user’s guide (7th
ed.). Los Angeles, CA: Author.
Pape, S. J., Bell, C. V., & Yetkin-Ozdemir, I. E. (2013).
Sequencing components of mathematics lessons to maxi-
mize development of self-regulation: Theory, practice, and
intervention. In H. Bembenutty, T. J. Cleary, & A. Kitsantas
(Eds.), Applications of self-regulated learning across diverse
disciplines (pp. 29–58). Charlotte, NC: Information Age.
Paunesku, D., Walton, G. M., Romero, C., Smith, E. N., Yeager,
D. S., & Dweck, C. S. (2015). Mind-set interventions are
a scalable treatment for academic underachievement.
Psychological Science, 26, 784–793.
Pham, L. B., & Taylor, S. E. (1999). From thought to action:
Effects of process-versus outcome-based mental simula-
tions on performance. Personality and Social Psychology
Bulletin, 25, 250–260.
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-
regulated learning components of classroom academic per-
formance. Journal of Educational Psychology, 82, 33–40.
Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J.
(1991). A manual for the use of the Motivated Strategies for
Learning Questionnaire (MSLQ). Retrieved from http://eric
Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J.
(1993). Reliability and predictive validity of the Motivated
Strategies for Learning Questionnaire (MSLQ). Educational
and Psychological Measurement, 53, 801–813.
Plant, E. A., Ericsson, K. A., Hill, L., & Asberg, K. (2005).
Why study time does not predict grade point average
across college students: Implications of deliberate prac-
tice for academic performance. Contemporary Educational
Psychology, 30, 96–116.
Schuman, H., Walsh, E., Olson, C., & Etheridge, B. (1985).
Effort and reward: The assumption that college grades are
affected by quantity of study. Social Forces, 63, 945–966.
Smith, C. A., & Ellsworth, P. C. (1987). Patterns of appraisal and
emotion related to taking an exam. Journal of Personality
and Social Psychology, 52, 475–488.
Walton, G. M. (2014). The new science of wise psychological
interventions. Current Directions in Psychological Science,
23, 73–82.
Walton, G. M., & Cohen, G. L. (2011). A brief social-belonging
intervention improves academic and health outcomes of
minority students. Science, 331, 1447–1451.
Weinstein, C. E., & Acee, T. W. (2013). Helping college students
become more strategic and self-regulated learners. In H.
Bembenutty, T. J. Cleary, & A. Kitsantas (Eds.), Applications
of self-regulated learning across diverse disciplines (pp.
197–236). Charlotte, NC: Information Age.
Wertz, R. T. (1995). Intention to treat: Once randomized, always
analyzed. Clinical Aphasiology, 23, 57–64.
Yeager, D. S., & Walton, G. M. (2011). Social-psychological
interventions in education: They’re not magic. Review of
Educational Research, 81, 267–301.
Zimmerman, B. J. (2011). Motivational sources and outcomes of
self-regulated learning and performance. In B. J. Zimmerman
& D. H. Schunk (Eds.), Handbook of self-regulation of learn-
ing and performance (pp. 49–64). New York, NY: Routledge.
Zimmerman, B. J., & Martinez-Pons, M. (1988). Construct vali-
dation of a strategy model of student self-regulated learn-
ing. Journal of Educational Psychology, 80, 284–290.
... Some interventions take the form of structural and curricular changes 2 , whereas others are more social-psychological in nature-targeting important mental, emotional, motivational, or social mechanisms of learning, and are often delivered directly to the individual student 3,4 . Successful social-psychological interventions have effectively raised the learning, performance, and well-being of tens of thousands of students across the achievement spectrum in rigorous doubleblind, randomized controlled trials (RCTs) [5][6][7][8][9][10] . ...
... To our knowledge, unlike school-wide program evaluations, there are no rigorous, large-scale naturalistic examinations of such effectiveness of student-level social, affective, or motivational interventions, after they are distributed for students to adopt on their own. There already exists many examples of efficacious social-psychological interventions 3 , established through goldstandard laboratory and classroom experiments-such as the Strategic Resource Use intervention, which was tested in two RCTs with relatively large effect sizes on course grades 6 ; the values affirmation intervention, which was replicated in many RCTs across different age groups and school sites 11,12 ; the socialbelonging intervention, which produced performance and health benefits among minorities across multiple RCTs, including a large field experiment across 21 colleges and universities 1,9 ; and the growth mindset intervention, which robustly replicated over many experiments and was recently tested in a randomized, controlled trial using a nationally representative sample of 65 U.S. high schools 10 . Some of these interventions have even been tested cross-culturally via Massive Open Online Courses (MOOCs) 13,14 . ...
... To study these questions, we adapted a previously-validated Strategic Resource Use intervention, which was designed to increase students' self-reflection about their resource use 6 , into an online app called the "Exam Playbook." We chose the Strategic Resource Use intervention because it had previously been experimentally tested and found efficacious at raising students' course grades by an average of one-third of a letter grade in two double-blind RCTs (total N = 361; effect size Cohen's d = 0.33 and 0.37) at the same university 6,18 ; it was online and self-administered, and therefore could be conveniently modified for testing across a variety of classes; and it benefitted diverse demographic groups of students in prior experiments 6 . Moreover, the university administration was supportive of widely distributing and testing the use of this intervention as a free resource for students. ...
Full-text available
Social-psychological interventions have raised the learning and performance of students in rigorous efficacy trials. Yet, after they are distributed “in the wild” for students to self-administer, there has been little research following up on their translational effectiveness. We used cutting-edge educational technology to tailor, scale up, and track a previously-validated Strategic Resource Use intervention among 12,065 college students in 14 STEM and Economics classes. Students who self-administered this “Exam Playbook” benefitted by an average of 2.17 percentage points (i.e., a standardized effect size of 0.18), compared to non-users. This effect size was 1.65 percentage points when controlling for college entrance exam scores and 1.75 [−1.88] for adding [dropping] the Exam Playbook in stratified matching analyses. Average benefits differed in magnitude by the conduciveness of the class climate (including peer norms and incentives), gender, first-generation status, as well as how often and how early they used the intervention. These findings on how, when, and who naturally adopts these resources address a need to improve prediction, translation, and scalability of social-psychological intervention benefits.
... Self-reflection is linked with improved learning and academic performance [15,16]. Many researchers have studied the role of intervention techniques in self-reflection and academic performance [17,18]. Additionally, researchers showed that being aware of student activities supports self-reflection [19,20]. ...
Full-text available
Student engagement with out-of-class activities is becoming more difficult as students spend fewer hours outside the classroom studying the content. This research developed a mobile educational platform, Dysgu, to provide students with an optimal learning experience outside of the classroom. Dysgu includes social networking and gamification features to increase student engagement. The platform offers interactive auto-graded assessments to help students practice concepts and take tests. Students can see their scores and a summary of the performance of the rest of the class. We used Dysgu for multiple out-of-class activities at two universities with different student demographics for two semesters. The data shows that students obtain better grades when using Dysgu. We also saw more on-time or ahead-of-time submissions with Dysgu. Survey responses indicated several Dysgu features which students found helpful. We conclude that digital educational platforms should consider features to support scaffolding to master the concept, peer influence to keep students engaged, self-reflection to foster critical thinking, and easy adaption of the platform to reduce faculty workload and improve students’ acceptance of the system.
... The question stated earlier asks "Can the RAPAL methodology be successfully adapted and applied to the task of enabling business management students at NEU to develop their entrepreneurial, creative and collaborative skills?" A recent research study, led by Patricia Chen at Stanford University (Chen et al, 2017) emphasized the usefulness and success of the approach and methodology outlined in this paper. The Stanford ...
Conference Paper
Full-text available
The 4 C's of 21st Century Learning (critical thinking, creativity, collaboration, and communication) have become increasingly important for entrepreneurship and employability. This paper reports ongoing research into the implementation of these issues in a work related educational setting. A methodology for enabling students to metacognitively develop their entrepreneurial, creative, and collaborative skills is introduced. Three major themes are integrated. Firstly, an innovative approach for entrepreneurial learning that helps students develop their self-understanding, creativity, and flexibility. Secondly, a series of learning tasks designed to improve collaborative learning is discussed. The third major theme, that of the use of business incubators as learning environments, provides the structure and processes which allow the methodology, techniques, and research to be implemented in a way that can be transferred to the workplace by participants. The result is a research-based, flexible model, useful for both education and training programs. The model can help provide students with necessary skills and initial experience, and companies with graduates and trainees possessing an enriched understanding of the demands of working in the digital and collaborative economy.
... The benefits of planning have been documented across several contexts, including competitive sports (Blumenstein & Orbach, 2020); business organizations (Balarezo & Nielsen, 2017), and academic performance (Chen et al., 2017). Within the investigative interviewing context, previous research has largely focused on the views of investigators towards pre-interview planning, how often investigators plan for interviews and their planning skills. ...
Full-text available
Pre-interview planning is vital in interviews with suspects. Via a questionnaire administered to 596 police investigators in Singapore, the current study examined potential associations between pre-interview planning, interviewing behaviours and interview outcomes. Interviewing behaviours were hypothesised to mediate the relationship between pre-interview planning and interview outcomes. It is posited that pre-interview planning fosters an investigative mindset, which in turn, influences the nature of interviewing behaviours employed by investigators. The study also sought to provide insights into police interviews with suspects in Singapore, given the limited research from Singapore on the topic. Rapport-based interviewing behaviours were found to mediate the relationship between pre-interview planning and positive interview outcomes, contributing empirical support to the importance of pre-interview planning. In addition, accusatorial interviewing behaviours were associated with negative interview outcomes. This study also found that police investigators in Singapore reported frequent planning prior to their interviews and used rapport-based interviewing behaviours with suspects. These behaviours are in line with the interviewing model adopted in Singapore. Regression analyses showed that participants’ endorsement of rapport-based approaches was predicted by investigator experience, confidence, and interview length. Endorsement of pre-planning of interviews was also predicted by investigator confidence and interview length. Implications of these findings are discussed.
... The mindset approach rests on the hypothesis that, if students view initial struggle in an introductory course as an indicator that they lack potential to be successful in the field, then they may not adopt and benefit from the effective study strategies that instructors provide. Meanwhile, students may experience less doubt about their potential and become more strategic about their learning if they attribute struggling in a course to the need to develop better study strategies (see also Chen et al., 2017). ...
Full-text available
Mindset interventions, which shift students' beliefs about classroom experiences, have shown promise for promoting diversity in science, technology, engineering, and mathematics (STEM). Psychologists have emphasized the importance of customizing these interventions to specific courses, but there is not yet a protocol for doing so. We developed a protocol for creating customized "peer-modeled" mindset interventions that elicit advice from former students in videotaped interviews. In intervention activities, clips from these interviews, in which the former students' stories model the changes in thinking about challenge and struggle that helped them succeed in a specific course, are provided to incoming life sciences students. Using this protocol, we developed a customized intervention for three sections of Introductory Biology I at a large university and tested it in a randomized controlled trial (N = 917). The intervention shifted students' attributions for struggle in the class away from a lack of potential to succeed and toward the need to develop a better approach to studying. The intervention also improved students' approaches to studying and sense of belonging and had promising effects on performance and persistence in biology. Effects were pronounced among first-generation college students and underrepresented racial/ethnic minority students, who have been historically underrepresented in the STEM fields.
... Given the range of life outcomes that depend on self-control [24,38] and the increasingly wide availability of educational resources (e.g., Khan Academy), it may be especially cost-effective to empower students to approach self-control more strategically. Possibilities include school curricula in which students learn and practice diverse approaches to self-control [see 3,39], as well as creative collaborations, such as self-regulation skill-building episodes (e.g., Cookie Monster modeling the strategy of looking away from cookies to resist eating them) in Sesame Street programming [40]. Since strategic self-control is also associated with subjective well-being [41], such psychoeducational interventions are an especially promising direction for helping young people thrive [42]. ...
Full-text available
Self-control is often thought to be synonymous with willpower, defined as the direct modulation of impulses in order to do what is best in the long-run. However, research has also identified more strategic approaches to self-control that require less effort than willpower. To date, field research is lacking that compares the efficacy of willpower to strategic self-control for consequential and objectively measured real-world outcomes. In collaboration with the College Board, we surveyed two national samples of high school students about how they motivated themselves to study for the SAT college admission exam. In Study 1 ( N = 5,563), compared to willpower, strategic self-control predicted more hours of SAT practice and higher SAT scores, even when controlling for prior PSAT scores. Additionally, the more self-control strategies students deployed, the higher their SAT scores. Consistent with dose-response curves in other domains, there were positive albeit diminishing marginal returns to additional strategies. Mediation analyses suggest that the benefits of self-control strategies to SAT scores was fully explained by increased practice time. These results were confirmed in Study 2, a preregistered replication with N = 14,259 high school students. Compared to willpower, strategic self-control may be especially beneficial in facilitating the pursuit of goals in high-stakes, real-world situations.
... The third phase is constituted by self-reflection (Chen et al., 2017;Lew & Schmidt, 2011). During this phase students and teachers evaluate the effectiveness of the strategies they employed; and consider if their approaches were fit for purpose in relation to the task criteria and set goals. ...
Student agency in the form of students’ active involvement in developing self-regulated learning skills by setting goals, monitoring, and adjusting their own learning process, is increasingly recognised as a key component of classroom self-assessment among researchers. The purpose of this article is to offer a conceptual and practical framework for scaffolding students’ agentic engagement in formative self-assessment as a co-regulatory, three-phase process centred around students’ competence development within the Zone of Proximate Development (ZPD). Agentic engagement in learning occurs when students make proactive, intentional, and constructive contributions to a learning activity, by offering input and making suggestions. This article explores process, product, and competence dimensions of formative self-assessment. It draws upon data from 256 Australian primary students involved in a one-setting practitioner study, conducted as a writing project in which students used a planning template. The findings show that when the planning template was used to scaffold teacher–learner transactions, a range of both direct and indirect teacher–learner transactions occurred, which were prompted by students’ agentic engagement in their learning. The direct teacher–learner transactions included joint, two-way transactions focused on co-regulation, which were either initiated by the learner or by the teacher. The findings also included examples of one-way teacher–learner transactions that involved interactions in which the transaction was aimed at addressing a specific learning need or challenge. These findings imply that using a self-assessment planning template to foster learning through co-regulation enables both instruction and feedback to occur at the point of need, in a task-specific context within the student’s ZPD.
... They have devised a strategy to combat an attack in the unfortunate but possible event. After the threat has passed, the team follows well-documented procedures to determine what went wrong and what preventative measures should be taken to avoid future occurrences [27]. ...
Full-text available
This paper focused on the effect of cyber security knowledge CSK on employees' personal growth PG in private hospitals in Libya and Yemen. Employees were evaluated for their responses to cyber practice, cyber threats targeting end-users in the healthcare industry (i.e., malware, social engineering, spam, phishing, and ransomware), and cyber awareness. The descriptive analytical approach was used to determine the relationship between the independent CSK and dependent variable PG, the current study took place in March 2021 and was applied to a stratified random sample of 164 managers from the middle and lower management. Results indicated a lack of understanding of cyber threats and recommended mitigations and uncertainty regarding applicable legislation governing electronic patient information; also, the level of CSK & PG in the studied hospitals was low. The findings also show that CSK positively affects employees' PG beyond the demographic differences in respondents. Finally, a set of recommendations for effective-based cyber security implementations and training programs were presented and discussed.
In this study, we investigated primary school students’ self-regulated vocabulary learning (SRVL) behaviours on a mobile app using learning analytics (LA) and their associated English vocabulary learning outcomes. Participants were 44 students in Grade 4 from one class in a primary school in Mainland China. Data collection included log data on the mobile app and pre-, mid- and post- vocabulary tests. Data analysis included LA using agglomerative hierarchical clustering and process-mining techniques to understand primary students’ SRVL behaviours in a mobile learning environment and quantitative data analysis to examine the association between students’ SRVL behaviours and their English vocabulary learning outcomes. The results show that three groups of students’ SRVL behaviours were identified using clustering. In addition, the similarities and differences of the characteristics of students’ SRVL behaviours among the identified three clusters were explored. Finally, the association between the identified three clusters and English vocabulary learning outcomes was discussed. The findings provide researchers, language teachers and learners with theoretical and practice insights into the characteristic of the dynamic SRVL behavioural learning patterns in a mobile learning environment and shed light on future research in making personalised recommendations to learners with different SRVL characteristics.
For decades, calls for an increase in the number of minority teachers have led local, state, and federal policy conversations. However, specific barriers to entry into the teaching profession for Black pre-service teachers have received less attention. Moreover, the minimally existing research on the topic is mixed. Despite being the most affected by barriers to entry into the teaching profession, little research has investigated how barriers specifically impact Black pre-service teachers during the teacher training or Educator Preparation Provider (EPP) process. This study examines one possible cause: licensure exams, of which Black test-takers have had the lowest pass rates of all racial or ethnic demographic groups since the inception of the exam. First, this study will thoroughly review existing literature on the various theoretical barriers to entry for Black pre-service teachers, including coursework, field experiences, and licensure exams. Next, the impact of a licensure exam policy change on Black test taker pass rates in Arkansas will be assessed using various descriptive data and the difference-in-differences estimator methodology. This study hypothesizes a change in licensure exam type has negatively impacted the Black teacher workforce in Arkansas, specifically elementary school teachers. Finally, recommendations for state and federal policy and practice will be discussed.
Full-text available
The most commonly used method to test an indirect effect is to divide the estimate of the indirect effect by its standard error and compare the resulting z statistic with a critical value from the standard normal distribution. Confidence limits for the indirect effect are also typically based on critical values from the standard normal distribution. This article uses a simulation study to demonstrate that confidence limits are imbalanced because the distribution of the indirect effect is normal only in special cases. Two alternatives for improving the performance of confidence limits for the indirect effect are evaluated: (a) a method based on the distribution of the product of two normal random variables, and (b) resampling methods. In Study 1, confidence limits based on the distribution of the product are more accurate than methods based on an assumed normal distribution but confidence limits are still imbalanced. Study 2 demonstrates that more accurate confidence limits are obtained using resampling methods, with the bias-corrected bootstrap the best method overall.
Full-text available
The efficacy of academic-mind-set interventions has been demonstrated by small-scale, proof-of-concept interventions, generally delivered in person in one school at a time. Whether this approach could be a practical way to raise school achievement on a large scale remains unknown. We therefore delivered brief growth-mind-set and sense-of-purpose interventions through online modules to 1,594 students in 13 geographically diverse high schools. Both interventions were intended to help students persist when they experienced academic difficulty; thus, both were predicted to be most beneficial for poorly performing students. This was the case. Among students at risk of dropping out of high school (one third of the sample), each intervention raised students' semester grade point averages in core academic courses and increased the rate at which students performed satisfactorily in core courses by 6.4 percentage points. We discuss implications for the pipeline from theory to practice and for education reform. © The Author(s) 2015.
A correlational study examined relationships between motivational orientation, self-regulated learning, and classroom academic performance for 173 seventh graders from eight science and seven English classes. A self-report measure of student self-efficacy, intrinsic value, test anxiety, self-regulation, and use of learning strategies was administered, and performance data were obtained from work on classroom assignments. Self-efficacy and intrinsic value were positively related to cognitive engagement and performance. Regression analyses revealed that, depending on the outcome measure, self-regulation, self-efficacy, and test anxiety emerged as the best predictors of performance. Intrinsic value did not have a direct influence on performance but was strongly related to self-regulation and cognitive strategy use, regardless of prior achievement level. The implications of individual differences in motivational orientation for cognitive engagement and self-regulation in the classroom are discussed.
Recent research has suggested strong relations between characteristic patterns of appraisal along emotionally relevant dimensions and the experience of specific emotions. However, this work has relied primarily upon ratings of remembered or imagined past events associated with the experience of relatively pure emotions. The present investigation is an attempt to examine cognitive appraisals and emotions during an emotional event in which subjects experience complex emotional blends. Subjects described both their cognitive appraisals and their emotional states just before taking a college midterm examination and, again, immediately after receiving their grades on the exam. Analysis of the ratings revealed that at both times the majority of subjects who felt emotion experienced complex blends of two or more emotions. Correlation and regression analyses indicated that even in the context of these blends, patterns of appraisal, highly similar to those discovered in our earlier research on remembered emotions (Smith & Ellsworth, 1985), characterized the experience of emotions as they were actually felt.
Citizens complete a survey the day before a major election; a change in the survey items' grammatical structure increases turnout by 11 percentage points. People answer a single question; their romantic relationships improve over several weeks. At-risk students complete a 1-hour reading-and-writing exercise; their grades rise and their health improves for the next 3 years. Each statement may sound outlandishmore science fiction than science. Yet each represents the results of a recent study in psychological science (respectively, Bryan, Walton, Rogers, & Dweck, 2011; Marigold, Holmes, & Ross, 2007, 2010; Walton & Cohen, 2011). These studies have shown, more than one might have thought, that specific psychological processes contribute to major social problems. These processes act as levers in complex systems that give rise to social problems. Precise interventions that alter themwhat I call wise interventionscan produce significant benefits and do so over time. What are wise interventions? How do they work? And how can they help solve social problems?