Content uploaded by Chris Hulleman
Author content
All content in this area was uploaded by Chris Hulleman on Nov 19, 2018
Content may be subject to copyright.
Manuscript In Press at the Journal of Developmental Education, 07-18-18. Please cite as:
Kosovich, J. J., Hulleman, C. S., Phelps, J., & Lee, M. (In press). Improving algebra success with
a utility value intervention. Journal of Developmental Education.
Improving Algebra Success with a Utility Value Intervention
Jeff J. Kosovich & Chris S. Hulleman
University of Virginia
Julie Phelps & Maryke Lee
Valencia College
Jeff J. Kosovich and Chris S. Hulleman, Center for Advanced Study of Teaching and
Learning, University of Virginia; Julie Phelps and Maryke Lee, Valencia College. This research
was supported by the U.S. Department of Education, through grant #R305B090002, to the first
author, by the National Science Foundation Grants (DRL 1252463) to the second author. The
opinions expressed are those of the authors and do not represent views of the Institute, the U.S.
Department of Education, or the National Science Foundation. Jeff Kosovich is currently a Post-
Doctoral Fellow at the Center for Creative Leadership, Greensboro, NC.
Corresponding author contact information:
Jeff. J. Kosovich, Ph.D
Center for Creative Leadership
1 Leadership Place
Greensboro, NC 27410
jjkosy@gmail.com
IMPROVING ALGEBRA SUCCESS 2
Abstract
Pass rates in community college front-door math courses are a national crisis. The current
study adapted a utility value intervention from Hulleman and Harackiewicz (2009) to facilitate
student success in community college math. In a double-blind experimental study (n = 180), we
found a significant effect of the intervention on student pass rates. Further analysis revealed the
intervention primarily improved men’s passing rates by 13% (d = .54), but did not affect
women’s (d = -.15). The current study demonstrates that the utility value intervention can boost
community college math outcomes. Intervention fidelity, practice, theory, and study limitations
are discussed.
IMPROVING ALGEBRA SUCCESS 3
Improving Algebra Success with a Utility Value Intervention
Students’ performance in their courses can drastically impact academic success—
especially when students fail classes early in their academic careers (Silva & White, 2013). This
crisis is highlighted in developmental math classes, including intermediate algebra (the focus of
the current study), where students may take and fail a course numerous times (Bryk, Gomez,
Grunow, & LeMahieu, 2015). At an individual and societal level, an inability to effectively train
students in basic math has consequences for most jobs today (National Science Board, 2006).
One solution proposed to address this problem is to move beyond traditional instruction and
target major drivers of academic failure (Bryk et al., 2015).
A growing body of research suggests that student success can be facilitated through
psychological interventions (Lazowski & Hulleman, 2016; Rosenzweig & Wigfield, 2016;
Yeager & Walton, 2011). The current study focuses on one such intervention based on
perceptions of value. When students believe what they are learning is useful, they are more likely
to be interested in the topic and successful in class (Hulleman, Durik, Schweigert, &
Harackiewicz, 2008). When students’ perceptions of value are increased via self-reflection
activities, their interest and performance increase (Harackiewicz, Canning, Tibbetts, Priniski, &
Hyde, 2015; Hulleman & Harackiewicz, 2009)—a finding that is strongest for low achieving
students (Hulleman, Godes, Hendricks, & Harackiewicz, 2010; Hulleman, Kosovich, Barron, &
Daniel, 2017b). Although value-related motivation interventions have not been studied in math
courses, they hold great potential for helping struggling students.
Developmental Mathematics in Community College
Over the past few decades, studies about the effectiveness of developmental education’s
impact on student success have attracted the attention of researchers and policymakers
IMPROVING ALGEBRA SUCCESS 4
(Melguizo, Kosiewicz, Prather, & Bos, 2014). Thus far, the research shows that about 60% of
community college students are designated as developmental, at-risk, low-achieving,
disadvantaged, non-traditional, skill-deficient, or underprepared for college-level coursework
(Bailey, 2009).
Many community colleges have used their own institutional and national data to
determine that developmental students’ persistence to associate degree within 8 years is around
25% (Bailey, 2009) as compared to non-developmental persistence of about 40%. This statistic
inspired community college leaders, researchers, and policymakers to explore possible solutions
to improve students’ outcomes through college level developmental (math and other) courses
(e.g., Hodara & Jaggars, 2014). Students who test into developmental math courses lack both
content knowledge and the appropriate learning and motivational strategies to succeed (Guy,
Cornick, & Beckford, 2015). In order to address this challenge, educators from across the nation
have been rethinking the first two years of college mathematics (Saxe & Braddy, 2015). What is
it now? What should it be? Independent organizations and colleges have begun developing new
mathematics pathways (Silva & White, 2013) to help students navigate these significant barriers
to success. Among the components built into these pathways are methods for facilitating student
motivation to energize and direct students academic behavior in useful directions. The study
described in this paper focuses on adapting one of these methods to intermediate algebra.
Utility Value Interventions for Student Success
Focusing on the usefulness of a task to promote motivation is an interdisciplinary concept
that spans academic domains in education. Usefulness is referred to by different labels in
numerous psychological theories and is known as utility value (Eccles et al., 1983), perceived
instrumentality (Husman & Lens, 1999; Raynor, 1982), introjected regulation (Deci & Ryan,
IMPROVING ALGEBRA SUCCESS 5
1985), relevance (Assor, Kaplan, & Roth, 2002), and purpose (Yeager et al., 2014). Regardless
of which label is adopted, leveraging the idea is demonstrably effective at improving individual’s
motivation. According to Eccles and colleagues (Eccles et al., 1983), utility value is specifically
defined as the perceived usefulness of a material to one’s future goals. Importantly, utility value
interventions have shown benefits in college statistics (Acee & Weinstein, 2010), college
biomedical science (Brown, Smith, Thoman, Allen, & Muragishi, 2015), college psychology
(Hulleman et al., 2010; Hulleman, et al., 2016), college biology (Harackiewicz et al., 2014), high
school science (Hulleman & Harackiewicz, 2009); and high school math (Gaspard et al., 2015)
by impoving achievement and interest. The fact that utility value interventions have
demonstrated effectiveness across different academic domains and age levels suggests that the
underlying process may be a general motivational mechanism. Because utility value
interventions tend to benefit students most at-risk for under-performance (Durik, Hulleman, &
Harackiewicz, 2015), community college developmental math students are likely to benefit from
making connections between what they are learning and their lives.
Utility value interventions require students to engage with material they are learning and
to connect it with their own lives. Different versions of these interventions have taken somewhat
different approaches for prompting students to develop these connections. For example,
Hulleman and Harackiewicz (Hulleman & Harackiewicz, 2009) asked students to write brief
essays in which they connected course material (from a high school science class) to their own
lives or the lives of people they knew. Their study showed that students who wrote utility value
essays reported greater interest in additional science classes, and performed better in their current
class than students who simply summarized materal. Students who reported lower initial
expectancy were also more likely to benefit from the interventions compared to those with
IMPROVING ALGEBRA SUCCESS 6
relatively higher initial expectancy. The dependence of utility value on students’ incoming
motivational beliefs is significant as it has emerged in several studies (e.g., Hulleman, et al.,
2010; Hulleman & Harackiewicz, 2009).
Hulleman and colleagues’ research was later adapted to nineth grade mathematics in
Germany (Gaspard et al., 2015). The intervention revised the method of fostering utility value by
asking students to evaluate quotes from other students who use math in daily life. Ultimately the
revised intervention was also able to boost student motivation and performance, with some
notable additions. First, Gaspard and colleagues demonstrated sustained effects of the
intervention on performance several months later. Second, they demonstrated that asking
students to evaluate utility value quotes actually resulted in larger improvements on self-reported
motivation. Although Gaspard and colleagues’ (2015) study focuses on 9th grade students, the
mathematics content is similar to developmental math content that covers introductory and/or
intermediate algebra. The results from Hulleman and colleagues (Hulleman & Harackiewicz,
2009; Hulleman, Schrager, Bodmann, & Harackiewicz, 2010) as well as Gaspard and colleagues
(Gaspard et al., 2015) highlighted that students needed to reflect on the utility value of their
course material.
Math instructors may try to convey the value of mathematics, but this approach of
directly informing students about math’s value may not be effective for all students (Carraher &
Schliemann, 2002). In fact, one laboratory study (Canning & Harackiewicz, 2015) showed that
biology students with high initial expectancy benefitted more from directly communicated utility
value than from the original writing activity. At the same time, direct communication of utility
value to low-expectancy students lowered motivation. In contrast, reflective writing activities
were more effective in generating utility value beliefs for individuals with low initial expectancy.
IMPROVING ALGEBRA SUCCESS 7
Students with high initial expectancy sometimes reported lower motivation after such an
intervention and instead benefitted from directly-communicated utility value. Canning and
colleagues (Canning & Harackiewicz, 2015) also highlight this complex pattern of results
because it demonstrates a potential for well-intentioned practices to be harmful. Thus, for
community college students who are thought to display lower motivation and weaker
foundations in content knowledge than non-developmental students, the reflection actitivies may
be crucial for properly adapting the utility value intervention.
Many of the utility value intervention studies were derived from the expectancy-value
framework for achievement motivation in education (Eccles, 1983). It is worth noting that, as the
name might suggest, motivation is determined by a combination of expectancies for success and
subjective task values. On the one hand, expectancies for success refer to an individuals belief
that they can achieve a high level task or goal (e.g., pass a math course). This is to be contrasted
with self-efficacy, which is a more granular construct that focuses on expectancy for a specific
task (e.g., I can solve this word problem). It is worth noting that expectancies are often
operationalized as confidence in one’s ability to succeed. On the other hand, subjective task
values refer to individual’s perception that a task is important, relevant, or interesting in some
way. Utility value represents one dimension of value (the other two including importance value
and intrinsic value). Thus, if an individual believes they can complete a task, and they want to
complete the task, they are more likely to be motivated. Utility value interventions represent a
concrete translation of expectancy-value theory in that study results often demonstrate an
interaction between assignment to a utility value condition and students’ pre-existing levels of
expectancy.
IMPROVING ALGEBRA SUCCESS 8
In the current research, we consolidate important advances from prior utility value
interventions to create a new version adapted for community college developmental math.
Similar to the previous study, we included a writing prompt for students to reflect on the material
(Hulleman & Harackiewicz, 2009). We also included a series of brief descriptions of the utility
of math for various aspects of life (Canning & Harackiewicz, 2015). To support the direct
communication and self-reflection, we included a selection of quotes from prior students to help
scaffold student thinking and provide additional opportunities for them to reflect (Gaspard et al.,
2015). Finally, to improve the potential for connections between math and students’ lives to
occur, we prompted students to write brief essays relating math to their lives (e.g., everday,
future career, and hobbies or interests) (Hulleman et al., 2017b). The goal of these adaptations
was to maximize the number of potential students that could be influenced by the intervention.
The Current Study
In the current study, we assess the newly updated utility value intervention in an
intermediate math course at a two-year college. There were three major research questions that
guided the current study. First, will students participating in the utility value intervention preport
higher utility value than a control group? Second, will students who participate in the
intervention be more likely to pass, or less likely to withdraw, than students in the control group?
Finally, will the effectiveness of the intervention be moderated by students’ initial levels of
competence?
Method
The current study was a longitudinal, double-blind experiment conducted in intermediate
algebra classes at a large urban community college in the southeastern US. Students received
homework or quiz credit for completing intervention activities during a 16-week semester. We
IMPROVING ALGEBRA SUCCESS 9
note that although the intermediate algebra course is considered a three-credit college-level
elective, it is considered a developmental math course at the institution.
Data/Materials
Sample. The total possible sample included 416 students nested within 22 classrooms.
We collected data from students at three time points during the semester; Time 1, Time, 2, and
Time 3. Time 1 included baseline data during the first two weeks of class, Time 2 included the
intervention delivery during the 3rd and 4th weeks of the class, and Time 3 included the
intervention follow-up during the 14th and 15th weeks of the semester. The participation rate was
44% for Time 1, 43% for Time 2, and 39% for Time 3. Approximately 77% of students
participated in at least one of the activities. The experimental sample included the 177 students
who were randomized to a treatment or control group (see the detailed description under
Procedures) at the start of the intervention activity (57% women, 18% black, 37% white, 36%
Hispanic, 59% receiving financial aid). Students were assigned the three study activities via
email from their instructors using a standardized template that included a brief description of the
activity.
Measures. The first and last activities were each a brief self-report questionnaire
designed to capture various aspects of student motivation and attitudes (see Appendix in the
supplemental materials). The questionnaire included measures of perceptions of expectancy in
course performance (i.e, “ How well do you expect to do in this class?”; 3-items, α = .88),
perceived value of learning math (i.e,. “How useful is the course material to your everyday
life?”; 3-items, α = .77), perceived psychological costs of learning (i.e., “ How often do you
sacrifice too many things in order to do well in this class?”; 4-items, α = .82), perceived utility
value of math for one’s future (i.e., “How important is the course material to your future?”; 2-
IMPROVING ALGEBRA SUCCESS 10
items, α = .83), perceived interest in the material (i.e., “ How interested are you in taking more
math classes?”; 3-items, α = .87). Each measure used a 5-point Likert-type scale ranging from 1
(Not at All) to 5 (Extremely). In addition, students also provided some demographic information.
Both questionnaires were expected to take less than 10 minutes on average. The second activity
consisted of a shortened pre-intervention questionnaire measuring expectancy, value, and cost;
the intervention activity (during which students were randomly assigned to the summary or
utility value condition); and a brief post-intervention questionnaire measuring utility value,
interest, and student demographics.
Outcomes. A student was coded as having passed (1) if they received a C or higher as
their final grade, otherwise they were coded not passing (0). Thus, a student may not have passed
because they failed or because they withdrew from the class. A student was coded as having
withdrawn (1) if they a withdraw or not (0).
Procedure. The first (Time 1, weeks 2 and 3) and third (Time 3, weeks 14 and 15)
activities were surveys, and the second activity (Time 2, weeks 6 and 7) was the utility value
activity. At the time of participation, students were randomly assigned to either a summary group
(n = 80) or a utility value group (n = 97); three participants were removed because they did not
complete proceed far enough to be randomized. The summary group was asked to write a short
summary of some of the class material they were learning in a series of three short essays. The
utility value group read a set of quotes by former students who had learned about the utility value
of math. The participants were then asked to write three short essays about the relevance of their
course material to their everyday lives, potential future careers, and hobbies or interests. In total,
students were asked to provide 9-12 sentences worth of writing. As in prior studies (e.g.,
Hulleman & Harackiewicz, 2009), those completing the summary activity were considered part
IMPROVING ALGEBRA SUCCESS 11
of the control group whereas those completing the utility value activity were considered part of
the intervention group.
Analyses. Analyses were conducted using a combination of descriptive statistics, multi-
level multiple regression, and multi-level logistic regression. To improve statistical precision of
outcome analyses, we created motivation covariates by averaging self-report measures of
expectancy, value, and cost from Time 1 and 2 (both questionnaires measured prior to
experimental assignment). We also included a battery of student demographics (i.e., sex, first
time in college, financial aid status, and race/ethnicity) and academic characteristics (i.e.,
developmental mandate, exemption from developmental requirements), as covariates to improve
the precision of our models. In order to reduce missing data - only 44% (Time 1) or 43% (Time
2) of students completed the Time 1 or Time 2 motivation measures – we created an average of
the Time 1 and time 2 motivation measures to create single baseline composites of motivation
for 74% of the sample. These composite covariates were only used for the purposes of statistical
precision in our regression analyses; any discussion of motivation prior to the intervention uses
the individual scores from the appropriate time point. All covariates were group-mean centered.
Intervention Fidelity. An important aspect of understanding an intervention is to assess
the degree to which it demonstrates fidelity—or alignment with expected processes and
functioning (Nelson, Cordray, Hulleman, Darrow, & Sommer, 2012; O’Donnell, 2008). In this
study we examined participant responsiveness, which is the degree to which individuals in the
program engage with materials as expected (Nelson et al., 2012).We assessed the extent to which
students wrote essays that were high quality, relevant to the intervention prompt, and of a desired
length. Two trained raters coded student responses on several dimensions of intervention fidelity
including quality of written utility value (α = .89) and general writing quality (α = .85).
IMPROVING ALGEBRA SUCCESS 12
Results
Pre-Intervention Descriptives
To test whether or not randomization was successful, we conducted a series of
randomization checks based on student characteristics and pre-intervention motivation. The
student characteristics included sex χ2 (1) = 0.84, p = .36; first time in college χ2 (1) = 5.49, p =
.02; mandated developmental courses χ2 (3) = 5.25, p = .15; financial aid status χ2 (1) = 0.07, p =
.79; exempt from developmental requirements, χ2 (1) = 0.67, p = .41; black, χ2 (1) = 0.09, p =
.77; white, χ2 (1) = 0.20, p = .66; Hispanic/latino(a), χ2 (1) = 0.03, p = .87; and other χ2 (1) =
1.82, p = .18. Pre-intervention motivation included expectancy F (1,197) = .015, p = .70, value F
(1,197) = .24, p = .63, and cost F (1,197) = .93, p = .34. Based on the relatively similarity
between conditions across student characteristics and baseline motivation, randomization was
acceptable. However, student demographic characteristics were still included as covariates in the
regression models.
Intermediate Outcomes
Using a hierarchical linear model, we regressed both post-intervention motivation
measures on our treatment indicator and covariates. The intervention had a positive and
significant effect on post-intervention self-reported utility value (Time 2), b = .45, p = .014, d =
.38, controlling for demographic variables and prior motivation. The difference in post-
intervention interest was small, b = .11, p = .57, d = .09 and not statistically significant. No
interaction effects were present between gender and experimental condition in predicting utility
value or interest.
Final Outcomes
IMPROVING ALGEBRA SUCCESS 13
When examining intervention effects on course outcomes, two pairs of logistic
regressions were conducted – two for withdrawal rates and two for pass rates. The first pair of
models tested for an interaction between pre-intervention expectancy and experimental group.
The second pair of models tested for an interaction between sex and experimental group. In
terms of withdrawal rates, the students in the utility value condition exhibited lower withdrawal
rates (d = -.11), but the effect was not statistically significant in either interaction model. There
was however a statistically significant main effect of the intervention on pass rates (b = 3.06, p =
.04), and a marginally significant interaction between experimental group and sex (b = -4.68, p
=.07; see Table 1 for the full model). There was not, however, an interaction between
experimental group and pre-intervention expectancy (b = 1.39, p = .51). Figure 1 presents the
intervention by sex interaction using the raw pass rates from each group, which illustrates the
intervention effect was primarily driven by benefits for male students (d = .54) whereas there
was only a small negative effect for female students (d = -.15). Both the main and interactive
effects remained whether covariates were included in the model or not.
Intervention Fidelity
Overall, there was acceptable treatment differentiation (Nelson et al., 2012) based on
coded utility value quality of students’ essays. Students in both groups demonstrated suboptimal
compliance with the activity’s instructions; each prompt asked students to write three or four
sentences, but only wrote 2.5 sentences were written on average. Students in the utility value
condition were more likely than the control to adhere to essay length, d = 0.45, and essay topic, d
= .47. Students in both groups were approximately equivalent in their reactance (i.e., negative
reaction to the activity), d = 0.00. Furthermore, students in the utility value condition spent more
IMPROVING ALGEBRA SUCCESS 14
time completing the activity, M = 12.0, SD = 7.5, minutes
1
, Median = 9.6 minutes, than the
summary condition, M = 9.6, SD = 8.1, minutes, Median = 6.6 minutes. Individuals in the
utilility value condition demonstrated higher writing quality (d = .75). Students in the utility
value condition were also rated as producing substantially higher utility value in their essays, d =
1.64; the difference was even larger after controlling for activity time and writing quality, b =
1.71, p < .001, d = 2.52.
Discussion
The current research was aimed at improving student success (i.e., pass rates) in
intermediate algebra courses. Students who received a utility value intervention were asked to
reflect on and briefly write about the usefulness of their math course to different aspects of their
lives (i.e., everyday life, future carreer, hobbies/interests). The reflection essays led to higher
self-reported utility value immediately following the intervention compared to control group.
Students who received the utility value intervention, particularly men, were also more likely to
pass their class than those in the control group. The results add to a growing body of literature
showing that utility value interventions can help students across a range of domains and ages.
Importantly, the current research also demonstrates that utility value may work as a lever through
which math instructors can improve student motivation and success. In the following sections,
we first discuss limitations and then discuss theoretical and practical implications.
Limitations
There were two major limitations to this study, the most problematic of which was poor
participation rates (an average of just over 40% at each time point). Unfortunately, this also
constrains the generalizability of this study. Given that students who participated had
1
Note that winsorized means were used because of response time errors (i.e., likely participants failing to click
submit).
IMPROVING ALGEBRA SUCCESS 15
substantially higher final course grades (d = .54, p < .001) than students who did not participate,
it would appear that the intervention did not reach the students most in need. This may also
explain why the interaction from prior research was not replicated. Further discussion with the
instructors provided some illumination to this issue. Generally speaking, the instructors agreed
that the students who did not participate in the activities were also least likely to attend or
participate in class. Future versions of the utility value intervention would benefit from being
more-covertly integrated into the class (i.e., not as an obviously-external activity tied to a study).
Related to the first limitation, we can also not claim an causality from our intervention
fidelity assessments. It is the case that those who showed better fidelity also seemed to benefit
from the intervention; however, the fact is that individuals self-selected their levels of fidelity. It
is possible that individuals who wrote more were simply more conscientious about following
directions and were more likely to improve with any sort of help. We note that randomization
occurred during the intervention, meaning that our causal estimates remain unbiased for the
students who did participate. We can say with confidence that the positive effects of the utility
value intervention are causal in nature. Because the study was only conducted with instructors
who were interested in implementing the utility value intervention, the current study is likely to
mimic real-life adoption of the intervention. Specifically, the intervention was assigned as
homework and, according to the instructors, participation rates in the activity were similar to
participation in other classwork.
Implications for Theory and Research
Whereas prior iterations of the intervention were effective for low-expectancy students
(Hulleman & Harackiewicz, 2009) or low-achieving students (Hulleman, Godes, et al., 2010),
the current study was effective primarily for men. The current research shows that the
IMPROVING ALGEBRA SUCCESS 16
intervention aids at least some struggling students by raising the pass rate among men by
approximately 13% relative to a control group. Although these results do not directly replicate
prior findings, they may be detecting a similar phenomenon using gender as a proxy variable.
Men in the control group displayed the lowest pass rate relative to the women in the control
group and utility value groups. This finding suggests that men represented the majority of low-
achieving students and that gender may have simply functioned as a proxy for prior math
achievement (a variable we were unable to include in the study). However, a follow-up chi-
square test showed that the groups were not statistically different in terms of raw pass rates, χ2
(3) = 5, p = 0.10. Based on these results, further work is needed to understand the exact role that
gender plays in these interventions. It is also possible that this effect can explain why the effect
of the intervention did not depend on pre-intervention expectancy.
Findings similar to the current research were observed in the Hulleman and colleagues
(Hulleman et al., 2017b) study of psychology students where the utility value intervention
prevented a decline in classroom performance during the semester, particularly among men. In
that study, men were the lower achieving group. Unfortunately, it is not possible to tease apart
the role of gender, prior achievement, and perceived expectancy in either of these studies. One
possible explanation is that women tend to report lower expectancy in their math ability than
men despite displaying higher performance (Beilock, Gunderson, Ramirez, & Levine, 2010;
Eccles & Wigfield, 2002). The result is a mismatch between expectancy and actual performance
that could lead to differential effects during analysis. The nature of this mismatch is a topic for
future research as it may illuminate the exact processes through which the intervention functions.
For example, is it the case that the intervention works best for low-achieving individuals
regardless of their expectancy? This may explain why some studies find an interaction between
IMPROVING ALGEBRA SUCCESS 17
experimental condition and achievement, but not expectancy (Hulleman & Harackiewicz, 2009).
Alternatively, is it the case that when expectancy and achievement are matched, the utility value
intervention is able to boost expectancy, which in turn boosts achievement? One recent study by
Hulleman and colleagues (Hulleman et al., 2017b) found that intervention effects depended on
initial exam performance, and that the effects worked through later perceptions of expectancy
(i.e., the intervention boosted expectancy which then boosted performance). Future research
needs examine why gender effects emerge in some cases (Hulleman et al., 2017b) but not others
(Canning & Harackiewicz, 2015; Hulleman & Harackiewicz, 2009).
One possible route for future work is to examine the long-term effects of the intervention
(i.e., future course-taking, graduation rates). Gaspard and colleagues (Gaspard et al., 2015)
demonstrated sustained intervention effects over several weeks, but their intervention was three
to five times longer than the one described in the current research, and it was delivered in person.
Because the research base on motivation interventions in general – and utility value interventions
in particular – have tended to focus on short-term outcomes, we do not have prior research to
consider when predicting the long-term effects of our intervention. This is an important factor to
consider when determining the overall effectiveness of this type of intervention.
It is also worth investigating an intervention where both utility value and expectancy are
manipulated. If it is the case that motivation is a combination of expectancy and value,
presumably a dual-target intervention could produce more powerful outcomes. At least one study
explored such an intervention ( Hulleman, Kosovich, Barron, & Daniel, 2017), but the
expectancy manipulation was not effective. Hulleman and colleagues’ study suggested that some
utility value interventions may increase achievement through impacts on expectancy, which is
IMPROVING ALGEBRA SUCCESS 18
consistent with the reciprocal relationship between expectancies and values (Kosovich, Flake, &
Hulleman, 2017).
Implications for Practice
The intervention tested in the current research is low-cost and brief—once developed it is
free to use and takes students an average of 20 minutes to complete. That such a seemingly
simple activity could improve pass rates may be met with skepticism or written off as common
sense, but it is important to consider what we know about motivation generally and what we
know about utility value specifically. The simple intervention is a product of years of careful
theoretical work combined with rigorous testing based on prior research (Canning &
Harackiewicz, 2015; Hulleman, Godes, et al., 2010; Hulleman & Harackiewicz, 2009; Hulleman
et al., 2017b). As we discussed earlier, the current iteration of the intervention drew from several
prior studies. Motivational interventions such as the utility value intervention from the current
research are thought to work through recursive processes (Yeager & Walton, 2011). The idea
behind recursive processes is that concepts like motivation can be self-propogating under the
right conditions. A student who does not find math useful may disengage from class and focus
more time on other more personally-important endeavors. However, changing the importance of
a math class in the students’ mind can change its position on their priority list. Self-efficacy
research further shows that success-experiences breed greater motivation and success (Bandura
& Schunk, 1981). Value may also foster positive academic emotions (Pekrun, Goetz, Daniels,
Stupnisky, & Perry, 2010), which are another potential antecedent to feelings of efficacy
(Bandura, 1982) as previously discussed in reference to the study by Hulleman and colleagues.
Thus, a simple connection between a class and daily life early in the semester has the potential to
improve a student’s grade (Hulleman & Harackiewicz, 2009).
IMPROVING ALGEBRA SUCCESS 19
How utility value is communicated matters. The idea that students want to see the
importance/relevance/usefulness in their courses is not novel. However, what students deem
useful and what instructors deem useful is not always the same (Carraher & Schliemann, 2002).
One problem that stems from this tension is that instructors may think they are communicating
value for the material but students do not perceive the information as having the same kind of
value. A greater concern is that telling students the material is important may actually harm their
motivation. Indeed, one study conducted in a laboratory setting showed that students with low
expectancy actually decreased in motivation and performance after they were directly told that a
topic was useful (Canning & Harackiewicz, 2015). The study found that directly communicating
utility value was only helpful for students with already-high expectancy. Thus, low-expectancy
students need to find utility on their own. That is not to say that instructors cannot prompt and
guide students to make these discoveries, but the reflection activity is hypothesized to be a
central component of this process (Hulleman, et al., 2010; Hulleman & Harackiewicz, 2009).
Instructors interested in adopting this research should work to help students discover personally
meaningful connections between the coursework and the students’ own lives.
Which connections students make matters. All utility value is not created equal
because students have different life experiences and goals. The extent to which a class is
perceived as important to achieving one’s goals depends on the individual (Vansteenkiste et al.,
2004). The intervention in the current study departed from prior work by explicitly prompting
students to consider utility value for three different possible areas: everyday life, future careers,
and hobbies or interests. The purpose for this adaptation was to give the students more chances
to make meaningful connections to their own personal goals and aspirations. How math is useful
to a math major is going to be very different from how math is useful to an art major. A
IMPROVING ALGEBRA SUCCESS 20
considerable amount of work was done by the current research team to develop intervention
materials that fit in the intermediate algebra context. The quotes that students read as a part of the
intervention were developed to be relatable and related to the context. Practitioners wishing to
adopt the materials from the current study will likely need to update the content to make it fit
their particular context. For example, teaching a general education math class versus a math
class major will require different examples and possibly different terminology.
Beyond writing. Although the writing task used in the current study represents one
scientifically tested method of producing utility value perceptions, it is certainly not the only
way. Gaspard and colleagues (2014) asked students to evaluate quotes about utility as another
method of reflection. It is likely that students can discover utility value in service projects, work
studies, or internships that explicitly tie together course material and real world applications. In
each case, we stress that the current research shows the reflection component (in whatever form
it takes) seems to be an important aspect of the motivational process. Finding such connections is
not necessarily an easy or straightforward process. For example, many students may not realize
that even if they will not use a specific mathemtical concept again it is still useful in improving
their reasoning or problem-solving skills. Thus, educators should feel emboldened to give
students opportunities to discover utility value so long as the students are also given adequate
time and support to genuinely reflect on and find such connections.
Conclusion
In recent years, math educators have been exploring new ways to structure the early years
of mathematics in college. This thinking has been spurred by the high failure rates of students in
such early classes. The current study demonstrates one tool that can be useful for educators in
helping students navigate obstacles. Fostering perceptions of value for students is helpful
IMPROVING ALGEBRA SUCCESS 21
because it relates to positive attitudes toward the course material in the short term and the long
term. At the same time, interventions like the one tested in the current study also boost student
performance—the result is a higher proportion of students passing courses. Though not a curall
for challenges faced by math educators and students, the combination of a students’ value for a
course and expectancy in their abilities can be a powerful motivational force for educators to
shape and direct.
IMPROVING ALGEBRA SUCCESS 22
References
Acee, T. W., & Weinstein, C. E. (2010). Effects of a value-reappraisal intervention on statistics
students’ motivation and performance. The Journal of Experimental Education, 78(4), 487–
512. https://doi.org/10.1080/00220970903352753
Assor, A., Kaplan, H., & Roth, G. (2002). Choice is good, but relevance is excellent: autonomy-
enhancing and suppressing teacher behaviours predicting students’ engagement in
schoolwork. The British Journal of Educational Psychology, 72(2), 261–278.
https://doi.org/10.1348/000709902158883
Bailey, T. (2009). Challenge and opportunity: Rethinking the role and function of developmental
education in community college. New Directions for Community Colleges, (145), 11–30.
https://doi.org/10.1002/cc
Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist.
https://doi.org/10.1037/0003-066X.37.2.122
Bandura, A., & Schunk, D. H. (1981). Cultivating competence, self-efficacy, and intrinsic
interest through proximal self-motivation. Journal of Personality and Social Psychology,
41(3), 586–598. https://doi.org/10.1037/0022-3514.41.3.586
Beilock, S. L., Gunderson, E. A., Ramirez, G., & Levine, S. C. (2010). Female teachers’ math
anxiety affects girls’ math achievement. Proceedings of the National Academy of Sciences
of the United States of America, 107(5), 1860–1863.
https://doi.org/10.1073/pnas.0910967107
Brown, E. R., Smith, J. L., Thoman, D. B., Allen, J. M., & Muragishi, G. (2015). From bench to
bedside: A communal utility value intervention to enhance students’ biomedical science
motivation. Journal of Educational Psychology, 107(4), 1116–1135.
IMPROVING ALGEBRA SUCCESS 23
https://doi.org/10.1037/edu0000033
Bryk, A. S., Gomez, L., Grunow, A., & LeMahieu, P. (2015). Learning to improve: How
America’s schools can get better at getting better. Cambridge, MA: Harvard Education
Publishing.
Canning, E. A., & Harackiewicz, J. M. (2015). Teach it, don’t preach it: The differential effects
of directly-communicated and self-generated utility–value information. Motivation Science,
1(1), 47–71. https://doi.org/10.1037/mot0000015
Carraher, D. W., & Schliemann, A. D. (2002). Is Everyday Mathematics Truly Relevant to
Mathematics Education? Journal for Research in Mathematics Education. Monograph, 11,
131. https://doi.org/10.2307/749968
Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human
behavior. New York: Plenum.
Durik, A. M., Hulleman, C. S., & Harackiewicz, J. M. (2015). One size fits some: Instructional
enhancements to promote interest. In K. A. Renninger, M. Neiswandt, & S. Hidi (Eds.),
Interest, the Self, and K-16 Mathematics and Science Learning (pp. 49–62). Washington,
DC: American Educational Research Association.
Eccles, J. S. (1983). Expectancies, Values, and Academic Behaviors. In J. T. (University of T. at
A. Spence (Ed.), Achievement-Related Motives and Behaviors (pp. 119–146).
Eccles, J. S., Adler, T., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., & Midgley, C.
(1983). Expectancies, Values, and Academic Behaviors. In J. T. Spence (Ed.), Achievement
and achievement motivation (pp. 75–146). San Francisco, CA: W. H. Freeman.
Eccles, J. S., & Wigfield, A. (2002). Motivational Beliefs, Values, and Goals. Annual Review of
Psychology, 53(1), 109–132. https://doi.org/10.1146/annurev.psych.53.100901.135153
IMPROVING ALGEBRA SUCCESS 24
Gaspard, H., Dicke, A., Flunger, B., Brisson, B. M., Häfner, I., Nagengast, B., & Trautwein, U.
(2015). Fostering adolescents’ value beliefs for mathematics with a relevance intervention
in the classroom. Developmental Psychology, 51(9), 1226–1240.
https://doi.org/10.1037/dev0000028
Guy, G. M., Cornick, J., & Beckford, I. (2015). More than Math: On the Affective Domain in
Developmental Mathematics. International Journal for the Scholarship of Teaching and
Learning, 9(2). https://doi.org/10.20429/ijsotl.2015.090207
Harackiewicz, J. M., Canning, E. a., Tibbetts, Y., Giffen, C. J., Blair, S. S., Rouse, D. I., & Hyde,
J. S. (2014). Closing the social class achievement gap for first-generation students in
undergraduate biology. Journal of Educational Psychology, 106(2), 375–389.
https://doi.org/10.1037/a0034679
Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Priniski, S. J., & Hyde, J. S. (2015). Closing
Achievement Gaps With a Utility-Value Intervention: Disentangling Race and Social Class.
Journal of Personality and Social Psychology. https://doi.org/10.1037/pspp0000075
Hodara, M., & Jaggars, S. S. (2014). An Examination of the Impact of Accelerating Community
College Students’ Progression Through Developmental Education. Journal of Higher
Education, 85(2), 246–276. https://doi.org/10.1353/jhe.2014.0006
Hulleman, C. S., Durik, A. M., Schweigert, S. B., & Harackiewicz, J. M. (2008). Task values,
achievement goals, and interest: An integrative analysis. Journal of Educational
Psychology, 100(2), 398–416. https://doi.org/10.1037/0022-0663.100.2.398
Hulleman, C. S., Godes, O., Hendricks, B. L., & Harackiewicz, J. M. (2010). Enhancing interest
and performance with a utility value intervention. Journal of Educational Psychology,
102(4), 880–895. https://doi.org/10.1037/a0019506
IMPROVING ALGEBRA SUCCESS 25
Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting interest and performance in high
school science classes. Science (New York, N.Y.), 326(5958), 1410–1412.
https://doi.org/10.1126/science.1177067
Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2017a). Making connections:
Replicating and extending the utility value intervention in the classroom. Journal of
Educational Psychology, 109(3). https://doi.org/10.1037/edu0000146
Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2017b). Making connections:
Replicating and extending the utility value intervention in the classroom. Journal of
Educational Psychology, 109(3), 387–404. https://doi.org/10.1037/edu0000146
Hulleman, C. S., Schrager, S. M., Bodmann, S. M., & Harackiewicz, J. M. (2010). A meta-
analytic review of achievement goal measures: different labels for the same constructs or
different constructs with similar labels? Psychological Bulletin, 136(3), 422–449.
https://doi.org/10.1037/a0018947
Husman, J., & Lens, W. (1999). The role of the future in student motivation. Educational
Psychologist, 34(2), 113–125. https://doi.org/10.1207/s15326985ep3402_4
Kosovich, J. J., Flake, J. K., & Hulleman, C. S. (2017). Short-term motivation trajectories: A
parallel process model of expectancy-value. Contemporary Educational Psychology, 49.
https://doi.org/10.1016/j.cedpsych.2017.01.004
Lazowski, R. A., & Hulleman, C. S. (2016). Motivation interventions in education: A meta-
analytic review. Review of Educational Research, 86(2), 602–640.
https://doi.org/10.3102/0034654315617832
Melguizo, T., Kosiewicz, H., Prather, G., & Bos, J. (2014). How are community college students
assessed and placed in developmental math? Grounding our understanding in reality. The
IMPROVING ALGEBRA SUCCESS 26
Journal of Higher Education, 85(5), 691–722.
https://doi.org/10.1080/00221546.2014.11777345
Nelson, M. C., Cordray, D. S., Hulleman, C. S., Darrow, C. L., & Sommer, E. C. (2012). A
procedure for assessing intervention fidelity in experiments testing educational and
behavioral interventions. The Journal of Behavioral Health Services & Research, 39(4),
374–396. https://doi.org/10.1007/s11414-012-9295-x
O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation
and its relationship to outcomes in K-12 curriculum intervention research. Review of
Educational Research, 78(1), 33–84. https://doi.org/10.3102/0034654307313793
Pekrun, R., Goetz, T., Daniels, L. M., Stupnisky, R. H., & Perry, R. P. (2010). Boredom in
achievement settings: Exploring control–value antecedents and performance outcomes of a
neglected emotion. Journal of Educational Psychology, 102(3), 531–549.
https://doi.org/10.1037/a0019243
Raynor, J. O. (1982). Future orientation, self-evaluation, and achievement motivation: Use of an
expectancy x value theory of personality functioning and change. In N. T. Feather (Ed.),
Expectations and actions: Expectancy-value models in psychology (pp. 97–124). Hillsdale,
NJ: Erlbaum.
Rosenzweig, E. Q., & Wigfield, A. (2016). STEM motivation interventions for adolescents: A
promising start, but further to go. Educational Psychologist, 51(2), 146–163.
https://doi.org/10.1080/00461520.2016.1154792
Saxe, K., & Braddy, L. (Eds.). (2015). A common vision for undergraduate mathematical
sciences programs in 2025. Washington, D.C.: Mathematical Association of America.
Silva, E., & White, T. (2013). Pathways to improvement: Using psychological strategies to help
IMPROVING ALGEBRA SUCCESS 27
college students master developmental math. Carnegie Foundation for the Advancement of
Teaching. Retrieved from
http://www.carnegiefoundation.org/sites/default/files/pathways_to_improvement.pdf
Vansteenkiste, M., Simons, J., Lens, W., Soenens, B., Matos, L., & Lacante, M. (2004). Less is
sometimes more: Goal content matters. Journal of Educational Psychology, 96(4), 755–764.
https://doi.org/10.1037/0022-0663.96.4.755
Yeager, D. S., Henderson, M. D., Paunesku, D., Walton, G. M., Mello, S. D., Spitzer, B. J., &
Duckworth, A. L. (2014). Boring but important : A self-transcendent purpose for learning
fosters academic self-regulation, 107(4), 559–580.
Yeager, D. S., & Walton, G. M. (2011). Social-psychological interventions in education: They’re
not magic. Review of Educational Research, 81(2), 267–301.
https://doi.org/10.3102/0034654311405999
IMPROVING ALGEBRA SUCCESS 28
Table 1
Multi-Level Logistic Regression for Pass Rates (N = 177)
b
SE
DF
t
p
Intercept
-15.00
8.39
10
-1.79
0.10
Treatment
3.06
1.48
156
2.07
0.04
Mandate
-1.57
0.89
10
-1.76
0.11
Female
1.70
2.59
10
0.65
0.53
Black
0.02
2.84
10
0.01
0.99
Hispanic
-0.34
2.67
10
-0.13
0.90
Other
-0.06
3.43
10
-0.02
0.99
First Time
0.99
3.81
10
0.26
0.80
Financial Aid
-1.51
2.77
10
-0.55
0.60
Bill1720
2.71
1.76
10
1.54
0.15
Confidence
3.21
1.78
10
1.80
0.10
Value
0.46
1.05
10
0.44
0.67
Cost
1.20
1.60
10
0.75
0.47
Tx by Female
-4.68
2.54
156
-1.84
0.07
Note: -2 Log Pseudo-Likelihood: 796.11, Between-Classroom Variance = 0.63 (SE =
.71).
IMPROVING ALGEBRA SUCCESS 29
Figure 1. Multi-level logistic regression analyses demonstrated a marginally significant
interaction effect between experimental group and gender. The effect was present with and
without covariates. Raw pass rates by condition and gender demonstrate that the primary
difference existed between the control and utility condition for men.
IMPROVING ALGEBRA SUCCESS 30
Appendix
Expectancy-Value-Cost, Interest, and Utility Value Items
Item
E1
How confident are you that you can learn the material in this class?
E2
How confident are you that you can be successful in this class?
E3
How well do you expect to do in this class?
V1
How relevant is the course material to your future career plans?
V2
How important is the course material to your future?
V3
How useful is the course material to your everyday life?
C1
How stressed out are you from taking this class?
C2
How often does this class require too much of your time or effort?
C3
How often do obstacles (class-related or other) limit the effort you can put into this
class?
C4
How often do you sacrifice too many things in order to do well in this class?
UV1
How relevant is the course material to your future career plans?
UV2
How important is the course material to your future?
I1
How interested are you in taking more math classes?
12
How interested are you in learning more about math?
13
How interested are you in learning about careers involving math?
Note. E = Expectancy, V = Value, C = Cost, UV = Utility Value, I = Interest. All items were
accompanied by a 5-point scale with labels for each anchor, for example: 1 - Not at all
Confident, 2 – Slightly Confident, 3 – Somewhat Confident, 4 - Very Confident, 5 -
Extremely Confident. Each response scale was adapted for the particular construct (e.g.,
slightly confident vs slightly useful).
