The Math You Need, When You Need It (TMYN): Leveling the Playing Field

Abstract

The Math You Need, When You Need It (TMYN) is a set of online tutorials designed to help students develop and review mathematical skills that are applied in undergraduate geoscience courses. We present results of a three-year study of more than 4000 students in 106 geoscience courses at a variety of post-secondary schools who were assigned TMYN tutorials as supplemental mathematics instruction. Changes in student scores from pre- to post-test suggest that the support provided by programs such as TMYN can begin to reduce the gap between mathematically well-prepared and underprepared students; in essence, TMYN levels the quantitative playing field for all geoscience students. On average, both high- and low-performing students who fully participated in the use of TMYN as a part of their course showed learning gains, although gains were larger for students who performed poorly on the pre-test. Our findings emphasize the conclusion that students who interact with context-specific quantitative problems can potentially improve their mathematical skills, regardless of initial level of mathematical preparation. We suggest that this type of support could generalize to other science courses.

Full text

Numeracy
Advancing Education in Quantitative Literacy
)96?70 E==?0 <>4.60

$e Math You Need, When You Need It
(TMYN): Leveling the Playing Field
Jennifer M. Wenner
University of Wisconsin OshkoshA0880<?A9=30/?
Eric M. D. Baer
Highline College0-,0<342364800/?
9669A>34=,8/,//4>498,6A9<5=,> 3H:=.396,<.97798=?=10/?8?70<,.C
#,<>91>30 ,<>3&.408.0=97798= ,>307,>4.=97798=,8/>30 &.408.0,8/ ,>307,>4.=
/?.,>49897798=
?>39<=<0>,48.9:C<423>91>304<7,>0<4,6?8/0<, <0,>4@097798=!989770<.4,6H<4-?>4984.08=0
%0.97708/0/4>,>498
*0880<088410< ,8/,0<<4. F0 ,>3+9?!00/*308+9?!00/>' +!0@06482>30#6,C482406/
Numeracy)96==<>4.60
"3H:/B/949<2
@,46,-60,> 3H:=.396,<.97798=?=10/?8?70<,.C@964==,<>

$e Math You Need, When You Need It (TMYN): Leveling the Playing
Field
Abstract
F0 ,>3+9?!00/*308+9?!00/>' +!4=,=0>91986480>?>9<4,6=/0=4280/>9306:=>?/08>=/0@069:
,8/<0@40A7,>307,>4.,6=5466=>3,>,<0,::640/48?8/0<2<,/?,>0209=.408.0.9?<=0=*0:<0=08><0=?6>=91,
>3<00C0,<=>?/C9179<0>3,8=>?/08>=48209=.408.0.9?<=0=,>,@,<40>C91:9=>=0.98/,<C=.3996=
A39A0<0,==4280/' +!>?>9<4,6=,==?::60708>,67,>307,>4.=48=><?.>4983,820=48=>?/08>=.9<0=1<97
:<0>9:9=>>0=>=?220=>>3,>>30=?::9<>:<9@4/0/-C:<92<,7==?.3,=' +!.,8-0248>9<0/?.0>302,:
-0>A0087,>307,>4.,66CA066:<0:,<0/,8/?8/0<:<0:,<0/=>?/08>=480==08.0' +!60@06=>30;?,8>4>,>4@0
:6,C482G06/19<,66209=.408.0=>?/08>="8,@0<,20-9>33423,8/69A:0<19<7482=>?/08>=A391?66C
:,<>4.4:,>0/48>30?=091' +!,=,:,<>91>304<.9?<=0=39A0/60,<84822,48=,6>39?232,48=A0<06,<20<19<
=>?/08>=A39:0<19<70/:99<6C98>30:<0>0=>"?<G8/482=07:3,=4D0>30.98.6?=498>3,>=>?/08>=A39
48>0<,.>A4>3.98>0B>=:0.4G.;?,8>4>,>4@0:<9-607=.,8:9>08>4,66C47:<9@0>304<7,>307,>4.,6=5466=
<02,</60==91484>4,660@06917,>307,>4.,6:<0:,<,>498*0=?220=>>3,>>34=>C:091=?::9<>.9?6/2080<,64D0>9
9>30<=.408.0.9?<=0=
Keywords
$?,8>4>,>4@0=5466=09=.408.0"864807,>307,>4.=>?>9<4,6=
Creative Commons License
F4=A9<54=64.08=0/?8/0<, <0,>4@097798=H<4-?>498!98.9770<.4,64.08=0
Cover Page Footnote
088410< *0880<4=,#<910==9<9109692C,>>30(84@0<=4>C91*4=.98=48"=359=3,8/3,=-008,.>4@048
/0=428482.<0,>482,8/0/4>482 e Math You Need =48.04>=48.0:>498480<<0=0,<.3.?<<08>6C19.?=0=
98>0,.3482;?,8>4>,>4@0=5466=48>30209=.408.0=,8/>30209.3074=><C91:<474>4@0-,=,6>=48>30=9?>30<8
,=.,/0=
<<4.,0<4=,88=><?.>9<48>3009692C#<92<,7,>4236480966020 e Math You Need 2<0A9?>91,
:<92<,730=>,<>0/,>4236480,8/304=98091>30.9.<0,>9<=,8/0/4>9<=91>30A0-=4>00.?<<08>6C
<0=0,<.30=;?,8>4>,>4@064>0<,.C48>30209=.408.0=,8/4=,707-0<91>3060,/0<=34:>0,719<>30&+
:<92<,7
F4=,<>4.604=,@,46,-6048!?70<,.C 3H:=.396,<.97798=?=10/?8?70<,.C@964==,<>

Introduction
Geoscience courses on post-secondary campuses have long been referred to as
“Rocks for Jocks” (e.g., DeLaughter et al. 1998; Gilbert et al. 2012), implying a
paucity of “real” science and the quantitative skills that go along with the pursuit
of scientific endeavors. Yet, the geosciences encompass the practical application
of biology, chemistry, mathematics, and physics (sciences often perceived to be
more quantitatively rigorous than geosciences) to the study of the Earth. For more
than two decades, the geoscience education community has been pushing for
more realistic representations of the quantitative nature of the geosciences in
college courses (e.g., Shea 1990; Vacher 1998; Bailey 2000; Lutz and Srogi 2000;
Macdonald et al. 2000; Baer et al. 2002; Manduca et al. 2008; Wenner et al.
2009). However, in order to increase the quantitative content of geoscience
classes, faculty must be able to address the wide range of students’ mathematical
preparation, support students as they apply or transfer mathematics concepts in
unfamiliar contexts (Bransford et al. 1999; Fike and Fike 2008; Planty et al. 2008)
and find multiple ways to expose students to the power of mathematics as a tool
to solve problems in STEM disciplines (Manduca et al. 2008).
The challenge that faces faculty who teach quantitative geoscience courses is
to offer adequate opportunities for students to explore the power of mathematics
applied to scientific problems and still retain sufficient class time to cover
appropriate content. Striking a balance between stimulating mathematically
prepared students while assisting those who are underprepared requires creative
solutions that encourage students to succeed at the application of mathematics in
geoscience contexts (Wenner et al. 2009). The use of web-based resources can
afford opportunities for “just-in-time” instruction (Kaseberg 1999; Mueller and
Brent 2004) and provide students with context-rich mathematical problems solved
at the student’s pace with immediate application in the subsequent class meeting.
Providing occasions to apply mathematics to well-conceived contextual examples
throughout science courses can also increase students’ motivation and self-
efficacy (Perin 2011; Wenner et al. 2011). When students are motivated,
supported and effective at addressing quantitative problems, the inequalities in
student skills can be reduced so that students find themselves on a more level
playing field, able to address the quantitative problems necessary for deep
learning in the geosciences.
This paper presents results of multiple successful interventions that employed
web-based, asynchronous tutorials to help students review and apply basic
quantitative concepts to geological problems in geoscience courses. We present a
study of 106 geoscience courses at 37 two- and four-year higher education
institutions between Fall 2010 and Fall 2013. The implementations in this study
demonstrate effective application of geoscience-based mathematics tutorials in
1
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

The Math You Need, When You Need It [TMYN], which provides support to
students enrolled in associated courses. The results suggest that effective
quantitative support should focus on application of mathematics to relevant
STEM disciplinary topics. This disciplinary focus facilitates transfer of pre-
existing and learned mathematics to a wide variety of scientific problems and,
therefore, increases students’ proficiency and success with quantitative science.
The Math You Need, When You Need It
The Math You Need, When You Need It is an online resource1 that provides
quantitative instruction to students enrolled in geoscience courses. Since 2010,
TMYN modules have been implemented in a wide variety classes across the
geoscience curriculum. Through TMYN, students with disparate mathematical
skills learn, review and gain skills, applying basic mathematics to solve
quantitative, contextualized geologic problems. Because the geosciences provide
a breadth of scientific topics, mathematical skills addressed by TMYN (Table 1)
are applied in multiple contexts and can be adapted to a range of geoscience
courses. The range of geoscience topics and the modular nature of TMYN allow
instructors to choose appropriate quantitative modules (Table 1), to assign them in
any order, and to build quantitative activities and assessments around geoscience
topics already in a course syllabus.
Table 1
Quantitative modules available through TMYN
Module Name
Web address
Calculating Density
http://serc.carleton.edu/mathyouneed/density/
Graphing (three sub-modules)
http://serc.carleton.edu/mathyouneed/graphing/
• Plotting Points
http://serc.carleton.edu/mathyouneed/graphing/plotting.html
• Constructing a Best Fit Line
http://serc.carleton.edu/mathyouneed/graphing/bestfit.html
• Reading a Point from a Curve
http://serc.carleton.edu/mathyouneed/graphing/interpret.html
Hypsometric Curve
serc.carleton.edu/mathyouneed/hypsometric/
Rates
http://serc.carleton.edu/mathyouneed/rates/
Rearranging equations
http://serc.carleton.edu/mathyouneed/equations/
Slope and topographic maps (two
sub-modules)
http://serc.carleton.edu/mathyouneed/slope/
• Calculating Slope from a
Topographic Map
http://serc.carleton.edu/mathyouneed/slope/slopes.html
• Constructing a Topographic
Profile
http://serc.carleton.edu/mathyouneed/slope/topoprofile.html
Trigonometry
http://serc.carleton.edu/mathyouneed/trigonometry/
Unit Conversions
http://serc.carleton.edu/mathyouneed/units/
1http://serc.carleton.edu/mathyouneed/index.html (last accessed June 1, 2015)
2
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5

Student-Oriented Modules
TMYN modules are open-access, student-centered, web-based tutorials designed
to support self-paced, “just-in-time” student learning (e.g., Kaseberg 1999;
Mueller and Brent 2004). Each module includes three student-oriented pages: (1)
an introduction to the quantitative concept, (2) worked practice problems, and (3)
a culminating post-module quiz.
Pages devoted to introducing the quantitative concept are steeped in
online/multimedia theory (e.g., Mayer 2001) and mathematical pedagogies (e.g.,
Harel 1998; Kaseberg 1999; Mueller and Brent 2004). Each page addresses
motivation for learning the concept, introduces a problem-solving algorithm for
approaching geoscience problems with similar mathematical underpinnings
(including steps that require assessment and evaluation), and walks students
through using the “rules” to solve a preliminary contextual problem (Fig. 1). This
initial exposure to a specific mathematical concept and related geoscience topics
is the first opportunity for students to discover, relearn, or review the quantitative
skills needed in their geoscience courses. Providing and supporting the use of an
algorithm promotes struggling students’ success, which can increase confidence
and self-efficacy when solving quantitative problems (Wigfield and Eccles 2000;
Wenner et al. 2011). In addition to the mechanics behind the mathematics,
introductory pages are also designed to support long-term learning by connecting
contextual geoscience knowledge with conceptual mathematics.
Contextual geoscience application and transfer of mathematical concepts are
underscored on the second student-oriented page – the practice problems page.
Here, students engage with a page of contextualized practice problems, solved
using the provided algorithm (Fig. 1). Drawing on many sub-disciplines within
the geosciences, each practice-problems page offers at least three distinct
contextual examples, promoting application of math concepts among applications.
The practice problems provide students with the opportunity to immediately apply
the mathematical concept they learned on the introductory page to a new context;
the design of these problems draws on the successful mathematical just-in-time
approach to problem solving (Kaseberg 1999; Mueller and Brent 2004) and the
necessity principle—that students are better poised for learning when there is an
immediate application (Harel 1998, 2000). The geosciences are rich in examples
of basic quantitative skills and provide a breadth of scientific contexts; instructors
often revisit mathematical concepts multiple times in the same course and TMYN
facilitates connections among topics. Repeated exposure to mathematical
concepts in multiple contexts has been shown to increase long-term retention
(Kenyon 2000; Stevens 2000; Steen 2004), boost student motivation (Wigfield
and Eccles 2000; Barkley 2010) and improve transfer of learning (e.g., Salomon
and Perkins 1989; Bransford et al. 1999).
3
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

Figure 1. REARRANGING EQUATIONS: an example module
Equations are widely used in geoscience (as well as other STEM disciplines); from
calculating rates of plate motion or groundwater flow to complex calculations of isostasy.
Students often struggle with this relatively basic mathematical concept because it
involves algebra and unfamiliar variables important in the Earth.
Introduction Page (http://serc.carleton.edu/mathyouneed/equations/index.html)
The Rearranging Equations module begins by addressing students’ fear of equations and
explaining that equations are important tools for understanding the natural world.
The module emphasizes the importance of manipulating equations before inserting
numbers; thereby creating a "new" equation that can be used in a variety of applications.
This page also includes a review of rules for algebraic manipulation with some
conceptual explanation of why it works. A step-by-step procedure is subsequently
embedded in a worked practice problem (with hidden answers).
4
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5

Practice Problems Page
(http://serc.carleton.edu/mathyouneed/equations/ManEqSp.html)
When students have completed the introductory page, they are directed to the practice
problem page where they find several other geoscience-related practice problems. A
series of practice problems using the procedure in at least three distinct contexts is linked
from the Introduction Page and are included in a linked PDF file that includes steps for
solving them. The Practice Problem page for Rearranging Equations includes worked
problems involving simple rates, density, and a more complicated isostasy equation. Each
example problem is worked through using the steps outlined on the Introduction Page
(see above) with the answers “hidden” until the student clicks on a “show” button to see
how to do each step. When they have completed the problems, students can follow links
to other examples at similar websites, or to the assessment.
Assessment (post-module quizzes)
(https://www.wamap.org/ - Requires account, contact authors for information)
Instructors create their own, course-specific assessments. Assessment questions that have
been tested and written by users of TMYN can be found in the WAMAP library. They
include questions in which students choose the rearranged equation they will use and then
apply it to solving the given problem, promoting use of the algorithm (Table 2).
5
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

Table 2
Examples of Problems Used in The Math You Need Assessments
Unit Conversions
Sample problem
You have noticed that, on a map, a group of islands makes up a small chain that decreases in age toward the southeast. When you
measure the distance and differences in age, you determine that the tectonic plate over the hot spot has moved 30 kilometers in a million
years.
1) Choose one or more conversion factors that can best be used to convert 30 km/Myr (kilometers per million years)
to cm/yr (centimeters per year).
Notes
This multi-step quiz question is
one of the most commonly used in
the TMYN question library. Note
that the 30 km/Myr in the
question text is a randomly
selected value between 20 and
200 km/Myr
□1min/60sec
□100cm/1m
□60min/1hr
□1000m/1km
□1hr/60min
□1km/1000m
□1Myr/1000000yr
2) Now convert 30 km/Myr to cm/yr : ______cm/yr
Estimating a Best- Fit line
Sample problem
The graph shows the location and age of a volcanic hotspot relative to the location of the present
day volcano. The x-axis (horizontal) has distance (in 100's of km) from present day volcano
plotted and the y-axis (vertical) has age in millions of years. The trend of the data is generally
linear. Estimate the location of the linear trend (plot a best fit line).
Notes
Using the software, the student
plots a best-fit line. WAMAP
evaluates the “fit” of the line
based on parameters set by the
instructor. If a student "connects
the dots" (a common
misconception) no credit is given.
Rearranging Equations
Sample problem
Notes
The equation for isostasy (the height and thickness of crust based on density) is:


=

1


where Htotal is the thickness of the crust, ρcrust is the density of the crust, ρmantle is the density of the mantle, and Habove is the height above
the mantle equilibrium level. 1) Rearrange this equation to solve for Htotal. Choose the correct solution:
Another example of a multi-step
quiz question. Crust and mantle
densities are randomly generated
within a narrow range of
reasonable values.
□
 = 
 

□
 = 
(
 )
□
 = 
 

□
 = 
 

2) Using this equation, calculate the thickness of the crust under Mt. Everest (elevation 8.85 km), if H
above
is 16 km (remember, sea level
is NOT the mantle equilibrium level), ρcrust is 2.67 g/cm3, and mantle is 3.3 g/cm3.
6
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5

Each TMYN module culminates in a graded online post-module quiz with
questions that include intermediate steps in the algorithm as well as contextual
problems. Quiz questions associated with TMYN are framed in the context of
geoscience application. There are no questions that simply ask the student to
complete mathematical operations (e.g., “solve for x”); instead, quiz (and pre- and
post-test) questions take the form of “word problems” that require assessment,
decision-making, and evaluation of one’s answer (Table 2). Instructors administer
quizzes using the free, open-source Washington Mathematics Assessment
Program, WAMAP.2 The majority of students in this study were allowed to take
the post-module quizzes multiple times, promoting mastery, success and self-
efficacy at solving scientifically relevant problems.
Study Design
Our study assesses whether the use of TMYN effectively (1) improves geoscience
students’ basic mathematical skills, (2) helps students apply mathematical skills to
contextual problems, and (3) provides appropriate support and skill development
(i.e., levels the playing field) for geoscience students with diverse incoming skills.
To test the effectiveness of TMYN, we used changes from pre- to post-test at the
student level as a measure of learning gains and thus the effectiveness TMYN.
Participants/Breadth of Sample
This study was conducted from Fall 2010 through Fall 2013 and included 106
courses offered at 37 institutions (Table 3). The institutions are diverse, ranging
from highly selective to open-door; approximately 60% are bachelor’s degree-
granting institutions and offered 56 of the 106 courses in this study (Table 3).
TMYN was used in multiple semesters/quarters across the range of institutions,
illustrating faculty perception that TMYN is a valuable and effective resource for
most students.
Faculty from institutions included in this study attended one or more
workshops focused on incorporating TMYN into a pre-existing course (Wenner et
al. 2011). During the workshop, faculty applied lessons learned from pilot studies
and prior implementations of TMYN, designed an implementation by modifying
their syllabus, and adapted course materials to include appropriate TMYN
modules.3 Workshop participants also designed a protocol for the administration
of pre- and post-tests as well as the implementation of modules and their
associated quizzes. Workshop facilitators guided faculty in effective practices
2 https://www.wamap.org/ (last accessed June 1, 2015)
3 http://serc.carleton.edu/mathyouneed/about/implementations.html (last accessed June 1, 2015)
7
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

such as integration of TMYN with the course (Wenner et al. 2011) and revisiting
mathematical concepts multiple times in multiple contexts (Manduca et al. 2008;
Wenner et al. 2009).
Table 3:
Institutions and Number of Classes (n) Implementing TMYN
College or University % admitted total
completers* classes
4-year College or University
Trinity College, CT
34%
12
1
SUNY Geneseo
36%
306
3
Lafayette College, PA
40%
42
2
Baylor University, TX
40%
392
3
SUNY Oneonta
43%
17
1
California University of Pennsylvania
45%
91
3
West Chester University, PA
47%
87
3
Boston University, MA
51%
37
5
Hofstra University, NY
54%
19
2
California State University-East Bay
61%
6
1
Eastern Kentucky University
66%
21
3
Fitchburg State University, MA
69%
83
5
Ursinus College, PA
70%
17
1
Fort Lewis College, CO
72%
184
10
Central Michigan University
73%
157
2
University of Washington - Tacoma
78%
29
1
University of Wisconsin - La Crosse
78%
68
1
University of Wisconsin Oshkosh†
79%
220
2
University of Maine at Farmington
82%
35
2
Morehead State University, KY
89%
42
2
University of Texas at El Paso
99%
173
3
2-year College
Hillsborough Community College, FL
100%
81
5
Harold Washington College, IL
100%
2
1
McHenry County College, IL
100%
144
8
Community College of Baltimore County, MD
100%
23
2
North Hennepin Community College, MN
100%
9
1
Rochester Community and Technical College, MN
100%
5
1
Wake Technical Community College, NC
100%
272
7
Bergen Community College, NJ
100%
58
4
Ulster County Community College, NY
100%
8
1
Linn Benton Community College, OR†
100%
111
5
University of South Carolina Lancaster
89%
41
2
Lone Star College, TX
100%
78
3
Austin Community College, TX
100%
50
4
Patrick Henry Community College, VA
100%
25
2
Highline Community College, WA
100%
32
2
Central Wyoming College, WY
100%
4
2
† more than one instructor
* students who completed all or all but one of the assigned modules in their course
8
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5

Courses included in this study covered a range of geoscience topics; common
first-year introductory courses (Physical Geology, Environmental Geology and
Earth System Science) made up approximately 63% (n=67) of all
implementations. Other introductory courses (n=28) were more varied in topic,
with titles such as Meteorology, Natural Hazards, Physical Geography and
Oceanography. The remaining 11 courses were upper-level courses and included
Hydrology (n=3), Geomorphology (n=1), Structural Geology (n=4) and
Geological Methods (n=3). In total, 4486 students were enrolled in the 106
courses involved in this study; individual course enrollments ranged from 5 to 202
students. Although courses were offered at a variety of levels, we examined all
uses of TMYN in this study because we observed no significant difference
between pre-test scores and subsequent achievement for introductory and upper-
level students (Wenner et al. 2012). The variety of course topics for which faculty
desired additional mathematical support emphasizes the need for resources such
as TMYN in the geosciences at all levels and underscores the diversity of students
who can benefit from the implementation of TMYN.
Use of TMYN
In the courses included in this study, students completed a pre-test prior to
engaging with TMYN; engaged with each module relevant to their course and
took the associated post-module quiz; and then took a post-test when all the
intervention and associated course content were completed. Equivalent pre- and
post-tests consisted of 10-25 questions relating to the assigned modules and
assessed quantitative knowledge and the ability to transfer that knowledge to
geoscience content. Equivalency in pre- and post-test questions was determined
by the format and quantitative skills of the questions; in some cases questions
were identical, but in most cases varied slightly by randomizing specific
numerical values used in questions (Table 2). In all but five cases, the pre- and
post-tests were administered online, sometimes in a computer-equipped
classroom; all post-module quizzes were administered using WAMAP.
Because we wished to provide flexibility for instructors to incorporate
mathematical skills that span the breadth of topics covered in geoscience courses,
instructors designed their own pre- (and equivalent post-) tests, as well as post-
module quizzes so that they included only topics that were covered in the
associated course. Each course varied in assigned modules, pre- and post-test
questions, and course subjects; thus, the student sample was normalized using
standard Z scores (Abdi 2007) so that we could compare results across
implementations. Three to eight modules were used in each implementation. Pre-
test, post-test and post-module quiz scores were converted to percent correct for
analysis.
9
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

Measuring engagement and effectiveness
If geoscience students are to gain skills from TMYN, they must engage with and
complete the material contained in the modules. Because we wished to assess the
effectiveness of TMYN modules at improving skills, we developed a measure of
student engagement with the material. Only students who took post-module
quizzes for all or all but one of the assigned modules were considered to have
completed the intervention (we call them “completers”) and were included in our
analysis of learning gains due to the application of TMYN in conjunction with a
course. Based on this criterion, 3408 students completed the intervention and
2979 took both the pre and post-test (Table 4A).
In our desire to ensure that all students had maximum access to tools that
could help them to succeed, we chose to administer the intervention to all
students, precluding a formal randomized control sample. However, 311 students
completed both a pre- and post-test but did not engage with a minimum number of
TMYN modules, providing a quasi-control group. Although this group is not a
randomized control, this group of non-completers (1) received the same
information outside of the module as those students who engaged with the
modules; and (2) appears representative of the students in the study because
average pre-test scores for both groups are similar (Table 4). Thus, we use these
non-completers to assess the effectiveness of the intervention at improving
learning gains.
Normalizing pre -test data (Zpre)
Because we wanted to maximize use and allow for flexibility in both topic and
coverage, instructors tailored the pre-test to their implementation. To be able to
compare diverse interventions, we computed a standard Z score (Abdi 2007) using
the individual course pre-test mean and standard deviation to calculate students’
normalized pre-test scores (Zpre):
 =
  (1).
where scorepre is the individual student pre-test score, meancourse is the course pre-
test mean, and SDcourse is the standard deviation of pre-test scores for the
individual course. For the purposes of this study, we subdivided students in the
study into four groups based on their Zpre scores (Table 4): Group 1 with Zpre less
than 1.0 (much below the class mean); Group 2 with Zpre between 1.0 and 0.0
(below the class mean); Group 3 with Zpre between 0.0 and 1.0 (above the class
mean), and Group 4 with Zpre greater than 1.0 (much above the class mean; Table
4). These designations will be used throughout the paper.
10
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5

Table 4:
A: Pre- to Post-test Gains and Mean Normalized Gain* (Hake, 1998) for Completers Disaggregated by Zpre**
Group ( range of
Zpre**) pre-test
(n)
pre- &
post-test
(n)
mean
NG***
(%)
NG
negative
(n)
NG
positive
(n)
NG = 0
(n)
pre-test
=100%
(n)
n
excluded***
(NG)
Average pre-
test score Average post-
test score
Group 1 (<

1) 506 430 44% 32 394 4 36% 64%
Group 2 (

1 to 0) 1068 915 38% 119 782 12 55% 72%
Group 3 (0 to 1) 1325 1173 25% 311 820 33 11 12
(

468%) 73% 80%
Group 4 (>1) 509 461 11% 144 237 27 53 20
(

672%) 85% 88%
Totals (n) 3408 2979 2883 606 2233 76 64 32
B: Pre- to Post-Test Gains and Mean Normalized Gain* (Hake, 1998) for Non-Completers Disaggregated by Zpre**
Group 1 (<

1) 122 80 24% 9 69 2 31% 48%
Group 2 (

1 to 0) 179 95 28% 18 70 7 52% 65%
Group 3 (0 to 1) 158 108 7% 30 69 9 71% 74%
Group 4 (>1) 48 28 -21% 7 10 2 8 1
(

467%) 83% 80%
Totals (n) 507 311 302 67 215 20 8 1
* Average of normalized gain [eq. (2)] for all students within a given Zpre category (Gery 1972, Hake 1998; Kaiser 1989; Williams and Zimmerman 1996).
**Zpre is the difference between individual pre-test score and course mean divided by standard deviation [eq. (1)]; a standard statistical measure)
*** 32 (~1%) completers and 1 (<<1%) control student with normalized gains below

250% (meaning they lost more than 2.5 times the points between their pre-
test score and 100%) were excluded from these calculations (see text for justification)
11
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

Results
Pre- and post-test scores
Table 4 shows pre- and post-test scores (far right columns) for completers (A) and
non-completers (B) with scores for both. Course means and individual pre-test
scores for 3408 completers and 507 non-completers were used to calculate each
student’s Zpre (Eq. (1); Table 4A). Analysis of student scores based on Zpre group
designations shows the disparity in student abilities at the outset of a geoscience
course. Average completers’ pre-test scores subdivided by Zpre show a nearly 50
percentage point spread. Students in Group 1 (n=506) averaged 36% on the pre-
test whereas students in Group 4 (n=461) scored an average of 85% on the pre-
test (Table 4A). For the non-completers, the spread is only slightly greater (52%)
with Group 1 scoring 31% and Group 4 scoring 83% (Table 4B).
Average post-test scores, which measure student achievement with the use of
TMYN, are also shown in Table 4. Post-test scores for completers range from
64% (Group 1) to 88% (Group 4); each Zpre designation shows improvement from
pre- to post-test. For the non-completers, student post-test scores vary from 48%
to 80%, and Group 4 shows a decline of 3 percentage points from pre- to post-
test. Although pre-test scores are relatively similar between completers and non-
completers in the same group, post-test scores for non-completers are
significantly lower than for their counterparts who completed the intervention.
Normalized gain scores
Normalized gain is a common way to measure learning gains (e.g., Gery 1972;
Hake 1998). It is calculated as a percentage using a student’s increase from pre-
test to post-test divided by the maximum possible gain (Gery 1972, Hake 1998):
 (%) = 
 (2).
where NG = percent normalized gain; scorepost = post-test score (%); and scorepre
= pre-test score (%). Normalized gain scores are used in this analysis rather than
difference scores because the former correlates less well with pre-test scores
(corr. coeff. = 0.221; Fig 2B) than the latter (corr. coeff.= 0.530, Fig. 2A)
(Kaiser 1989). If the difference between the pre- and post-test score were to be
used, initially high-scoring individuals’ changes would be muted (Fig. 2A). An
additional advantage to using normalized gain is that by choosing a change
measurement that is poorly correlated to the pre-test value, the reliability of this
difference is greater (Williams and Zimmerman 1996).
12
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5

Figure 2. Plots showing Zpre (symbol color) and correlation between pretest score and (A) simple
difference (pre-test - post-test with student scores excluded from calculations of average NG
(Table 4) shown in orange (extreme NG) and black (perfect pre-test) diamonds. This plot
illustrates the muting of small but substantial changes in scores for students who score high on the
pre-test. Note the clustering of green and blue points for high pre-test scores (B) pre-test score vs.
normalized gains (NG). White field shows student with positive gains or no change, approximately
78% (n=2309) of students. Gray field shows students with negative gains. Orange diamonds in A
plot off the bottom of the gray field.
13
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

Normalized gain scores for completers and non-completers in this study with
both pre- and post-test scores (n=2979 and n=311, respectively) were calculated
using Equation (2). Table 4 shows scores subdivided into number of students with
positive, negative or zero NG and average NG for each group. Positive NG scores
record student improvement throughout the semester; negative NG scores indicate
that students had a lower score on the post-test than on the pre-test. Mean
normalized gains for each Zpre group are plotted for comparison in Fig. 3. Note
that NG cannot be computed for students who score 100% on the pre-test
(denominator would be zero); therefore, 64 completers who had perfect pre-test
scores (~2%; Fig. 2A black symbols) and 8 non-completers (2.5%) were excluded
from analysis of NG scores (Table 4; Fig. 3).
Figure 3. Normalized gains (subdivided by Zpre) for completers (black) and non-completers (gray)
with both pre- and post-test scores. See Table 4 for n in each group.
Using normalized gain scores can exaggerate small negative changes for
students with high pre-test scores, resulting in outliers with extreme negative NG
scores (e.g., one student in our sample who scored 99% on the pre-test and 78.8%
on the post-test resulting in a NG = 2020%). Large negative normalized gains,
when included in mean calculations, mask the true effectiveness of the
intervention for the majority of high-scoring students. For the purposes of
assessing learning gains, we omit 32 completers (1.1%; orange symbols Fig. 2A)
and 1 non-completer (0.3%) whose normalized gain scores were equal to or less
14
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5

than 250%. All large negative gains are for students who scored above the mean
on the pre-test: 12 in Group 3 and 21 (plus one non-completer) in Group 4.
Exclusion of these outliers changes the average NG from 22% to +11% for
Group 4 and from +21% to +25% for Group 3 completers.
Fig. 3 shows calculated mean normalized gains for completers and non–
completers in each group (Table 4). With the exclusion of the students discussed
above, the sample size for NG drops to 2883 completers and 302 non-completers.
Completers show positive mean NGs for all groups, with those who score below
the mean on the pre-test (Groups 1 and 2) showing greater gains (44% and 38%,
respectively) than their higher-scoring counterparts (Group 3: 25%; Group 4:
11%). Non-completers (n=302) who did not fully engage with the intervention
realized diminished normalized gains relative to their fully engaged counterparts
in this study. Although Groups 1, 2 and 3 show nominal learning gains (24%,
28%, and 7%, respectively), Group 4 – the highest scoring individuals – averaged
negative NG (21%; Fig. 3).
Discussion
TMYN effectively improves mathematical skills
The effectiveness of TMYN at improving students’ skills is illustrated in Fig. 3.
Faculty included in this study used an integrated approach, requiring all students
to complete the intervention regardless of pre-test score, a decision based on prior
research revealing that completion rates dropped when high-performing students
were given the option to forego the modules (Wenner et al. 2011). Completers
(black bars) show improvement (on average) across all skill levels whereas non-
completers (gray bars) realize much lower gains (and, in Group 4, gains are
actually negative). Non-completers’ average scores on the pre-test (Table 4)
illustrate that these students (although a considerably smaller sample) represent a
relatively good cross-section of the students who participated in the study. Non-
completers and completers received identical course information; yet non-
completers did not fully engage with the tutorials. Students who do not engage
with the program consistently score lower on the post-test (Table 4) and realize
lower normalized gains (Fig. 3) than their engaged counterparts. The combination
of positive learning gains plus consistent learning gains for completers, and lower
NG and post-test scores for the quasi-control group support the conclusion that
learning gains are, at least in part, the result of employing TMYN tutorials in
conjunction with a geoscience class. Furthermore, application of the intervention
to all students seems to be justified by the small sample of students with pre- to
post-test changes who did not engage with the modules.
15
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

TMYN helps students apply mathematical skills to
geoscience
Mean normalized gains reveal that TMYN modules can help students at all
incoming levels to successfully apply mathematics to contextualized quantitative
science problems (Fig. 3; Table 4). Because pre- and post-tests included
mathematical problems in the context of geoscience, individual learning gains can
illustrate increased transfer of skills from mathematics to geoscience. Individual
normalized gains vs. pre-test scores for completers are plotted on Fig. 2B; points
that fall in the white box represent positive or no change (80%; n=2309; Table
4A); completers with negative NG plot in the gray portion of the plot (20%;
n=574). Students who initially lack quantitative skills (based on low pre-test
scores; red and yellow symbols; Fig. 2B) show the greatest improvements with
the implementation of TMYN; nearly 89% of low-scoring students (1129 of
1345) showed positive normalized gains and cluster in the white field. Although a
larger proportion of Groups 3 and 4 (green and blue symbols respectively; Fig 2)
have learning gains that reflect lower post-test scores (gray box), a majority of
both groups showed positive gains (Group 3=70%; Group 4=51%; Table 4).
Individual student learning gains on pre- and post-test questions suggest that
TMYN supports the successful transfer of students’ basic mathematical skills to
geoscience topics. Thus, a majority of students who engage with the material, no
matter where their pre-test scores fall in relation to their peers, realize learning
gains.
TMYN levels the playing field
The use of TMYN reduced the nearly 50 percentage point difference in pre-test
scores between the highest and lowest performing group to less than 25
percentage points between the same groups of students on the post-test (Fig. 4).
The average change from pre- to post-test score for 430 students in Group 1 was
from 36 to 64% ‒ a positive change of 28 percentage points ‒ whereas the average
post-test score for the 461 students in Group 4 went from 85 to 88% (Fig. 4; Table
4). The intervening groups showed intermediate change – from 55% to 72% for
Group 2 and 73 to 80% for Group 3 students (Fig. 4). Whereas all groups
averaged some improvement between pre- and post-test, students who scored
lowest on the pre-test showed greater gains than higher-performing students
indicating that the integration of TMYN in a course closes the gap between low-
and high-performing students (Fig. 4). Indeed, the gap between high- and low-
scoring students was reduced by more than 50% – from 49 percentage points on
the pre-test to 24 percentage points on the post-test (Fig. 4) – suggesting that
engagement with TMYN helps to “level the playing field” for students of
differential abilities.
16
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5

Figure 4. Average pre- and post-test scores for completers in 106 courses that used TMYN
(n=2979). Colors represent different groups with warm colors representing low scoring students
and cool colors representing high scoring students: red: Group 1 (more than 1 SD below mean);
yellow: Group 2 (0-1 SD below mean); green: Group 3 (0-1 SD above mean); blue: Group 4 (>1
SD above mean). Note the wide disparity among pre-test scores that is more than halved (from 49-
24 percentage points difference) from pre- to post-test, illustrating the leveling of the playing field
for students using TMYN.
Conclusions
Our findings reinforce the idea that students who interact with context-specific
quantitative problems, such as those embedded in TMYN, realize learning gains
and improve their quantitative skills. Individual learning gains on contextualized
pre- and post-test questions illustrate that, when quantitative problem solving is
integrated into a science course, modules that support quantitative learning can
promote knowledge transfer from mathematical concept to geoscience contexts.
Student improvement from pre- to post-test across all levels of initial quantitative
skills indicate that The Math You Need, When You Need It is effective at
“leveling the playing field,” no matter a student’s prior preparation. Furthermore,
the disparity in post-secondary students’ mathematical preparation is not a
problem specific to the geosciences; many general science courses require basic
quantitative knowledge. Thus, although the current modules are solely in the
context of the geosciences, we contend that the support provided by quantitative
17
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

modules integrated into a course could generalize to a variety of science
disciplines.
Acknowledgments
This work is supported by NSF grants DUE-0633402 and -0920583 to Wenner,
and DUE-0633755, and -0920800 to Baer. Thank you to Helen Burn for advice
about educational protocol and for some of the data analysis. We are grateful to
all of the users of TMYN and acknowledge the support of Cathy Manduca, Sean
Fox, John McDaris and staff of the Science Education Resource Center at
Carleton College. We would also like to thank H. Lehto, S. Schellenberg and an
anonymous reviewer for constructive and thoughtful reviews, which greatly
improved the manuscript.
References Cited
Abdi, Hervé. 2007. Z-scores. In Encylopedia of Measurement and Statistics, edited by
Neil Salkind. Thousand Oaks, CA: Sage.
Baer, Eric M., Emanuela A. Baer, Helen Burn, Lisa Gilbert, and Carla Whittington. 2002.
MathPatch - A "just-in-time" approach to teaching quantitative skills in introductory
courses. Geological Society of America Abstracts with Programs, Cordilleran
Section, 98th Annual Meeting, May 13-15.
Bailey, Christopher M. 2000. A quantitative approach to introductory geology courses.
Journal of Geoscience Education 48: 413-418.
Barkley, Elizabeth F. 2010. Student Engagement Techniques: A Handbook for College
Faculty. San Francisco, CA: Jossey-Bass.
Bransford, John D., Ann L. Brown, and Rodney R. Cocking. 1999. How People Learn:
Brain, Mind, Experience, and School. Washington, DC: National Academies Press.
DeLaughter, John E, Seth Stein, Carol A. Stein, and Kenneth R. Bain. 1998.
Preconceptions abound among students in an introductory earth science course. Eos
Trans. AGU 79 (36):429-432. http://dx.doi.org/10.1029/98EO00325
Fike, David S., and Renea Fike. 2008. Predictors of first-year student retention in the
community college. Community College Review 36 (2): 68-88.
http://dx.doi.org/10.1177/0091552108320222
Gery, F. W. 1972. Does mathematics matter? In Research Papers in Economic
Education, edited by Arthur Welsh, 142–157. New York: Joint Council on
Economic Education.
Gilbert, Lisa A, Jennifer Stempien, David A McConnell, David A Budd, Katrien J van
der Hoeven Kraft, Ann Bykerk-Kauffman, Megan H Jones, Catharine C Knight,
Ronald K Matheney, and Dexter Perkins. 2012. Not just “rocks for jocks”: Who are
introductory geology students and why are they here? Journal of Geoscience
Education 60 (4): 360-371. http://dx.doi.org/10.5408/12-287.1
Hake, Richard R. 1998. Interactive-engagement versus traditional methods: A six-
thousand-student survey of mechanics test data for introductory physics courses.
American Journal of Physics 66: 64-74. http://dx.doi.org/10.1119/1.18809
18
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5

Harel, Guershon. 1998. Two dual assertions: The first on learning and the second on
teaching (or vice versa) The American Mathematical Monthly 105 (6): 497-507.
http://dx.doi.org/10.2307/2589401
———. 2000. Three principles of learning and mathematics: Particular reference to
linear algebra-old and new observations. In On the Teaching of Linear Algebra,
edited by J. Dorier, 177-190. Kluwer Academic Publishers.
Kaiser, L. . 1989. Adjusting for baseline: Change or percentage change? Statistics in
Medicine 8: 1183-1190. http://dx.doi.org/10.1002/sim.4780081002
Kaseberg, Alice M. 1999. Intermediate Algebra: A just-in-time approach. Stamford, CT:
Thomson-Brooks/Cole.
Kenyon, Patricia M. 2000. Building quantitative skills in geoscience courses using
homework assignments of increasing difficulty. Journal of Geoscience Education 48
(4): 418-422.
Lutz, Timothy M., and LeeAnn Srogi. 2000. The role of a shadow course in improving
the mathematics skills of geoscience majors. Journal of Geoscience Education 48
(4): 474-478.
Macdonald, R. Heather, LeeAnn Srogi, and Glen Stracher, eds. 2000. Special Issue:
Building the Quantitative Skills of Students in Geoscience Courses. Vol. 48, Journal
of Geoscience Education: National Association of Geoscience Teachers.
Manduca, Cathryn A., Eric M. Baer, Greg Hancock, R. Heather Macdonald, Sam
Patterson, Mary E. Savina, and Jennifer M. Wenner. 2008. Making undergraduate
geoscience quantitative. EOS 89: 149-150. http://dx.doi.org/10.1029/2008EO160001
Mayer, Richard E. 2001. Multimedia Learning. New York: Cambridge University Press.
http://dx.doi.org/10.1017/CBO9781139164603
Mueller, Guntram, and Ronald I. Brent. 2004. Just-In-Time Algebra and Trigonometry
for Students of Calculus. Third ed: Addison Wesley.
Perin, D. 2011. Facilitating student learning through contextualization: A review of the
evidence. Community College Review 39 (3):268-295.
http://dx.doi.org/10.1177/0091552111416227
Planty, Michael, William Hussar, Thomas Snyder, Stephen Provasnik, Grace Kena,
Rinkes Dinkes, Angelena KewalRamani, and Jana Kemp. 2008. The Condition of
Education 2008. Washington, DC: National Center for Education Statistics, Institute
of Education Sciences, U.S. Department of Education.
Salomon, Gavriel, and David N. Perkins. 1989. Rocky roads to transfer: Rethinking
mechanisms of a neglected problem. Educational Psychologist 24 (2): 113-142.
http://dx.doi.org/10.1207/s15326985ep2402_1
Shea, James H. 1990. Mathematics in physical-geology textbooks. Journal of Geoscience
Education 38: 138-148.
Steen, Lynn Arthur. 2004. Achieving Quantitative Literacy: An Urgent Challenge for
Higher Education. Washington DC: Mathematical Association of America.
Stevens, Reed. 2000. Who counts what as math?: Emergent and assigned mathematics
problems in a project-based classroom. In Multiple Perspectives on Mathematics
Teaching and Learning, edited by Jo Boaler, 105-144. Westport, CT: Ablex.
Vacher, H. L. 1998. Computational Geology, 1-Significant Figures! Journal of
Geoscience Education 46: 292-295.
19
Wenner and Baer: Leveling the Playing Field with TMYN
Published by Scholar Commons, 2015

Wenner, Jennifer M., Eric M. Baer, Cathryn A. Manduca, R. Heather Macdonald, Sam
Patterson, and Mary E. Savina. 2009. The case for infusing quantitative literacy into
introductory geoscience courses. Numeracy 2 (1): Article 4.
http://dx.doi.org/10.5038/1936-4660.2.1.4
Wenner, Jennifer M., H. Burn, and E.M.D. Baer. 2011. The Math You Need, When You
Need It: Online modules that remediate mathematical skills in introductory
geoscience courses. Journal of College Science Teaching 41: 16-24.
———. 2012. Improving geoscience students' quantitative skills with The Math You
Need, When You Need It. In the Trenches 2 (2): 1-5.
Wigfield, A., and J. S. Eccles. 2000. Expectancy - value theory of motivation.
Contemporary Educational Psychology 25: 68-81.
http://dx.doi.org/10.1006/ceps.1999.1015
Williams, R. H., and D. W. Zimmerman. 1996. Are simple gain scores obsolete? Applied
Psychological Measurement 20 (1): 59-69.
http://dx.doi.org/10.1177/014662169602000106
20
Numeracy, Vol. 8 [2015], Iss. 2, Art. 5
http://scholarcommons.usf.edu/numeracy/vol8/iss2/art5
DOI: http://dx.doi.org/10.5038/1936-4660.8.2.5