ChapterPDF Available

The Role of Continuous Assessment and Effective Teacher Response in Engaging all students

Authors:

Abstract and Figures

In this chapter, we report on three practices that, together, reduce marginalization of students from effective learning in mathematics. We have observed that the combined use of continuous formative assessment, responsive teaching, and effective patterns of variation form a powerful way of engaging and sustaining mathematics learning for students who might otherwise be marginalized by being left behind or not being appropriate challenged. Our observational categories draw from work in formative assessment and variation theory, our own approach to mastery learning, and iterations of testing and refinement in light of our own classroom observations. We begin by situating the practical and theoretical perspectives of our work with respect to mastery learning and variation theory, and then further explain the relevance of our work in preventing marginalization of learners. From there, we describe our work in classrooms and its role in the development of an observation protocol based on the categories we describe. Finally, we offer four classroom episodes that highlight the significance of these categories.
Content may be subject to copyright.
R. Hunter et al. (Eds.), Mathematical Discourse that Breaks Barriers and Creates Space
for Marginalized Learners, 101–119.
© 2018 Sense Publishers. All rights reserved.
PAULINO PRECIADO-BABB, MARTINA METZ,
SOROUSH SABBAGHAN AND BRENT DAVIS
6. THE ROLE OF CONTINUOUS ASSESSMENT
AND EFFECTIVE TEACHER RESPONSE IN
ENGAGING ALL STUDENTS
INTRODUCTION
In this chapter, we report on three practices that, together, reduce marginalization of
students from effective learning in mathematics. We have observed that the combined
use of continuous formative assessment, responsive teaching, and effective patterns
of variation form a powerful way of engaging and sustaining mathematics learning
for students who might otherwise be marginalized by being left behind or not being
appropriate challenged. Our observational categories draw from work in formative
assessment and variation theory, our own approach to mastery learning, and iterations
of testing and refinement in light of our own classroom observations. We begin
by situating the practical and theoretical perspectives of our work with respect to
mastery learning and variation theory, and then further explain the relevance of our
work in preventing marginalization of learners. From there, we describe our work in
classrooms and its role in the development of an observation protocol based on the
categories we describe. Finally, we offer four classroom episodes that highlight the
significance of these categories.
Decades of research and practice of mastery learning (Bloom, 1968; Guskey,
2010) have confirmed the importance of assigning work to students only when they
are ready to engage in such work. Identifying the time a student is ready to move
forward is particularly relevant for learners disadvantaged by issues such as ability,
language, culture, and socio-economic status. We believe teaching practices that
neglect mastery learning marginalize students who, for whatever reason, are not
able to engage productively in the classroom discourse. We propose that continuous
formative assessment followed by appropriate teacher responses during instruction
supports all students’ engagement in classroom activities and plays a key role in
developing equitable teaching practices. Although formative assessment strategies
have been extensively suggested by recent literature (e.g., Box, Skoog, & Dabbs,
2015; Chappuis, 2015; Heidi & Cizek, 2010; Wiliam, 2011; William & Leahy,
2015), there is little advice for the teacher on how to respond in the moment to
emergent situations in the classroom. We argue that such responses can be informed
by Marton’s (2015) variation theory of learning, addressing not only those students
This is an open access chapter distributed under the terms of the CC-BY-NC 4.0 License.
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
P. PRECIADO-BABB ET AL.
102
who struggle to understand a particular mathematical instruction, concept, or
procedure, but also, students who quickly master the targeted content or goal of a
lesson.
In this chapter we present some results from research on an intervention aimed
at improving numeracy at the elementary level in Canada. The research is part of
a partnership among a major school district (Calgary Separate School District),
a charity that develops mathematics curricular materials (JUMP Math), and the
Werklund School of Education of the University of Calgary. We have observed,
trough longitudinal data, a steady improvement of performance in mathematics.
However, not all the groups showed similar improvement, and teaching practices
varied form one classroom to another.
We have identified three teaching practices that seem to be key in allowing
students to engage meaningfully in the mathematical discourse in the classroom,
namely: continuous assessment of all students during class, particular forms of
teacher responsiveness, and use of effective variation to draw attention to critical
content features. We have adapted previous work on mastery learning by (a) placing
a stronger emphasis on prevention rather than remediation, (b) stressing the
importance of moment-by-moment assessment to all students, and (c) engaging all
students in the same type of activities instead of streaming students for remediation
or enrichment. These practices are used in this chapter to describe how teachers
can support all students’ learning; we argue that when one or more practice falters,
students are marginalized through lack of opportunity to engage meaningfully in
mathematics learning activities in class.
MARGINALIZATION AND MATHEMATICS EDUCATION
Mathematics ability, or lack thereof, has been identified as a marginalizing
factor impacting people’s lives. A recent report from the Organization for Economic
Co-operation and Development (OECD, 2016) acknowledged that “Numeracy
skills are used daily in many jobs and are important for a wide range of outcomes
in adult life, from successful employment to good health and civic participation”
(p. 36). In many cases, mathematics is a filter for further education; students need to
complete high school programs and to pass admission tests that include mathematics
to be accepted in postsecondary programs. The OECD report also concluded:
³>0@DWKHPDWLFV HGXFDWLRQ RIWHQ UHLQIRUFHV UDWKHU WKDQ PRGHUDWHV LQHTXDOLWLHV LQ
education” (p. 3). In this sense, poor quality instruction in mathematics from the
early years perpetuates and may even foster social inequity.
The OECD (2016) report has identified persistent problems in mathematics
education; however, the proposed solutions seem very general with no attention to
particular mathematical content or to particular mathematical knowledge required
for teaching. One of the problems is that K-12 mathematics instruction seems to
be differentiated with regard to socio-economical status and initial mathematical
ability, as reflected in identified differences across schools:
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
THE ROLE OF CONTINUOUS ASSESSMENT AND EFFECTIVE TEACHER RESPONSE
103
>7@HDFKHUVXVHFRJQLWLYHDFWLYDWLRQVWUDWHJLHVWRGHHSHQWKHFXUULFXOXPFRQWHQW
and support the development of problem-solving abilities among students in
advantaged schools. By contrast, in disadvantaged schools, it appears that
there might be a price to pay for using strategies that emphasize thinking and
reasoning for an extended time: less material is covered. (p. 113)
This statement, however, seems to place problem solving and content coverage at
odds with one another for those disadvantaged schools. The proposed solutions are
based on findings from the Programme for International Student Assessment (PISA),
which advises teachers to: encourage students to work in small groups; provide
extra help to students who need it; reduce the mismatch between taught content
and assessment; and promote the use computers in the mathematics classroom.
These suggestions do not consider a more nuanced mathematical knowledge for
teaching, which has been studied for decades by the mathematics education research
community (see Da Ponte & Chapman, 2016).
In this chapter, we not only consider general mathematics teaching practices,
but also their relation to a nuanced knowledge of critical features for learning
particular mathematical ideas (Marton, 2015). These features inform both
assessment and instruction. In our experience, learning gaps often get interpreted
as learning disabilities, and boredom may be interpreted as lack of motivation. Both
interpretations may contribute to marginalizing students from learning mathematics
and, therefore, from life opportunities. The teaching practices we focus on are
intended to address both learning gaps and lack of motivation.
Many issues of inequity and social justice regarding marginalized populations
go well beyond the scope of this chapter. However, regardless of the factors
impacting equity or of the approach for breaking barriers impacting marginalized
students, we stress that teachers and schools must consider teaching practices and
teachers’ knowledge that support the learning of all students in the classroom.
While there is an obvious emphasis on disadvantaged students in terms of factors
such as socioeconomic status and language, we also consider practices that
marginalize students who meet basic expectations but, with better instruction,
could develop deeper mathematical understanding. Students who have mastered
expected learning outcomes may be marginalized from further learning if they
are not provided with activities or tasks that extend their mathematical ability or
knowledge.
THEORETICAL BACKGROUNDS
The teaching practices that we attended to in the intervention were informed by the
extensive research on mastery learning and by the variation theory of learning: While
the former is an instructional approach, the later is a theory highlighting necessary
conditions for learning that can inform teaching.
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
P. PRECIADO-BABB ET AL.
104
The variation theory of learning prompts attention to key features necessary
for understanding particular ideas (in our case, mathematical ideas), and to
developing patterns of variation that draw attention to these features (Marton,
2015). According to this theory, we learn through experiencing patterns of variance
against a background of relative invariance. For instance, a student can learn what
a square is by contrasting squares with other shapes, each of which differ from
a square in a particular way (e.g., more than 4 sides, unequal sides, angles that
are not right). Such contrast would prompt attention to the critical features to
be discerned in order to identify a square, which is what Marton described as
the intended object of learning. This student may generalize her understanding
of a square by considering squares in different positions (shape is invariant and
position varies) or with different patterns. In other words, the learner first discerns
the critical features of a square, and then discerns non-essential features. In both
cases, certain patterns of variation against a background of invariance allow the
learner to focus attention on critical features of the intended object of learning.
While this theory has similarities to the development of concepts described by
Bruner (1960), it is not inherently constructivist. Nonetheless, Runesson (2005)
showed how using variation theory as an analytical framework may complement
constructivist or social constructivist analyses, as it draws attention to features not
considered by either.
Mastery learning (Bloom, 1968; Bloom, Hastings, & Madaus, 1971; Guskey,
2007, 2010) emphasizes formative assessment, remediation, and enrichment. In the
intervention described in this chapter, we combined these ideas with instructional
practices that emphasize the use of effective patterns of variation to draw attention
to the critical features of clearly-defined objects of learning. We will now elaborate
the way in which we bring these ideas together.
Guskey (2010) suggested that formative assessments should vary in frequency
from one to several weeks. In his view, a key purpose of these assessments is
to identify students who have already mastered the learning goals and those
who need remediation. Teachers may then prepare “enrichment activities that
provide valuable, challenging, and rewarding learning experiences for learners
who have mastered the material and do not need corrective instruction” (p. 3).
These activities are often selected by the students and might involve projects or
reports, games, or complex problem-solving tasks (Guskey, 2007). We concur
with Guskey (2010) and Bloom (1968) that enrichment activities should be more
than a way to keep students busy and thereby allow teachers time to support
other students and that they should also provide opportunities for deepening
students’ learning. However, we further emphasize the role of using of effective
patterns of variation and continuous assessment in preventing the need for
excessive remediation, and we stress the importance of continuous extension for
all students.
The focus on remediation initiated by mastery learning is evident in the
extensive literature on formative assessment (e.g., Box, Skoog, & Dabbs, 2015;
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
THE ROLE OF CONTINUOUS ASSESSMENT AND EFFECTIVE TEACHER RESPONSE
105
Chappuis, 2015; Heidi & Cizek, 2010; Stiggins, Arter, Chappuis, & Chappuis,
2004; Wiliam, 2011; William & Leahy, 2015), which prompt to remediation.
Further, while many scholars have agreed that formative assessment should happen
moment by moment, the advice for teachers is often expressed in terms of general
strategies, such as re-explaining or re-teaching the lesson. Little is said about how
the explanation or lesson could be adjusted to better meet students’ needs.
The approach to formative assessment we describe here differs from mastery
learning as it has been commonly reported in the literature: We have stressed the
need to assess not only regularly at the end or in the middle of a unit of content,
as suggested in mastery learning, but also continuously during class time with a
focus on prevention rather than remediation. The idea is that teachers fine-tune
instruction moment by moment, including posing extra activities for some students
and addressing students who require support. Similar to the enrichment activities in
mastery learning, the curricular material used by teachers in the intervention advises
teachers to “be ready to write bonus questions on the board from time to time during
the lesson for students who finish their quizzes or tasks earlier” (Mighton, Sabourin,
& Klebanov, 2010, p. A-8). More specifically, it suggests that teachers may: use
larger numbers; introduce new terms or elements; ask students to correct mistakes;
ask students to complete missing terms in a sequence; vary the task or the problem
slightly; look for applications of the concept; and ask students to find and describe
patterns. Consistent with Mighton (2007), teachers in the intervention were also
encouraged to prepare extra material for all students, including those who initially
performed lower in class. These extra activities should be tied to the common task
that everyone is doing in the lesson. Further, proposed activities are often small
variations of the tasks in which all students engage in class, but with the potential
to prompt emergent insights regarding the broader mathematical ideas (cf. Aljarrah,
Preciado Babb, Metz, Sabbaghan, Pinchbeck, & Davis, 2016). This offers another
contrast with many of the enrichment activities described with respect to mastery
learning (Guskey, 2007), as these typically extend beyond the learning goals of the
lesson.
Variation theory (Marton, 2015) has the potential to inform both initial instruction
and teachers’ responses to student feedback, supporting both students who need
it and challenging those who have met the expectation of a lesson or a part of a
lesson. In this intervention, we have used strategies for continuous assessment
during class that are similar to Wiliam’s (2011) all-student response systems and
Wylie and Wiliam’s (2007) hinge questions; that is, questions are often framed
such that students may answer them in less than a minute, allowing the teacher to
make informed “on-the-fly” decisions. These decisions can be informed by critical
features of the intended object of learning, as described by Marton (2015). For
instance, Sabbaghan, Preciado Babb, Metz, and Davis (2015) proposed a type of
micro-scaffolding informed by the variation theory of learning: After noticing that
not all students are providing the expected answers in short questions, the teacher
SUHVHQWVDQHDVLHUWDVNWRWKHVWXGHQWVíRQHWKDWHYHU\RQHFDQXQGHUVWDQGíDQGWKHQ
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
P. PRECIADO-BABB ET AL.
106
re-builds the concept through a series of tasks informed by structured variation
(Metz, Sabbaghan, Preciado Babb, & Davis, 2015). Students who more quickly
demonstrate an understanding of a particular aspect of the lesson are offered direct
extensions of completed tasks; these consist of small but thoughtful variations of
the completed task.
Our focus on teaching practices includes how teachers use educational resources
in the classroom. The research in this intervention also focused on teacher and student
interactions with provided resources, as well as consideration of the possibilities that
such resources offer for students’ learning opportunities.
DATA AND RESEARCH METHOD
The intervention was a design-based research (Cobb, Confrey, diSessa, Lehrer, &
Schauble, 2003) focused on teachers’ knowledge, the relationship with a particular
curriculum material, and the impact on students’ learning. Consequently, results
from the ongoing research have informed the direction of the intervention since
its beginning in 2012. The study initially involved one elementary school (K to 6
with around 150 students) with a long history of poor performance in mathematics,
high diversity in terms of language and ethnicity, and common issues related to
a low socio-economic status. We will refer to this school as School 1. Another
elementary school (K to 5 with a similar number of students) became involved in
the research in 2014. We refer to this school as School 2. Both schools received
JUMP Math materials including assessment-and-practice books for each student, a
teacher’s guide, and pre-designed Smart-Board slides for each teacher. Additionally,
mini-whiteboards were provided to every student as a means to assess their
understanding during instruction (Wiliam, 2011). The instructional practices
suggested in the lesson plans include direct instruction, problem solving, guided
discovery, and both individual and team work.
Teachers were expected to adopt and follow the curricular materials from JUMP
Math, participate in targeted professional development, and attend observations from
the research team. Professional development for teachers was planned in conjunction
with the research team, a representative from JUMP Math, and a representative from
the school district.
The data used for the findings reported in this chapter include weekly classroom
observations, video recordings of lessons, and year-end student and teacher
interviews. The research focused strongly on School 1. A member of the research
team observed each group on a weekly basis, providing feedback to teachers.
Video-recordings were scheduled once a month. However, for a variety of reasons,
including medical and maternity leaves and the reluctance of some teachers to be
video-taped, it was not possible to record all the videos according to plan. Table 1
shows the number of students and the number of videos recorded sorted by grade
level over two years at this school.
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
THE ROLE OF CONTINUOUS ASSESSMENT AND EFFECTIVE TEACHER RESPONSE
107
Table 1. Groups, students and number of videos for school 1
2014–2015 2015–2016
Groups Students Videos Groups Students Videos
Grade 1 19 7 Grade 1 19 7
Grade 1 20 1 Grade 1 22 9
Grade 2 25 1 Grade 2 17 9
Grade 2/3 27 0 Grade 2 17 8
Grade 4 24 8 Grade 3 27 6
Grade 5 19 7 Grade 4 18 4
Grade 6 18 1 Grade 5 24 8
Grade 6 18 7
There were fewer video recording sessions in School 2 than in School 1. During
the 2014–2015 school year, the research team only planned to video-record three
lessons during the year. The data from this school is shown in Table 2.
Table 2. Groups, students and number of videos for school 2, 2014–2015
Groups Students Videos
Grade 1 20 3
Grade 2 26 3
Grade 3 8 2
Grade 3 19 3
Grade 4/5 22 3
Grade 4 21 3
Grade 5 24 3
A classroom observation protocol was initiated in 2014 with the purpose of
identifying teaching practices that impact students’ engagement in, and learning of,
mathematics. Three team members, who observed lessons from participant teachers
weekly, participated in the development of this protocol. The protocol, which was
used to record written notes on each observation, initially contained more than ten
categories. Based on weekly discussions throughout the 2014–2015 school year, we
condensed the protocol to four categories and developed descriptors for four levels
within each category. In 2015, the categories and descriptors were refined as the
three researchers watched and rated the 20 video-recorded lessons from School 2.
The videos were analyzed and scored on a scale of 1 to 4 for each category. No
transcriptions were used for this analysis. We then checked for consistency among
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
P. PRECIADO-BABB ET AL.
108
members of the research team. Disagreement about ratings prompted further
discussion and refinement of descriptions for each category. After two rounds of
such refinement, our scores were very consistent. We describe the categories and
descriptors in the next section.
While these continue to evolve as we analyze further data, the key ideas have
remained stable. Here, we show how the categories may be used to identify
teaching practices that foster all students’ engagement in mathematical activities
and learning, thus preventing marginalization from opportunities to learn. The
findings presented in this chapter were selected from the data set to showcase how
particular teaching practices can either support students’ learning or marginalize
students during class.
FRAMEWORK FOR CLASS OBSERVATION
The four categories that are the focus of our class observations are presented in Table
3, including an overall guiding question for each category. The first three categories
correspond to the teaching practices we focus on in this chapter. Specific descriptors
for the rates we assigned to each category are still under develop and are beyond the
scoop of this chapter.
Table 3. Four categories for classroom observation
Category Guiding questions
Continuous assessment How does the teacher collect information from all students
to make decisions during class?
Teacher’s responses How does the teacher respond to the information from
students?
Attention to critical features How does the teacher prompt attention to critical features of
the concepts, procedures or ideas taught in class?
Student engagement How do students engage in the mathematical activities
during class?
Continuous Assessment
Higher scores in this category were assigned when the teacher effectively used
strategies for making student responses visible at several moments during class. To
do so, he or she might have looked at simple responses on individual students’ mini-
whiteboards or asked students to hold their hands near their chest and using their
fingers to show their answers; one way or other, the teacher systematically ensured
every child was checked. By contrast, lessons received a low score when the teacher
failed to assess all students or failed to obtain information at key points during the
lesson. A common source of low scores was when a teacher asked a question to
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
THE ROLE OF CONTINUOUS ASSESSMENT AND EFFECTIVE TEACHER RESPONSE
109
the group, one student gave the correct answer, and then the teacher moved on,
assuming that all students had requisite understanding to continue with the lesson.
Teachers’ Responses
As indicated before, there is a substantial body of literature with suggestions
for formative assessment, but there is scarce indication of what to do with the
information collected from these assessments. This category focuses on whether the
teacher responded to student feedback. Higher scores in this category were assigned
when the teacher used students’ responses to guide the next steps of the lesson. This
might include modifying or skipping pre-designed slides, practice pages, or parts of
a lesson as well as posing further challenges or extension to students who met the
goals of a lesson or part thereof. The lower scores in this categories corresponded
to lessons in which teachers continued with a planned lesson without considering
student feedback. Note that here we did not judge the effectiveness of the response,
but whether there was a response. The effectiveness of the response is addressed in
terms of effective variation in the following category.
Attention to Critical Features
This category was directly informed by Marton’s (2015) variation theory and
includes both the way material and activities were initially posed to students
and how teachers responded to student feedback. Such responses were typically
influenced both by teachers’ own knowledge and by the educational material used
for instruction. Descriptions of lessons scored at the higher level include attention to
critical aspects of well-identified objects of learning and the use of task sequences
that systematically varied one critical aspect at a time. High scores also included
opportunities for students to attend simultaneously to critical features mastered prior
to, or during, a particular lesson. Variations of a task or activity might be part of the
planned lesson or a response to student feedback. Lower scores for this category
included cases in which more than one object of learning was conflated in the lesson
(or lesson segment) or in which effective patterns of variation were not used to draw
attention to critical features. Questions or tasks may not have effectively built on
previous ones. Support for struggling students was limited to repeating an initial
explanation, while extra challenges for students who completed the work sometimes
failed to consider critical aspects, thereby resulting in student frustration. Low
scores for variation also occurred when students shared different strategies to solve
a problem, but the teacher did little to compare and contrast them.
Student Engagement
In this category we observed students’ participation in the mathematical activity of
the lesson. Lessons that scored higher included cases in which most or all students
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
P. PRECIADO-BABB ET AL.
110
participated without prompting in nearly all parts of the lesson, and students either
requested further challenges or created their own extensions for themselves or
their peers. Lessons that scored lower in this category included occasions in which
many students were waiting for help or extension. In some cases, large numbers
of students were incapable of completing the work without individual support; in
others, students engaged in activities not related to the lesson, such as reading a book
or drawing at their desks.
EPISODES OF TEACHING PRACTICES AND MARGINALIZATION
We used the developed framework to analyse further data from the research. We
selected four teaching episodes to showcase how different levels of attention to
critical features and continuous assessment were enacted. The first two episodes
serve to contrast assessment during class; both teachers offered strong patterns of
variation in their teaching and responses, but a key critical feature was overlooked
in the Episode 1, and weak assessment practices resulted in a lesson that was
largely remedial with many students waiting a long time for the help they needed.
Episodes 2 and 3 are presented to contrast teachers’ responses: In both lessons,
students were assessed and demonstrated early mastery of the respective topics.
In Episode 2, however, the teacher prompted students to move ahead, whereas in
Episode 3, the class spent an inordinate amount of time rehashing material that most
found easy. Episode 4 describes a lesson where the teacher used a strong assessment
and response pattern to help uncover a more effective pattern of variation than the
one offered in the resource.
Episode 1: Medium Variation, Weak Assessment, Weak Response
The object of learning in this Grade 2 lesson was for students to recognize the core
(i.e., the repeating part) of patterns for which the first and last terms are the same
(e.g., ABA ABA); such cores were separated for particular attention due to the
observed difficulty students often have in separating them from a longer sequence.
In this episode, we observed that (1) the manner in which the teacher assessed
made it difficult to address students’ struggles at the beginning of the class and
(2) attention to certain critical features of the intended mathematical object was not
prompted.
The teacher began the lesson by leading a class discussion on how to identify
patterns and their cores. She carefully drew attention to the meaning of patterns,
cores and terms, and she used contrasting examples to illustrate a core that starts and
ends with the same term:
Teacher: Who can tell me what the main part of a pattern is, so we
know there is a pattern out there?
One student: It’s the core.
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
THE ROLE OF CONTINUOUS ASSESSMENT AND EFFECTIVE TEACHER RESPONSE
111
Teacher: It’s the core; that’s right. What do the cores do so that we
know that a pattern is happening?
Another student: Repeat itself.
Teacher:,WKDVWRUHSHDWLWVHOI,I,GLGWKLV>FODSSLQJWZLFHSXWWLQJ
KHUKDQGVRQKHUKHDGDQGSDXVLQJ@'R\RXNQRZZKDW
WKHFRUHLVRUQRW">Sequence 1@
Students in chorus: No.
The teacher then clapped twice, touched her head and put her hands on her knees
(repeated three times), and contrasted this sequence of moves with the initial
H[DPSOH>Sequence 2@
Teacher: To have a core, the core has to repeat in order for us to know what
the core looks like, correct?
This excerpt exemplifies features of variation and assessment. The teacher
contrasted two sequences of moves. In Sequence 1, it was not possible to identify
a pattern, because nothing repeated; Sequence 2 offered 3 repetitions of the core.
The variation in these examples prompted students’ attention to a critical feature of
repeating patterns: They must have a core that repeats. Regarding assessment, the
teacher received student feedback in the form of individual responses or from the
whole group in chorus.
This response pattern was similar in other parts of the episode. At one point, the
teacher called on three volunteers to generate patterns using hand signals. All of these
patterns had four-term cores with different start and end terms (e.g., ABAB, AABB,
ABBB). Following these, the teacher contrasted an example of a core beginning and
ending with the same term with one that did not; however, when she asked whether
both cores started and ended with the same term, only a few students responded. In
other words, although she attempted to use contrast to highlight the critical feature
of starting and ending with the same term, she did not assess whether all students
actually made this discernment.
The teacher then prompted attention to three cores that started and ended with the
same term. She asked individual students to identify the term that started and ended
the core in each case. She then provided a non-example. Again, she asked a single
student to identify whether the first and last terms were the same, and moved on once
the answer was provided. Next, the teacher showed a core (with the same first and
last term) and asked an advanced student to draw and repeat the core on the board
by placing the terms in pre-drawn circles. Two other cores were presented, and the
teacher drew attention to the matching beginning/ending terms. During this time,
only those students invited to the board were assessed.
The teacher then used three connecting cubes to create a core that began and
ended with the same color. She elicited what colors she would need to repeat the
core and had the students chant the colour sequence. The teacher gave each student
connecting cubes with which to create several copies of their own cores. They were
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
P. PRECIADO-BABB ET AL.
112
asked to join the cores to form a longer sequence (“core trains”), and then copy the
pattern onto grid paper.
Throughout this practice segment, students worked individually and the teacher
checked on their work by either walking to different tables or by sitting at a table
where students came to her to show their work. Many students had difficulty creating
core trains: They successfully created the first core, but began the second core with a
different color cube than the color of the ending cube of the first core. For example,
if the first core were red-blue-red, they would make the second one blue-red-blue,
resulting in an AB pattern (red-blue, repeat). In isolation, each core appears to have
just such an alternating pattern, and students had difficulty recognizing that the entire
core needed to repeat (red-blue-red, repeat). Once joined, the place where one core
ends and the other begins creates a spot where one colour repeats itself (ABAABA)
that is not evident in the isolated cores. As a result, students perceived the pattern as
an AB rather than an ABA pattern.
The teacher attempted to remediate this problem by asking students to first create
three identical cores and then join them. Many students were able to create trains
using these instructions. However, when they were asked to draw their trains, the
previous confusion between AB and ABA patterns persisted. To remediate, the
teacher asked students to dismantle their trains into core constituents and draw them
individually. In some instances, students tried to reconnect the cores in an effort to
make sense of why their drawings did not match their trains, but they were told to
stop what they were doing, break their trains into cores, and draw them. The teacher
then spent a great deal of time working with individual students in an attempt to
remediate their difficulties.
From this episode, we stress that the teacher attempted to draw attention to critical
features that would support students in designing patterns composed of cores with
matching start and end terms. However, students did not recognize that the place
where one term ends and the other begins results in a term being repeated at that
point in the pattern (e.g., ABAABA; ABBAABBA). As initial feedback was largely
based on individual student response, it took some time for the teacher to identify
and respond to this difficulty with a pattern of variation that contrasted separated
cores (ABA ABA), which can easily be perceived as a simple alternating pattern,
with combined cores (ABAABA) that remove the illusion of alternating colours.
Although such a contrast may have been effective earlier in the lesson, by this
point it was remedial, and most of the remediation was done through one-on-one
consultations with students as they worked (or waited).
Episode 2: Strong Variation, Strong Assessment, Strong Response
In this episode from a different Grade 2 classroom, we highlight a combination of
patterns of variation (both in the resource and in the way the teacher emphasized
important distinctions) aimed at helping students discern tens and ones in numbers
to 20. We draw particular attention to the teachers’ assessment of all students from
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
THE ROLE OF CONTINUOUS ASSESSMENT AND EFFECTIVE TEACHER RESPONSE
113
the beginning of the class. In this case, she adapted her planned lesson in response
to student feedback: Students seemed to understand the topic faster than expected
and the teacher decided to ‘pick up the pace.’ The episode is part of a longer lesson
aimed at counting up to 100.
The teacher opened the lesson by having all students count in unison to 20. She
put up a slide with numbers from 21 to 29 arranged vertically, which helped draw
attention to the constant two in the ten’s place and the increasing pattern in the one’s
place. She asked individual students to find and show the number after 20, then
21 and 22. Building on an insight from a student, she then drew attention to the
changing patterns in the one’s place. Before moving past 29, she contrasted 30 with
twenty-10, thir-zero, and thirteen (in terms of how they are written as well as how
they sound). The teacher asked students what came after thirty; a student said 31.
She then asked what comes next, and asked students to write the answer on their
mini-boards. Most wrote 32; she acknowledged this, and drew attention to the fact
that the 3 for thirty stayed the same, while the 1 changed to a 2. She then allowed
students to continue from 31 to 39, this time with each student writing the sequence
on their mini-whiteboards. All were successful, some with a bit of help as the teacher
circulated; they chanted the sequence together before she asked them what comes
after 39. Here, the teacher emphasized that the three needed to go up by one and
that the next number would have a “ty” on the end. One student identified 40, and
all continued from 41 to 49 on their whiteboards. Again, the teacher checked in to
see if students knew what came next; she emphasized the connection between “fif”
and “five” and reminded them of the “ty” suffix. At this point, she had not formally
assessed all students’ ability to move from a number ending in 9 to the next set of 10;
however, many students were moving ahead, and some could be heard complaining
that the work was too easy. She asked if they wanted to continue writing numbers to a
hundred. An excited chorus shouted, “Yeah!” All students reached 100 successfully,
and some went beyond.
In this episode we stress that the teacher used patterns of variation to draw
attention to the necessary features required for students to count and write numbers
to 50. She also used both visual and auditory cues to emphasize what was changing
from example to example. As in the previous episode, some assessment was based
on individual or choral response. In contrast to the previous episode, however, she
also frequently assessed all students by asking them to indicate their responses
on mini-whiteboards. The teacher responded to student assessment by offering
necessary support and by moving faster than planned. By the end of the first segment
of the lesson, the entire class was excited to continue writing numbers from 50 to
100 independently, with several going beyond.
Episode 3: Weak Variation, Medium Assessment, Low Response
This episode from a Grade 5 lesson focused on the number of sides and vertices
in polygons; students experienced high success early in the lesson, but there were
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
P. PRECIADO-BABB ET AL.
114
no effective extensions that allowed students to build on that success. As a result,
engagement remained low. This episode offers a marked contrast to the excitement
in the previous episode on counting to one hundred. It appears that all students
successfully counted the sides and vertices on various polygons that the teacher
offered to open the lesson. However, they then spent considerable time counting
sides and vertices on various shapes, without consideration, for instance, of whether
this would always be the case—which may have triggered a more significant
mathematical exploration. When a student asked if the number of sides and vertices
would always match, the teacher replied, “Try it and find out”! but did not follow
up to see how the student engaged with the question. The team member observing
this class challenged some students to try to find a polygon that did not have equal
numbers of sides and vertices, and many students spent considerable time exploring
this possibility. However, it was not discussed in the larger group. Instead, the lesson
moved on to defining “polygon” and to naming and sorting polygons with up to 5
sides. Again, students successfully completed independent work, but many finished
very quickly and spent a good deal of time waiting as the teacher seemed to stick to
the lesson plan instead of responding to student feedback that suggested that most, if
not all, students were ready to move on to more challenging work.
Episode 4: Medium Variation, Strong Assessment, Strong Response
In this episode, we show how a Grade 1 teacher made numerous attempts to support
students in representing numbers to 20 using unit and ten-blocks. The episode also
shows examples of extensions for students who met the expected outcomes during
class. In addition to providing several opportunities for each child to respond to
questions, this teacher paid regular attention to one student, Heidi, who struggled
far more than her classmates. Although the variation in the lesson provided in
the teachers’ guide omitted a variation that emerged as significant, the teacher’s
careful use of assessment, persistence in seeking an effective response, and effective
adaptation of the given variation allowed everyone to reach understanding by the
end of the lesson.
As in Episode 1, the teacher opened with a series of questions and took answers
either in chorus or from individual students. It was clear to the teacher, however,
that some students were giving wrong answers. In particular, there seemed to
be confusion about the number of ten-blocks and one-blocks used to represent
a particular number. The instructional sequence involved showing blocks for 11,
12, 13, 14, 15, and 18; students were to identify the number of ten-blocks and
unit blocks, then write the number. Note that in this sequence, the number of units
varied while the number of ten-blocks remained constant: There was always one
ten-block.
After offering the first item in the sequence (1 ten and 2 ones), the teacher checked
on each student individually. She then offered explanations for 10, 11, 12, and 13. At
this point, she asked the class to represent 14 and checked each student’s response.
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
THE ROLE OF CONTINUOUS ASSESSMENT AND EFFECTIVE TEACHER RESPONSE
115
Heidi was struggling, so the teacher gave her 4 one blocks and one ten-block. Then
she requested the attention of all students and provided an explanation on the board.
The teacher asked students to show blocks for 18 by themselves. She walked through
the class checking on students, offering assistance and posing further challenges
(19 and 17). This gave her some time to work individually with some students. Only
two or three students, including Heidi, still had trouble representing the numbers.
The class moved to the next part of the lesson, in which the students worked
in their assessment and practice books. The teacher introduced this work by using
a document camera to show an example solved in the students’ book. She then
asked individual students to solve the given exercises in front of the class. The page
showed tables with 2 rows of 10, numbered from 1 to 20. In the initial example, the
first row, with numbers 1 to 10, was marked with a darker color, as were the places
corresponding to the numbers 11 to 18 in the second row (see Figure 1).
Figure 1. Example from the assessment and practice book
Underneath this image, the following sentence was written:
18 is 1 tens bock and 8 ones blocks
The first practice exercise included a similar image with the top row coloured in and
5 blocks coloured in the second row. Beneath this was written:
15 is ___ tens block and ___ ones blocks
When the teacher asked, “How many ten-blocks,” some students answered in chorus,
“10.” The teacher then explained that there was only one ten-block. She gave every
student one ten-block, asking, “How many ten-blocks did I give you?” until every
student answered one. Similar questions asked students to identify 17 and 11.
Subsequent questions required students to identify tens and ones without the
support of individual charts: They were now supposed to place blocks on a given
20-chart to represent the given number, and then complete a sentence similar to the
previous page. For instance, the first task was:
14 is ___ tens block and ___ ones blocks
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
P. PRECIADO-BABB ET AL.
116
One student showed the answer on the board to the whole class, and the teacher
re-explained the use of one-blocks and ten-blocks. Subsequent tasks in the practice
book asked students to consider 19, 11, 13, 12, and 20. Note that so far, all but one
of the examples focus on a single ten paired with varying numbers of units (and that
one example comes at the very end). Given that effective variation uses contrast
to draw attention to critical features, it is perhaps not surprising that we observed
students who struggled to name the number of tens blocks.
Students started to work individually and the teacher walked through the
class checking on each student. At least three students, including Heidi, still had
trouble differentiating the ten-block from the one-block. At this point, the teacher
started giving these students more ten-blocks. She started with Heidi, showing three
ten-blocks and counting, “One, two three.” For the first time, the pattern of variation
now included contrasting numbers of ten-blocks.
The teacher went to another student, who had apparently written a 5 in the space
for ten-blocks; she indicated that in the case of 15, there were not 5 ten-blocks, but
only one ten-block. This student seemed to understand and corrected his work.
Another student had written 15 in the space for ten-blocks. The teacher showed
her 15 ten-blocks in her hand while saying, “Here are 15 ten-blocks” and asking,
³+RZ PDQ\ WHQEORFNV DUH WKHUH >LQ WKH JLYHQ H[DPSOH@"´ 7KH WHDFKHU JDYH RQH
ten block to the student, asking: “How many blocks did I give you?” The following
dialogue ensued:
Student: One.
Teacher gave another: Now how many blocks?
Student: Two .
Teacher took away one block and asked: OK? How many ten blocks did I give
you right now?
Student: One.
Teacher: Then write one.
The class finished, and the teacher asked Heidi to stay; she spent 4 minutes
working with her.
The teacher held up ten ten-blocks: “This is ten ten-blocks.” She gave the
blocks to her to hold in her hand.
Teacher: See ten blocks?
Heidi started giving the blocks black, one by one to the teacher. The teacher
counted while the student was giving the blocks back: “One, two three tens.”
Teacher: Can you show me 5 ten-blocks?
Heidi:2QHWZRWKUHHIRXUILYH>PRYLQJWKHWHQEORFNVRQHE\RQH@
Teacher: So that’s five ten-blocks. Now show me one ten block, just one.
>+HLGLPRYHGRQHEORFN@
Teacher: Now, how many ten-blocks do you have here?
Heidi: One.
Teacher: So, write the number one. How many one blocks? How many ones?
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
THE ROLE OF CONTINUOUS ASSESSMENT AND EFFECTIVE TEACHER RESPONSE
117
In this episode, we stress that the teacher led group discussions in which one student
or students in chorus responded to her questions, instead of using an all-student
response system. Nonetheless, it was clear to her that there was confusion about
the distinction between one ten-block, and ten unit-blocks. The teacher prompted
attention to this difference several times during the lesson, which included giving
one ten-block to each student and asking, “How many blocks did I give you?” Still,
some students were confused.
By the end of the lesson, the teacher started to vary the number of ten-blocks in
her explanation to students. This seemed to be more effective for the students who
were still confused. In the case of Heidi, this seemed to be more effective when she
actually counted the blocks—as opposed to the first time when the teacher showed
her the blocks. This use of contrast is a keystone of the variation theory of learning,
which takes as its fundamental conjecture that new meanings must be experienced
as difference against a background of sameness rather than as sameness against a
background of differences (Pang & Marton, 2013).
DISCUSSION
The episodes presented here serve to illustrate the three teaching practices we wish
to emphasize, corresponding to the three categories of the observation protocol:
We acknowledge the existence of other aspects of teaching quality not addressed in
this discussion, which have been omitted in order to highlight these categories. The
first two episodes presented in the previous section serve to contrast the all-student-
response approach to assessment encouraged at the intervention. While in Episode
1 the teacher only assessed responses from individual students or in chorus, the
teacher in Episode 2 used the whiteboards to assess all students at the same time. In
both episodes, the teachers assessed during class, but in the first, the teacher assessed
students one at a time after they were working independently rather than as a group
to determine readiness for independent work.
Two different teachers’ responses to student feedback are contrasted in
Episodes 2 and 3. In Episode 2, the teacher modified her plan by moving faster
in response to students’ feedback and comments, while in Episode 3 the teacher
did not move on, even though students seemed to be ready for the next part of the
lesson. The difference in student engagement was that in Episode 2, students were
excited about counting beyond 50 by themselves, whereas in Episode 3 students
were bored and therefore marginalized from expanding their mathematical
understanding.
In Episode 4, the teacher was responsive to student feedback. She provided
additional questions to students who completed the work early and attempted to
address the confusion between one ten-block and ten one-blocks in different ways,
including several explanations and giving one ten-block to each student. However,
it was at the end of the lesson, when she tried to vary the number of ten-blocks that
she seemed to find a more effective way of prompting attention to these differences.
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
P. PRECIADO-BABB ET AL.
118
Introducing this variation early during class may have prevented the need for
individual remediation, thereby allowing all students to move faster, or to engage
in more challenges. In future iterations of this lesson, this teacher may more quickly
invoke the more powerful patterns of variation she uncovered through her work with
students, allowing greater opportunities for early mastery and further extension for
more students. Nonetheless, we categorized it as a strong example in which careful
assessment and response coupled with appropriate adjustments to variation led to
favorable outcomes for all students.
CONCLUSIONS
The framework developed for classroom observation described in this chapter
served to identify teaching practices that can either support students’ learning
if used effectively or marginalize them from learning if not. The categories
are interrelated: Continuous assessment of all students can lead to timely and
effective responses to student feedback, and these may be informed by effective
variation of critical features. It is important to notice that effective variation
requires specialized mathematical knowledge to identify the critical features in
each lesson.
We did not intend to critique teachers, but to identify teaching practices that
prevent marginalization in terms of engagement in further mathematical learning. In
this study, we have witnessed teachers with a strong commitment to their students,
as evident in the way they responded to student assessment. However, for various
reasons, the enacted lessons did not always effectively support mathematical learning
for all students. We observed that the combination of assessment based on all-student
responses and variation that prompts attention to critical features during instruction
seemed to better support students’ understanding in class. Additionally, teacher
responses to student feedback can support both students who still struggle with the
content of the lesson and those students who meet the expected understandings at a
given moment.
Professional development sessions for participating teachers were included as part
of the intervention, and some of the aspects discussed in this chapter were addressed
in these sessions. Yet, we continued to observe marginalizing practices such as
lack of continuous assessment, lack of teacher response to student feedback, and
ineffective use of variation. In some cases, teachers who demonstrated an exemplary
lesson for certain lessons received lower scores on other lessons. In continuing our
research, we hope to better understand the mechanisms that could support teachers
in enacting these practices in their classrooms.
While we have identified the three teaching practices described in this chapter, we
are still in the process of refining and testing the complete descriptors for the levels
in the observations protocol. Once developed, this protocol could serve to support
teachers adopt these practices in the classroom through feedback from peers and
external observers.
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
THE ROLE OF CONTINUOUS ASSESSMENT AND EFFECTIVE TEACHER RESPONSE
119
REFERENCES
Aljarrah, A., Preciado Babb, A. P., Metz, M., Sabbaghan, S., Pinchbeck, G. G., & Davis, B. (2016).
Transforming mathematics classroom settings into spaces of expanding possibilities. In
M. Takeuchi, A. P. Preciado Babb, & J. Lock (Eds.), IDEAS 2016: Designing for innovation
selected proceedings (pp. 162–170). Calgary: Werklund School of Education. Retrieved from
http://hdl.handle.net/1880/51201
Bloom, B. S. (1968). Learning for mastery. Evaluation Comment (UCLA-CSIEP), 1(2), 1–12.
Bloom, B. S., Hastings, J. T., & Madaus, G. F. (Eds.). (1971). Handbook on the formative and summative
evaluation of student learning. New York, NY: McGraw-Hill.
Box, C., Skoog, G., & Dabbs, J. M. (2015). A case study of teacher personal practice assessment theories
and complexities of implementing formative assessment. American Educational Research Journal,
52(5), 956–983. doi:10.3102/0002831215587754
Bruner, J. (1960). The process of education. Cambridge, MA: Harvard University Press.
Chappuis, J. (2015). Seven strategies of assessment for learning (2nd ed.). Upper Saddle River, NJ:
Pearson Education.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational
research. Educational Researcher, 32(1), 9–13.
Da Ponte, J. P., & Chapman, O. (2016). Prospective mathematics teachers’ learning and knowledge for
teaching. In L. D. English & D. Kirshner (Eds.), Handbook of international research in mathematics
education (pp. 275–295). New York, NY: Routledge.
Guskey, T. R. (2007). Closing achievement gaps: Revisiting Benjamin S. Bloom’s “learning for mastery.”
Journal of Advanced Academics, 19(1), 8–31. doi:10.4219/jaa-2007-704
Guskey, T. R. (2010). Lessons of mastery learning. Educational Leadership, 68(2), 52–57.
Heidi, L., & Cizek, G. J. (Eds.). (2010). Handbook of formative assessment. New York, NY: Routledge.
Marton, F. (2015). Necessary conditions of learning. New York, NY: Routledge.
Metz, M., Sabbaghan, S., Preciado Babb, A. P., & Davis, B. (2015). One step back, three forward success
through mediated challenge. In A. P. Preciado Babb, M. Takeuchi, & J. Lock (Eds.), Proceedings
of the IDEAS 2015: Design responsive conference (pp. 178–186). Calgary: Werklund School of
Education. Retrieved from http://dspace.ucalgary.ca/handle/1880/50872
Mighton, J. (2007). The end of ignorance: Multiplying our human potential. Toronto: Alfred A. Knopf.
Mighton, J., Sabourin, S., & Klebanov, A. (2010). JUMP math 1 teachers’ resources: Workbook 1.
Toronto: JUMP Math.
Organization for Economic Co-Operation and Development. (2016). Equations and inequalities: Making
mathematics accessible to all. Paris: PISA & OECD Publishing. Retrieved from http://dx.doi.org/
10.1787/9789264258495-en
Pang, M., & Marton, F. (2013). Interaction between the learners’ initial grasp of the object of learning and
the learning resource afforded. Instructional Science, 41, 1065–1082.
Runesson, U. (2005). Beyond discourse and interaction. Variation: A critical aspect for teaching and
learning mathematics. Cambridge Journal of Education, 35(1), 69–87. Retrieved from http://doi.org/
10.1080/0305764042000332506
Sabbaghan, S., Preciado Babb, A. P., Metz, M., & Davis, B. (2015). Dynamic responsive pedagogy:
Implications of micro-level scaffolding. In A. P. Preciado Babb, M. Takeuchi, & J. Lock (Eds.),
Proceedings of the IDEAS 2015: Design responsive conference (pp. 198–207). Calgary: Werklund
School of Education. Retrieved from http://dspace.ucalgary.ca/handle/1880/50874
Stiggins, R. J., Arter, J. J., Chappuis, J., & Chappuis, S. (2004). Classroom assessment for students
earning: Doing it right – using it well. Portland, OR: Assessment Training Institute.
Wiliam, D. (2011). Embedded formative assessment. Bloomington, IN: Solution Tree Press.
Wiliam, D., & Leahy, S. (2015). Embedding formative assessment: Practical techniques for K-12
classrooms. West Palm Beach, FL: Learning Sciences International.
Wylie, C., & Wiliam, D. (2007). Analyzing diagnostic items: What makes a student response interpretable?
Paper presented at Annual Meeting of the National Council on Measurement in Education (NCME),
Chicago, IL. Retrieved from http://www.dylanwiliam.org/Dylan_Wiliams_website/Papers_files/
Paulino Preciado-Babb, Martina Metz, Soroush Sabbaghan and Brent Davis - 9789463512121
Downloaded from Brill.com07/03/2019 08:17:14PM
via free access
... Overall, those findings are in-line with traditional features of continuous assessment, i.e. the ability to apply corrective actions while the education is still ongoing [25,40] and the superior retention of information due to repeated testing spacedout over time [32]. These are also in-line with several observations given in [41]: (i) assessment should not encourage surface learning and (ii) adaptive assessment provides benefits to both summative and formative assessment. ...
... Further research should include broader student modeling. In line with [40], further research could also expand onto teacher responsiveness, which builds upon continuous results provided by the proposed assessment system. ...
Chapter
Full-text available
This chapter presents a model of a novel adaptive online knowledge assessment system and tests the efficiency of its implementation. System enables continual and cumulative knowledge assessment, comprised of sequence of at least two interconnected assessments, carried-out throughout a reasonably long period of time. Important characteristics of the system are: (a) introduction of new course topics in every subsequent assessment, (b) re-assessment of earlier course topics in every subsequent assessment iteration, (c) in an adaptive manner, based on student’s achievements during previous assessments. Personalized post-assessment feedback guides each student in preparations for upcoming assessments. The efficiency has been tested on a sample of 78 students. Results indicate that the proposed adaptive system is efficient on an individual learning goal level.
... While still evolving, our model has been shown to support robust and accelerated learning, with the most significant improvements in conceptual understanding and problem solving (Preciado-Babb et al., 2018;Davis and Metz, in press). Yet even with multiple years of evidence across multiple populations, we ran into an unexpected resistance when we set about to scale up the project a few years ago. ...
Article
Full-text available
There are hundreds, perhaps thousands, of “theories of learning” at play in the field of education. Given scant agreements on the meaning of “learning” and the purpose of “theory,” such quantity is perhaps unsurprising. Arguably, however, this situation is indefensible and debilitating in an academic domain so focused on interpreting and influencing learning. We describe our own efforts to come to terms with this matter. Oriented by Conceptual Metaphor Theory and network theory, we are attempting to “map” contemporary treatments of learning—whether implicit or explicit, written or spoken, descriptive or prescriptive, formal or informal, scientific or folk. We report on our iterative process, evolving design, and emergent insights. We discuss the potential relevance of this and similar efforts for the future of educational research and practice.
... JUMP Math is highly dedicated to improving mathematical performance, reducing mathematical anxiety as well as building student and teacher confidence towards mathematical learning (Solomon, et al., 2019). JUMP Math offers diverse and balanced materials that minimize differences in learning capabilities, in the hopes of narrowing the wide gap present in student performances (Preciado-Babb, et al., 2018). ...
... While the JUMP Math resource has carefully identified critical discernments to be noticed by learners and has sequenced topics coherently, we have observed that adapting the resource using more clearly structured variation (Marton 2015;Pang et al. 2016;Watson 2017) and remaining mindful of broader learning targets can provide better opportunities for students' learning. Doing so has opened pathways that both supported the weakest students and challenged even the most capable students Preciado-Babb et al. 2017). ...
Chapter
Full-text available
The use of systematic variance and invariance has been identified as a critical aspect for the design of mathematics lessons in many countries where different forms of lesson study and learning study are common. However, a focus on specific teaching strategies is less frequent in the literature. In particular, the use of systematic variation to inform teachers’ continuous decision-making during class is uncommon. In this chapter, we report on the use of variation theory in the Math Minds Initiative, a project focused on improving mathematics learning at the elementary level. We describe how variation theory is embedded in a teaching approach consisting of four components developed empirically through the longitudinal analysis of more than 5 years of observations of mathematics lessons and students’ performance in mathematics. We also discuss the pivotal role of the particular teaching resource used in the initiative. To illustrate, we offer an analysis of our work with a Grade 1 lesson on understanding tens and ones and a Grade 5 lesson on distinguishing partitive and quotitive division.
Article
Full-text available
Research with economic utility and social value has been increasingly valued. Such an emphasis can be evidenced by the newly included assessment element of 'societal impact' in performance-based research funding (PBRF) schemes in different higher education systems around the world. This paper investigates how the non-academic impact is constructed and perceived in the Hong Kong Research Assessment Exercise (RAE) 2020, taking into account the local socio-cultural characteristics in the context of Hong Kong. Data sources include 13 impact case studies in the education panel submitted for the Hong Kong RAE 2020 and semi-structured interview with 17 education academics in Hong Kong. Findings revealed that the non-academic impact was constructed through a narrative pattern: (1) problem identification: tensions and synergies between local and international discourse; (2) problem resolution: prioritization of the evidence-based applied education research (with funding); (3) resolution dissemination: strategic employment of promotional genre. The paper discusses how decolonization, academic en-trepreneurialism and collectivist culture have characterized the framing and understanding of the non-academic impact in the Hong Kong acade-mia, contributing to the discourse on neoliberalism in higher education by providing a nuanced, local perspective on the impact agenda. Policy implications for a more localized and flexible impact agenda are also provided.
Article
Full-text available
In the wake of the COVID-19 pandemic, Malaysia's education system has witnessed a significant transformation, prompting a shift towards lifelong learning and greater flexibility in learning assessments. A comprehensive analysis was conducted on 86 relevant studies conducted between 2013 and 2023 to provide an overview of important advancements and patterns in the areas of education transformation in Malaysia, flexible assessments, and lifelong learning. The reviewed journals were chosen from peer-reviewed scientific journals, books, conference proceedings, and renowned internet databases such as JSTOR, Google Scholar, and the Directory of Open Access Journals (DOAJ). Articles that were not authored in English or were published outside the specified timeline and criteria were excluded. The findings indicate a significant focus on the implementation of online and mixed learning, with a focus given to digital platforms and microcredentialing. The important contributions of Recognition of Prior Learning (RPL) efforts and collaborative ventures with industry and communities have also been identified as beneficial. The importance of teacher training and continuous professional development was emphasized for its pivotal role in promoting lifelong learning. The review confirms that these instructional changes have a positive effect on student engagement and the alignment of skills with real-world requirements especially the professional settings. These initiatives serve as a significant benchmark for other nations grappling with educational issues in the aftermath of COVID-19, establishing Malaysia as a frontrunner in pioneering educational practices.
Article
Full-text available
After four decades of research and development on language in mathematics classrooms, there is consensus that enhancing language is crucial for promoting students' mathematics learning. After briefly sketching the theoretical contexts for work on this topic, in this paper we present six design principles for instruction that enhances language for mathematics learning. We then review the research that provides an empirical foundation for these principles, (a) concerning the design of learning environments to enhance language for mathematics learning and (b) on teaching practices (including teacher moves and classroom norms) involved in the enactment of those designed learning environments. Without claiming completeness, this review of the state of development and research shows that some aspects of design and instruction that enhance language for mathematics learning have been well researched, whereas research gaps for other aspects persist.
Article
The institutional form and conception of Higher Education have changed through the growth of mass higher education, which in many national systems now operates on market logics. Drawing on theories of credentialism, this article provides a critical analysis of the inter-relationship between massification and marketization and examines a range of consequences this has for institutional relations and dynamics. A central feature of credential inflation in mass systems has been the growing competition for scarce status goods and the reproduction of structural inequalities in accessing sought-after occupational outcomes. The policy context of marketization has concurrently reinforced the pressures on institutions to fulfil the promise held by governments, employers and graduates of enhancing human capital and Higher Education institutions’ economic value. Accompanying New Public Management policy levers have further established institutional conditions based on competitive accountability and performative evaluation. We show how these pressures are manifested in new forms of instrumental rationality that valorize the commodification of academic credentials, and relatedly, studentship and academic scholarship. We finally consider the possible ways forward in appraising the goals of HE beyond credential inflation.
Article
Neoliberalism is invariably presented as a governing regime of market and competition-based systems rather than as a set of migratory practices that are re-setting the ethical standards of the academy. This article seeks to explore the way in which neoliberalism is shifting the prevailing values of the academy by drawing on two illustrations: the death of disinterestedness and the obfuscation of authorship. While there was never a golden age when norms such as disinterestedness were universally practiced they represented widely accepted aesthetic ideals associated with academic life. By contrast, neoliberal academics embrace a new set of assumptions and norms that stand in sharp relief to many of the values that were previously espoused. Practices that might have been regarded as ethically dubious by earlier generations of academics, such as grantsmanship, self-justificatory expressions of interestedness and tangential claims to authorship, are now regarded as legitimate and positive virtues in a more aggressive age of hyper-performativity.
Book
Full-text available
The Handbook of Formative Assessment in the Disciplines addresses current developments in the field, with a focus on domain dependency. Building from an updated definition of formative assessment, the book covers the integration of measurement principles into practice; the operationalization of formative assessment within specific domains, beyond generic strategies; evolving research directions including student involvement and self-regulation; and new approaches to the challenges of incorporating formative assessment training into pre-service and in-service educator training.
Article
Full-text available
The value and effectiveness of formative assessment in the classroom has gained an increasing amount of attention during the past decade, especially since the publication of seminal work by Black and Wiliam titled Assessment and Classroom Learning. Since that time, there has been a renewed interest in describing and evaluating teacher practices related to formative assessment. Based on evidence of its effectiveness in the classroom and on improving standardized test scores, many prominent educational entities have initiated reform efforts to promote the use of formative assessment, yet these practices have not been embraced by classroom teachers. This case study investigated internally constructed and externally imposed contextual elements that constrained or facilitated the use of formative assessment by three high school science teachers. Cornett’s curriculum development model of personal practice theories was modified to include assessment, termed personal practice assessment theories (PPATs), and chosen as a framework for the study. This research revealed distinct differences among the three teachers’ PPATs and several different factors that constrained or facilitated the use of formative assessment in their instruction. Most notable of these factors were the forms of teacher knowledge that played a critical role in shaping their assessment practices and had a bearing on their ability to convert espoused theories about assessment into actual classroom practice. Other externally imposed barriers that constrained the use of formative assessment included expectations, habits, and dispositions of students; the pressure that teachers felt to “cover” all of the curriculum in order to prepare students for the end-of-year, high-stakes exam; and an instructivist rather than constructivist approach to teaching and learning. Results from this study add to the growing body of knowledge about the complex terrain teachers negotiate in making teaching and assessment decisions and provides a framework for future studies.
Article
Full-text available
The problem of achievement gaps among different subgroups of students has been evident in education for many years. This manuscript revisits the work of renowned educator Benjamin S. Bloom, who saw reducing gaps in the achievement of various groups of students as a simple problem of reducing variation in student learning outcomes. Bloom observed that teaching all students in the same way and giving all the same time to learn—that is, providing little variation in the instruction—typically results in great variation in student learning. Students for whom the instructional methods and amount of time are appropriate learn well, and those for whom the methods and time are less appropriate learn less well. Bloom believed that all students could be helped to reach a high criterion of learning if both the instructional methods and time were varied to better match students' individual learning needs. In other words, to reduce variation in the achievement of diverse groups of students and have all students learn well, Bloom argued that educators and teachers must increase variation in instructional approaches and learning time. Bloom labeled the strategy to accomplish this instructional variation and differentiation mastery learning. Research evidence shows that the positive effects of mastery learning are not limited to cognitive or achievement outcomes. The process also yields improvements in students' confidence in learning situations, school attendance rates, involvement in class sessions, attitudes toward learning, and a variety of other affective measures.
Article
Full-text available
In this article, the authors first indicate the range of purposes and the variety of settings in which design experiments have been conducted and then delineate five crosscutting features that collectively differentiate design experiments from other methodologies. Design experiments have both a pragmatic bent—“engineering” particular forms of learning—and a theoretical orientation—developing domain-specific theories by systematically studying those forms of learning and the means of supporting them. The authors clarify what is involved in preparing for and carrying out a design experiment, and in conducting a retrospective analysis of the extensive, longitudinal data sets generated during an experiment. Logistical issues, issues of measure, the importance of working through the data systematically, and the need to be explicit about the criteria for making inferences are discussed.
Article
Full-text available
Article
The main ideas of the book are: • We can increase student achievement by changing our assessment practices • To do this, we need to assess accurately and involve students Why I chose this book: Traditionally, we have thought of assessment as a way to measure student learning. I like that the authors propose a powerful new way to use assessment to improve learning and motivate students. They suggest we use a formative type of assessment called assessment for learning which occurs during learning and has a great deal of student involvement. The book contains concrete instructional strategies to excite educators and breathe new life into their assessment practices. The book, and the CD/DVDs that accompany it, serve as professional development tools to help educators learn these new strategies. I also chose the book because the authors of the book are leading experts on formative assessment. They are better known on the west coast and all instructional leaders should know their work. How the summary is organized The book outlines five practices that are necessary to make sure classroom assessment is accurate and effective. The summary is divided into five sections, each of which discusses one of these five key components of sound assessment practice: The Scoop (In this summary you will learn…) √ How to use assessment to benefit students (I. Clear Purpose, pp. 1-2) How many teachers have given a low grade to a student and been surprised that this doesn't motivate the student to perform better? Instead, learn how formative assessment is a better way to increase student motivation and learning. √ How to create clear learning targets (II. Clear Goals, p. 3) Imagine a student returning home and instead of saying, "We did reading today," being able to say, "I learned to make good inferences. This means I can make a guess that is based on clues from the story." Learn how to clarify learning goals. Imagine a conference led by a student who has found, from looking at her corrected math test, that she has trouble multiplying three-digit numbers. She sets a concrete goal with a plan and a time for a retest. Learn to make communication tools, such as conferences, more effective. √ How to involve students (V. Student Involvement, pp. 9-10) Instead of handing students a "B," imagine students looking at samples of writing and designing their own rubrics to assess writing quality. In this summary, there are 24 activities that can be used to involve students in taking ownership of their learning.
Article
Two studies are reported in this paper. The object of learning in both is the economic principle of changes in price as a function of changes in the relative magnitude of changes in demand and supply. The patterns of variation and invariance, defining the conditions compared were built into pedagogical tools (text, graphs, and worksheets). The first study is the latest in a series of studies aiming to test the fundamental conjecture of the Variation Theory of Learning that new meanings are acquired from experiencing differences against a background of sameness, rather than experiencing sameness against a background of differences. The study compares the learning outcomes under conditions consistent with the basic conjecture with the learning outcomes under conditions not consistent with the theory. The results support the conjecture. The second study shows, however, that the conditions that are consistent with the theory cannot be decided unless the learners’ pre-requisites for the task in question are taken into consideration. One set of the pedagogical tools was found to be highly effective for learners with a better initial grasp of the object of learning, while another set was found to be equally effective for learners with a weaker initial grasp of the object of learning. The two sets were equally ineffective when used for the “wrong” group of learners.