Attending and Responding to What Matters: A Protocol to Enhance Mathematics Pedagogy

Abstract

For four years we have invested in improving mathematics teaching at the elementary level. By drawing from diverse research emphases in mathematics education and by considering the impact of lessons in terms of student engagement and performance, we have identified four key elements impacting learning in mathematics. Here, we describe the protocol currently used to structure feedback for teachers in the Math Minds Initiative. The key elements that comprise the protocol are: (1) effective variation, (2) continuous assessment, (3) responsive teaching, and (4) engagement.

Full text

2017. In P. Preciado Babb, L. Yeworiew, & S. Sabbaghan (Eds.). Selected Proceedings of the
IDEAS Conference: Leading Educational Change, pp. 179-190. Calgary, Canada: Werklund
School of Education, University of Calgary.
ATTENDING A N D RESPON DING TO WHAT MATTERS: A PROTOCOL
TO ENHANCE MATHEMATICS PEDAGOGY
Martina Metz, Paulino Preciado-Babb, Soroush Sabbaghan, Brent Davis, and Alemu Ashebir
Werklund School of Education, University of Calgary
For four years we have invested in improving mathematics teaching at the
elementary level. By drawing from diverse research emphases in mathematics
education and by considering the impact of lessons in terms of student engagement
and performance, we have identified four key elements impacting learning in
mathematics. Here, we describe the protocol currently used to structure feedback
for teachers in the Math Minds Initiative. The key elements that comprise the
protocol are: (1) effective variation, (2) continuous assessment, (3) responsive
teaching, and (4) engagement.
Keywords: Variation Theory of Learning, continuous assessment, mastery learning,
intrinsic motivation
THE PROTOCOL
As part of the Math Minds Initiative, we have used a design-based approach to develop an
observation protocol that integrates data from classroom observations with research on formative
assessment (Wiliam, 2011), intrinsic motivation (Pink 2011), mastery learning (Guskey, 2010)
cognitive load (Clark, Kirschner, & Sweller, 2012), and variation theory (Marton, 2015). The
protocol is based on classroom observations spanning four years of weekly, bi-weekly, or monthly
observations of 10-15 teachers. Three researchers independently coded a set of 20 videos spanning

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 180
a year of practice with 7 teachers to begin refining and validating our descriptors. Here, we describe
the various elements of the protocol: (1) effective variation, (2) visible learning, (3) responsive
teaching, and (4) student engagement. We then offer a brief classroom illustration that uses the
terms of the protocol to describe a two-part lesson in which one part of the lesson yielded higher
levels of both success and engagement. Finally, we discuss the evolution of the protocol as we
further refine how we draw attention to each of its key emphases.
Effective Variation
The international comparison of teaching styles in seven countries did not show a clear pattern of
best teaching practices (Hiebert et al, 2003). Instead of focussing on teaching practises, Marton
(2015) suggested attending to critical aspects that a learner must discern; this is a necessary
condition for learning. We have used the Variation Theory of Learning (Marton, 2015) both as an
analytical framework to interpret given lessons and to inform teachers’ immediate responses to
student feedback at various checkpoints during a lesson. We have found that effective variation
respects the limits of working memory identified by Clark, Kirschner, and Sweller (2012), while
further offering clear strategies for directing attention toward key ideas: separation of key aspects
(through contrast), generalization (through induction), and fusion (through combination).
Because teachers in the project use a supplied resource, many aspects of variation are supported
and partially constrained by what is offered in the teachers’ guide, prepared slides, and student
materials. Teachers, however, must be aware of the distinctions made in the resource so that they
can effectively highlight them with students and so that they can effectively adapt the materials to
support diverse student needs (cf. Metz, Preciado-Babb, Sabbaghan, & Davis, 2017); we discuss
this further in the section on responsive teaching.

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 181
We have recently begun to use the term “ribboning” to describe the way key lesson elements are
separated for attention; this phrase stems from work with teachers to colour-code video-taped
recordings of their own lessons according to whether a selected moment involves instruction,
assessment, or practice. In doing so, we have found that effective lessons typically resulted in
narrow bands of colour that resemble ribbons. In other words, these lessons alternated frequently
between drawing attention to important discernments and checking whether students made the
intended discernments. This may be contrasted with lessons in which large chunks of information
are clustered, either in large instructional chunks or in problems that involve multiple new ideas.
For these and other reasons, we distinguish our work from discovery, direct, as well as somewhere-
in-between approaches to teaching mathematics (cf. Metz, Preciado-Babb, Sabbaghan, Davis,
Pinchbeck, & Aljarrah, 2016).
Visible Learning and Continuous Assessment
On one level, “visible learning” seems very simple: Make each child’s work visible such that the
teacher may offer feedback. Doing so is also key to the forms of mastery learning described by
Guskey (2010). Initially, however, we found that assessment during class more often took the form
of sampling students, either through volunteered or invited response. Often, this did not provide a
good indication of next steps required by those students whose responses were not observed.
“Making learning visible” also begs the question of what it is that needs to be made visible. As we
discuss further in the section on variation, we have seen that what teachers ask students to engage
in often combines multiple discernments that can become invisible to fluent users. These must be
separated to make them visible to the student, which can also make them visible to the teacher (cf.
Preciado-Babb, Metz, Sabbaghan, & Davis, 2016). Attending to ideas at this scale means
assessment must occur frequently within a single lesson; checking in at the end of a lesson, series

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 182
of lessons, or unit is not sufficient. Doing so informs a form of deliberate practice that emphasizes
aspects of performance that can be overlooked in the context of more complex tasks
(Christodoulou, 2016).
Continuous assessment does more than provide fine-grained opportunities for feedback from
teacher to student; each response also provides feedback from student to teacher, allowing the
teacher to make appropriate decisions about next steps. In doing so, there is a strong emphasis on
preventing misunderstanding rather than remediating: By starting at a level at which students are
capable, they may continuously extend understanding.
In observing teachers, we found that even when teachers made learning visible, they sometimes
proceeded without using feedback from students to inform the next steps of the lesson. For this
reason, we separate visible learning from responsive teaching.
Responsive Teaching
Once all students have been assessed, teachers have the difficult task of deciding what to do next.
If students were successful, what next step might press the boundaries of their understanding
without overwhelming? If students were unsuccessful, might they return to a place of success, then
proceed with a task that helps bridge the known and the unknown? Might the teacher offer a piece
of instruction or ask a guiding question that helps address an apparent gap? Might a clearer pattern
of variation help draw attention to a key idea that the student has not yet discerned?
Note the contrast this forms with common forms of remediation whereby a teacher supports
students in completing multiple steps of complex tasks with which they are struggling (cf.
Sabbaghan, Preciado-Babb, Metz, & Davis, 2015). If a student does require assistance, we
emphasize the importance of following up with a similar task that the student may then complete

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 183
independently. One of the most poignant messages from the youngest participants was that they
liked math best when they didn’t need help. If only some students are successful, task extensions
may be offered to those who are ready, thereby allowing the teacher to bring all students back to a
place where they might continue together.
In each of these cases, the goal is for all students to discern key ideas then continuously extend
their understanding (see also Preciado-Babb, Metz, Sabbaghan, & Davis, in press): nobody bored,
nobody waiting for help.
Engagement
Engagement serves a different purpose than the first three items in our protocol: Effective variation,
visible learning, and responsive teaching all point to teacher actions. Engagement, however, acts
more as a means of “global monitoring” of the impact of the lesson on student engagement: Again,
are all students given opportunities to both master and continuously extend their understanding? In
so doing, each new challenge may be conceived of as an intrinsic reward (Pink, 2011).
CLASSROOM ILLUSTRATION
Here, we offer a brief example of how the protocol may be used to interpret what is happening in
a first-grade classroom. The selected lesson was interesting in that it involved two parts, the first
of which was highly successful in terms of student engagement and understanding and the second
of which was less so. This allowed a clear contrast that involved the same teacher, same students,
same day, and same topic—with dramatically different results.
In the first part of the lesson, the teacher gathered the students on the carpet. She asked them to
identify a number pair that made ten. Someone suggested “3+7.” She wrote 3 + 7 = 10 on a mini-

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 184
whiteboard, thus ensuring that everybody was starting with a correct equation. Underneath that,
she wrote “3 + 8 = ___,” thus offering a direct juxtaposition of the two number sentences:
3 + 7 = 10
3 + 8 = ___
She drew attention to what had changed and asked students what the new sum would be. She started
with sums to ten and then moved to doubles; she also moved from adding one to one of the addends
to subtracting one from one addend. Each time she offered a new question, she monitored the
responses of all students and used their responses to gauge their readiness to move on. Each time
she moved to something new, she told the students that since they were doing so well, she was
going to give them a “stumper” and asked them to look closely to see what had changed. In other
words, she carefully ribboned the pieces she wanted students to notice, assessing each child along
the way, adapted her sequence according to the readiness of the students, and drew careful attention
to important connections within the lesson and to previous lessons on doubles and making ten.
In the second part of the lesson, the teacher placed a set of laminated cards with equations of the
form a + b = ____ on each of the tables in the classroom. Pairs of students were instructed to select
a card and write the sum of the two addends. One partner was then to create a variation of the
original on the back of the card; they could either add or subtract one from either of the addends.
The other partner was to answer the new question. Then they were supposed to switch roles. It
turned out that this was much more difficult than the opening set of tasks; most students understood
that they could change one addend, but the seemingly simple act of transferring the equation to the
back of the card required them to hold the entire equation in their memories; they could no longer
see the contrast directly. This made it much more difficult for students to attend to the intended
relationship between the two equations. Also, sometimes they did not answer the given question

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 185
correctly, which of course led to errors in the derived equation. In the initial lesson sequence, the
teacher intentionally offered new variations as the students demonstrated their growing
understanding; now the starting equations were more random and remained limited by the
parameters to add or subtract one to each addend. Some students broke this rule and added more
than one, but the receiving partner was not always ready for this extension, particularly since strong
students were paired with weaker students. Assessment during this portion of the lesson was
largely dependent on students checking one another; the teacher circulated and corrected the
mistakes that she noticed, but she could not be with every group all the time. Part-way through this
activity, she told students they could write the new equation on the same side as the original; this
was helpful for some. However, those ready for a greater challenge were still left practicing more
of the same with partners who were not ready to extend further. Perhaps both partners would have
been ready for greater challenge had the initial approach been gradually extended, say, to include
adding or subtracting more than one or to adjusting both addends at once.
The protocol has helped us draw attention to aspects of lessons that impact student understanding
and engagement and has thus become a valuable tool to use with teachers, both for planning and
feedback.
IMPLICATIONS AND NEXT STEPS
While observing lessons, we have noticed that it is possible for a teacher to use clear patterns of
variation to parse and sequence content, to carefully assess all students, to adapt in response to
student feedback, and to maintain strong engagement—all without adequately addressing broader
connections within or between lessons. Although doing so is key to strong variation, we have been
exploring the possibility of separating variation that is used to introduce new ideas and that which
explicitly connects and integrates those ideas. Doing so points to an evolving relationship between

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 186
teacher and resource; while effective variation can and should be pre-planned to a certain degree,
and a strong resource can do much to support this, there is much a teacher can do to draw attention
to these connections. Furthermore, what defines effective variation must be based on adaptive
response to student feedback, which often means teachers need to create their own examples, either
to support struggling learners or to extend work beyond that which is offered.
In working with teachers with the protocol, we have begun to develop models for teaching and for
teacher learning. Figure 1 presents an emerging framework that integrates key aspects of the
observation protocol in a model that serves these ends. We also include an appendix that we have
used with teachers to elaborate each section of the model.
Ribboning
Monitoring
Adapting
Connecting
Use structured
variation to draw
attention to
potentially novel
discernments
necessary to a
concept.
Ensure every student
is able and obligated
to provide feedback
to the teacher in
response to each
ribboned query.
Revise/devise tasks,
explanations, and
other engagements to
fit with demonstrated
understandings.
Move between “part”
and “whole” when
ribboning to ensure
that learners do not
lose sight of the
concept(s) under
study.
Figure 1: Math Minds Principles (adapted from Davis, 2016)
Note that here, the protocol’s emphasis on engagement is included with adapting; i.e., if teachers
effectively adapt their lessons in response to the needs of both high and low learners, they meet the
protocol’s criteria for engagement. Effective variation is essential to each aspect of the model: The
planned lesson and task sequence should incorporate clear patterns of variation (both to separate
and re-connect), and adaptive response requires further attention to how variation is structured.
In summary, we have found that the ways concepts are broken down, extended and re-connected,
the ways attention is drawn to key ideas, the ways each child is assessed, and the ways that
information from those assessments are used to inform next steps in a lesson are critical to our

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 187
work. Moving forward, we plan to involve more observers external to the project in further
validating and refining the criteria in our observation protocol. The key ideas of variation
(including connections), monitoring, and response, however, have proven effective and stable.
References
Christodoulou, D. (2016). Making good progress? The future of assessment for learning. Glasgow:
Oxford University Press.
Clark, R., Kirschner, P., & Sweller, J. (2012). The case for fully-guided instruction. American
Educator 36(1), 6-11.
Davis, B. (2016, November 28). Keynote speech presented at The Art and Science of Math
Education, Carleton University, Ottawa, Ontario, Canada.
Guskey, T. (2010). Lessons of mastery learning. Educational Leadership 68(2), 52-57.
Hiebert, J., Gallimore, R., Garnier, H, Givvin, K.B., Hollingsworth, H., Jacobs, J., … Stigler, J.
(2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 video study.
Washington: National Center for Education Statistics.
Marton, F. (2015). Necessary conditions of learning. New York, NY: Routledge.
Metz, M., Preciado-Babb, A. P., Sabbaghan, S., Davis, B., Pinchbeck, G., & Aljarrah, A. (2016).
Transcending traditional / reform dichotomies in mathematics education. In Wood, M. B.,
Turner, E. E., Civil, M., & Eli, J. A. (Eds.), Proceedings of the 38th Annual Meeting of the North
American Chapter of the International Group for the Psychology of Mathematics Education,
(pp. 1252-1258). Tucson, AZ: The University of Arizona.
Metz, M., Preciado Babb, A. P., Sabbaghan, S., & Davis, B. (2017). Using variation to critique and
adapt mathematical tasks. In A. P. Preciado-Babb, L. Yeworiew, & S. Sabbaghan (Eds.)

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 188
Selected Proceedings of the IDEAS 2017 Conference, (pp. 169-178). Calgary, Canada:
Werklund School of Education.
Pink, D. (2011). Drive: The surprising truth about what motivates us. New York: Riverhead.
Preciado-Babb, A. P., Metz, M., Sabbaghan, S., & Davis, B. (Forthcoming). The role of continuous
assessment and effective teacher response in engaging all students. To appear in R. Hunter, M.
Civil, H. Eisenmann, N. Planas, & D. Wagner (Eds.), Mathematical discourse that breaks
barriers and creates space for marginalized learners. Sense Publishers.
Preciado Babb, A. P., Metz, M., Sabbaghan, S., & Davis, B. (2016). Fine-Grained, Continuous
Assessment for the Diverse Classroom: A Key Factor to Increase Performance in Mathematics.
In Proceedings of the American Education Research Association Annual Meeting 2016.
Sabbaghan, S., Preciado Babb, A. P., Metz, M., & Davis, B. (2015). Dynamic responsive
pedagogy: Implications of micro-level scaffolding. In A. P. Preciado Babb, Takeuchi, M., &
Lock, J (Eds.) Proceedings of the IDEAS 2015 Conference, (pp. 198-207). Calgary, Canada:
Werklund School of Education. http://dspace.ucalgary.ca/handle/1880/50874 .
Wiliam, D. (2011). Embedded formative assessment. Bloomington, IN: Solution Tree.

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 189
Appendix A: Planning for Teaching
Ribboning
Identify critical features. Use contrast to separate and generalize each; ensure that each
variation is posed as a question that can be assessed.
• Ensure fluency of requisite knowledge.
• Identify critical features that students need to discern.
• Start somewhere that allows all to engage, then build.
• Select a baseline all can reach, then refer to the rest as bonus.
• Provide clear explanations of what can’t be figured out (terminology, background, etc.); do
not let volunteer answers to questions substitute for teaching.
• Offer examples and non-examples (“yes – no – also”).
• Change only one thing from example to example or task to task.
• Change what you want to draw attention to.
• Name and highlight what you want students to notice.
• Organize to make contrasts obvious; juxtapose to allow easy comparison.
• Avoid visual and auditory clutter (e.g. extraneous teacher talk, cluttered boards, cluttered
space).
• Before moving from example to example or task to task, check each child for
understanding.
Monitoring
Make learning visible by asking questions that highlight each new understanding; assess every
child for every idea; use independent mastery as indicator of success.
• Monitor understanding of each key idea; no long segments of talk or practice without
checking each child for understanding.
• Ask questions that allow the teacher to know whether students have made key
discernments; e.g., plotting (1,1) on a Cartesian plane won’t tell you whether a student
discerns x from y.
• Allow time for all to respond; a few extra seconds can make a big difference.
• Attend first to the weakest students.
• Don’t rely on self-reports; many won’t respond honestly.
• Ensure frequent involvement of all (no lengthy discussions with single kids or long gaps
between when any given child has an opportunity to respond / be assessed).
• Avoid overuse of single-child response like hands-up or one student at the board.
• Look for the sense in students’ responses; these offer insight into what is needed; also,
they may have offered the correct response to the question they thought you asked
• When helpful, gather, juxtapose, and compare/contrast diverse student responses.
• Make students feel like they’re scaling mountains: e.g., “I don’t think I can make questions
any harder than that!” Don’t say, “This will be easy.”
• Get excited over new insights, even if they seem trivial.
• Aim for independent mastery. If a student needed help on a task, offer another.

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir
IDEAS 2017 190
Adapting
Decide whether next steps should step back (to address a newly identified critical feature or to
strengthen patterns of variation for one that has already been identified), offer further practice
(to allow independent mastery), or extend ideas for which students have demonstrated success;
consider potential bonus questions for those who are ready to move on before others.
If you encounter difficulty, step back, then proceed with a smaller step or a clearer pattern of
variation; do not attempt to walk students through tasks that require multiple new
discernments.
• Contrast errors with correct responses to highlight the source of error.
• Fill in instructional gaps as needed.
• Continuously extend in clear increments.
• Allow enough practice for students to experience independent mastery.
• Move quickly.
• Include bonuses for all; have extra bonuses for quick finishers.
Connecting
Vary multiple critical features simultaneously (i.e., fuse); solve problems that combine
multiple understandings that have been previously discerned; ask students to extend variation
of particular features; ask students to manage variation by working systematically
• Ensure separated components are put back together. What is the big idea(s) students
should take away from the lesson?
• Draw attention to connections (including logical relationships) between key lesson
components and between lessons.
• After individual variables have been varied one at a time, vary more than one at a time.
• Ask students to generate bonus questions (within clear parameters); reflect on what makes
them effective / ineffective.
• Ask students to solve problems that involve multiple ideas that have been previously
discerned.
• Teach students to manage variation by working systematically and attending to the patterns
and relationships that emerge as they do so.