Content uploaded by Armando Paulino Preciado-Babb

Author content

All content in this area was uploaded by Armando Paulino Preciado-Babb on Jun 16, 2017

Content may be subject to copyright.

Content uploaded by Martina Metz

Author content

All content in this area was uploaded by Martina Metz on May 02, 2017

Content may be subject to copyright.

2017. In P. Preciado Babb, L. Yeworiew, & S. Sabbaghan (Eds.). Selected Proceedings of the

IDEAS Conference: Leading Educational Change, pp. 179-190. Calgary, Canada: Werklund

School of Education, University of Calgary.

ATTENDING A N D RESPON DING TO WHAT MATTERS: A PROTOCOL

TO ENHANCE MATHEMATICS PEDAGOGY

Martina Metz, Paulino Preciado-Babb, Soroush Sabbaghan, Brent Davis, and Alemu Ashebir

Werklund School of Education, University of Calgary

For four years we have invested in improving mathematics teaching at the

elementary level. By drawing from diverse research emphases in mathematics

education and by considering the impact of lessons in terms of student engagement

and performance, we have identified four key elements impacting learning in

mathematics. Here, we describe the protocol currently used to structure feedback

for teachers in the Math Minds Initiative. The key elements that comprise the

protocol are: (1) effective variation, (2) continuous assessment, (3) responsive

teaching, and (4) engagement.

Keywords: Variation Theory of Learning, continuous assessment, mastery learning,

intrinsic motivation

THE PROTOCOL

As part of the Math Minds Initiative, we have used a design-based approach to develop an

observation protocol that integrates data from classroom observations with research on formative

assessment (Wiliam, 2011), intrinsic motivation (Pink 2011), mastery learning (Guskey, 2010)

cognitive load (Clark, Kirschner, & Sweller, 2012), and variation theory (Marton, 2015). The

protocol is based on classroom observations spanning four years of weekly, bi-weekly, or monthly

observations of 10-15 teachers. Three researchers independently coded a set of 20 videos spanning

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 180

a year of practice with 7 teachers to begin refining and validating our descriptors. Here, we describe

the various elements of the protocol: (1) effective variation, (2) visible learning, (3) responsive

teaching, and (4) student engagement. We then offer a brief classroom illustration that uses the

terms of the protocol to describe a two-part lesson in which one part of the lesson yielded higher

levels of both success and engagement. Finally, we discuss the evolution of the protocol as we

further refine how we draw attention to each of its key emphases.

Effective Variation

The international comparison of teaching styles in seven countries did not show a clear pattern of

best teaching practices (Hiebert et al, 2003). Instead of focussing on teaching practises, Marton

(2015) suggested attending to critical aspects that a learner must discern; this is a necessary

condition for learning. We have used the Variation Theory of Learning (Marton, 2015) both as an

analytical framework to interpret given lessons and to inform teachers’ immediate responses to

student feedback at various checkpoints during a lesson. We have found that effective variation

respects the limits of working memory identified by Clark, Kirschner, and Sweller (2012), while

further offering clear strategies for directing attention toward key ideas: separation of key aspects

(through contrast), generalization (through induction), and fusion (through combination).

Because teachers in the project use a supplied resource, many aspects of variation are supported

and partially constrained by what is offered in the teachers’ guide, prepared slides, and student

materials. Teachers, however, must be aware of the distinctions made in the resource so that they

can effectively highlight them with students and so that they can effectively adapt the materials to

support diverse student needs (cf. Metz, Preciado-Babb, Sabbaghan, & Davis, 2017); we discuss

this further in the section on responsive teaching.

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 181

We have recently begun to use the term “ribboning” to describe the way key lesson elements are

separated for attention; this phrase stems from work with teachers to colour-code video-taped

recordings of their own lessons according to whether a selected moment involves instruction,

assessment, or practice. In doing so, we have found that effective lessons typically resulted in

narrow bands of colour that resemble ribbons. In other words, these lessons alternated frequently

between drawing attention to important discernments and checking whether students made the

intended discernments. This may be contrasted with lessons in which large chunks of information

are clustered, either in large instructional chunks or in problems that involve multiple new ideas.

For these and other reasons, we distinguish our work from discovery, direct, as well as somewhere-

in-between approaches to teaching mathematics (cf. Metz, Preciado-Babb, Sabbaghan, Davis,

Pinchbeck, & Aljarrah, 2016).

Visible Learning and Continuous Assessment

On one level, “visible learning” seems very simple: Make each child’s work visible such that the

teacher may offer feedback. Doing so is also key to the forms of mastery learning described by

Guskey (2010). Initially, however, we found that assessment during class more often took the form

of sampling students, either through volunteered or invited response. Often, this did not provide a

good indication of next steps required by those students whose responses were not observed.

“Making learning visible” also begs the question of what it is that needs to be made visible. As we

discuss further in the section on variation, we have seen that what teachers ask students to engage

in often combines multiple discernments that can become invisible to fluent users. These must be

separated to make them visible to the student, which can also make them visible to the teacher (cf.

Preciado-Babb, Metz, Sabbaghan, & Davis, 2016). Attending to ideas at this scale means

assessment must occur frequently within a single lesson; checking in at the end of a lesson, series

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 182

of lessons, or unit is not sufficient. Doing so informs a form of deliberate practice that emphasizes

aspects of performance that can be overlooked in the context of more complex tasks

(Christodoulou, 2016).

Continuous assessment does more than provide fine-grained opportunities for feedback from

teacher to student; each response also provides feedback from student to teacher, allowing the

teacher to make appropriate decisions about next steps. In doing so, there is a strong emphasis on

preventing misunderstanding rather than remediating: By starting at a level at which students are

capable, they may continuously extend understanding.

In observing teachers, we found that even when teachers made learning visible, they sometimes

proceeded without using feedback from students to inform the next steps of the lesson. For this

reason, we separate visible learning from responsive teaching.

Responsive Teaching

Once all students have been assessed, teachers have the difficult task of deciding what to do next.

If students were successful, what next step might press the boundaries of their understanding

without overwhelming? If students were unsuccessful, might they return to a place of success, then

proceed with a task that helps bridge the known and the unknown? Might the teacher offer a piece

of instruction or ask a guiding question that helps address an apparent gap? Might a clearer pattern

of variation help draw attention to a key idea that the student has not yet discerned?

Note the contrast this forms with common forms of remediation whereby a teacher supports

students in completing multiple steps of complex tasks with which they are struggling (cf.

Sabbaghan, Preciado-Babb, Metz, & Davis, 2015). If a student does require assistance, we

emphasize the importance of following up with a similar task that the student may then complete

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 183

independently. One of the most poignant messages from the youngest participants was that they

liked math best when they didn’t need help. If only some students are successful, task extensions

may be offered to those who are ready, thereby allowing the teacher to bring all students back to a

place where they might continue together.

In each of these cases, the goal is for all students to discern key ideas then continuously extend

their understanding (see also Preciado-Babb, Metz, Sabbaghan, & Davis, in press): nobody bored,

nobody waiting for help.

Engagement

Engagement serves a different purpose than the first three items in our protocol: Effective variation,

visible learning, and responsive teaching all point to teacher actions. Engagement, however, acts

more as a means of “global monitoring” of the impact of the lesson on student engagement: Again,

are all students given opportunities to both master and continuously extend their understanding? In

so doing, each new challenge may be conceived of as an intrinsic reward (Pink, 2011).

CLASSROOM ILLUSTRATION

Here, we offer a brief example of how the protocol may be used to interpret what is happening in

a first-grade classroom. The selected lesson was interesting in that it involved two parts, the first

of which was highly successful in terms of student engagement and understanding and the second

of which was less so. This allowed a clear contrast that involved the same teacher, same students,

same day, and same topic—with dramatically different results.

In the first part of the lesson, the teacher gathered the students on the carpet. She asked them to

identify a number pair that made ten. Someone suggested “3+7.” She wrote 3 + 7 = 10 on a mini-

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 184

whiteboard, thus ensuring that everybody was starting with a correct equation. Underneath that,

she wrote “3 + 8 = ___,” thus offering a direct juxtaposition of the two number sentences:

3 + 7 = 10

3 + 8 = ___

She drew attention to what had changed and asked students what the new sum would be. She started

with sums to ten and then moved to doubles; she also moved from adding one to one of the addends

to subtracting one from one addend. Each time she offered a new question, she monitored the

responses of all students and used their responses to gauge their readiness to move on. Each time

she moved to something new, she told the students that since they were doing so well, she was

going to give them a “stumper” and asked them to look closely to see what had changed. In other

words, she carefully ribboned the pieces she wanted students to notice, assessing each child along

the way, adapted her sequence according to the readiness of the students, and drew careful attention

to important connections within the lesson and to previous lessons on doubles and making ten.

In the second part of the lesson, the teacher placed a set of laminated cards with equations of the

form a + b = ____ on each of the tables in the classroom. Pairs of students were instructed to select

a card and write the sum of the two addends. One partner was then to create a variation of the

original on the back of the card; they could either add or subtract one from either of the addends.

The other partner was to answer the new question. Then they were supposed to switch roles. It

turned out that this was much more difficult than the opening set of tasks; most students understood

that they could change one addend, but the seemingly simple act of transferring the equation to the

back of the card required them to hold the entire equation in their memories; they could no longer

see the contrast directly. This made it much more difficult for students to attend to the intended

relationship between the two equations. Also, sometimes they did not answer the given question

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 185

correctly, which of course led to errors in the derived equation. In the initial lesson sequence, the

teacher intentionally offered new variations as the students demonstrated their growing

understanding; now the starting equations were more random and remained limited by the

parameters to add or subtract one to each addend. Some students broke this rule and added more

than one, but the receiving partner was not always ready for this extension, particularly since strong

students were paired with weaker students. Assessment during this portion of the lesson was

largely dependent on students checking one another; the teacher circulated and corrected the

mistakes that she noticed, but she could not be with every group all the time. Part-way through this

activity, she told students they could write the new equation on the same side as the original; this

was helpful for some. However, those ready for a greater challenge were still left practicing more

of the same with partners who were not ready to extend further. Perhaps both partners would have

been ready for greater challenge had the initial approach been gradually extended, say, to include

adding or subtracting more than one or to adjusting both addends at once.

The protocol has helped us draw attention to aspects of lessons that impact student understanding

and engagement and has thus become a valuable tool to use with teachers, both for planning and

feedback.

IMPLICATIONS AND NEXT STEPS

While observing lessons, we have noticed that it is possible for a teacher to use clear patterns of

variation to parse and sequence content, to carefully assess all students, to adapt in response to

student feedback, and to maintain strong engagement—all without adequately addressing broader

connections within or between lessons. Although doing so is key to strong variation, we have been

exploring the possibility of separating variation that is used to introduce new ideas and that which

explicitly connects and integrates those ideas. Doing so points to an evolving relationship between

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 186

teacher and resource; while effective variation can and should be pre-planned to a certain degree,

and a strong resource can do much to support this, there is much a teacher can do to draw attention

to these connections. Furthermore, what defines effective variation must be based on adaptive

response to student feedback, which often means teachers need to create their own examples, either

to support struggling learners or to extend work beyond that which is offered.

In working with teachers with the protocol, we have begun to develop models for teaching and for

teacher learning. Figure 1 presents an emerging framework that integrates key aspects of the

observation protocol in a model that serves these ends. We also include an appendix that we have

used with teachers to elaborate each section of the model.

Ribboning

Monitoring

Adapting

Connecting

Use structured

variation to draw

attention to

potentially novel

discernments

necessary to a

concept.

Ensure every student

is able and obligated

to provide feedback

to the teacher in

response to each

ribboned query.

Revise/devise tasks,

explanations, and

other engagements to

fit with demonstrated

understandings.

Move between “part”

and “whole” when

ribboning to ensure

that learners do not

lose sight of the

concept(s) under

study.

Figure 1: Math Minds Principles (adapted from Davis, 2016)

Note that here, the protocol’s emphasis on engagement is included with adapting; i.e., if teachers

effectively adapt their lessons in response to the needs of both high and low learners, they meet the

protocol’s criteria for engagement. Effective variation is essential to each aspect of the model: The

planned lesson and task sequence should incorporate clear patterns of variation (both to separate

and re-connect), and adaptive response requires further attention to how variation is structured.

In summary, we have found that the ways concepts are broken down, extended and re-connected,

the ways attention is drawn to key ideas, the ways each child is assessed, and the ways that

information from those assessments are used to inform next steps in a lesson are critical to our

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 187

work. Moving forward, we plan to involve more observers external to the project in further

validating and refining the criteria in our observation protocol. The key ideas of variation

(including connections), monitoring, and response, however, have proven effective and stable.

References

Christodoulou, D. (2016). Making good progress? The future of assessment for learning. Glasgow:

Oxford University Press.

Clark, R., Kirschner, P., & Sweller, J. (2012). The case for fully-guided instruction. American

Educator 36(1), 6-11.

Davis, B. (2016, November 28). Keynote speech presented at The Art and Science of Math

Education, Carleton University, Ottawa, Ontario, Canada.

Guskey, T. (2010). Lessons of mastery learning. Educational Leadership 68(2), 52-57.

Hiebert, J., Gallimore, R., Garnier, H, Givvin, K.B., Hollingsworth, H., Jacobs, J., … Stigler, J.

(2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 video study.

Washington: National Center for Education Statistics.

Marton, F. (2015). Necessary conditions of learning. New York, NY: Routledge.

Metz, M., Preciado-Babb, A. P., Sabbaghan, S., Davis, B., Pinchbeck, G., & Aljarrah, A. (2016).

Transcending traditional / reform dichotomies in mathematics education. In Wood, M. B.,

Turner, E. E., Civil, M., & Eli, J. A. (Eds.), Proceedings of the 38th Annual Meeting of the North

American Chapter of the International Group for the Psychology of Mathematics Education,

(pp. 1252-1258). Tucson, AZ: The University of Arizona.

Metz, M., Preciado Babb, A. P., Sabbaghan, S., & Davis, B. (2017). Using variation to critique and

adapt mathematical tasks. In A. P. Preciado-Babb, L. Yeworiew, & S. Sabbaghan (Eds.)

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 188

Selected Proceedings of the IDEAS 2017 Conference, (pp. 169-178). Calgary, Canada:

Werklund School of Education.

Pink, D. (2011). Drive: The surprising truth about what motivates us. New York: Riverhead.

Preciado-Babb, A. P., Metz, M., Sabbaghan, S., & Davis, B. (Forthcoming). The role of continuous

assessment and effective teacher response in engaging all students. To appear in R. Hunter, M.

Civil, H. Eisenmann, N. Planas, & D. Wagner (Eds.), Mathematical discourse that breaks

barriers and creates space for marginalized learners. Sense Publishers.

Preciado Babb, A. P., Metz, M., Sabbaghan, S., & Davis, B. (2016). Fine-Grained, Continuous

Assessment for the Diverse Classroom: A Key Factor to Increase Performance in Mathematics.

In Proceedings of the American Education Research Association Annual Meeting 2016.

Sabbaghan, S., Preciado Babb, A. P., Metz, M., & Davis, B. (2015). Dynamic responsive

pedagogy: Implications of micro-level scaffolding. In A. P. Preciado Babb, Takeuchi, M., &

Lock, J (Eds.) Proceedings of the IDEAS 2015 Conference, (pp. 198-207). Calgary, Canada:

Werklund School of Education. http://dspace.ucalgary.ca/handle/1880/50874 .

Wiliam, D. (2011). Embedded formative assessment. Bloomington, IN: Solution Tree.

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 189

Appendix A: Planning for Teaching

Ribboning

Identify critical features. Use contrast to separate and generalize each; ensure that each

variation is posed as a question that can be assessed.

• Ensure fluency of requisite knowledge.

• Identify critical features that students need to discern.

• Start somewhere that allows all to engage, then build.

• Select a baseline all can reach, then refer to the rest as bonus.

• Provide clear explanations of what can’t be figured out (terminology, background, etc.); do

not let volunteer answers to questions substitute for teaching.

• Offer examples and non-examples (“yes – no – also”).

• Change only one thing from example to example or task to task.

• Change what you want to draw attention to.

• Name and highlight what you want students to notice.

• Organize to make contrasts obvious; juxtapose to allow easy comparison.

• Avoid visual and auditory clutter (e.g. extraneous teacher talk, cluttered boards, cluttered

space).

• Before moving from example to example or task to task, check each child for

understanding.

Monitoring

Make learning visible by asking questions that highlight each new understanding; assess every

child for every idea; use independent mastery as indicator of success.

• Monitor understanding of each key idea; no long segments of talk or practice without

checking each child for understanding.

• Ask questions that allow the teacher to know whether students have made key

discernments; e.g., plotting (1,1) on a Cartesian plane won’t tell you whether a student

discerns x from y.

• Allow time for all to respond; a few extra seconds can make a big difference.

• Attend first to the weakest students.

• Don’t rely on self-reports; many won’t respond honestly.

• Ensure frequent involvement of all (no lengthy discussions with single kids or long gaps

between when any given child has an opportunity to respond / be assessed).

• Avoid overuse of single-child response like hands-up or one student at the board.

• Look for the sense in students’ responses; these offer insight into what is needed; also,

they may have offered the correct response to the question they thought you asked

• When helpful, gather, juxtapose, and compare/contrast diverse student responses.

• Make students feel like they’re scaling mountains: e.g., “I don’t think I can make questions

any harder than that!” Don’t say, “This will be easy.”

• Get excited over new insights, even if they seem trivial.

• Aim for independent mastery. If a student needed help on a task, offer another.

Metz, Preciado-Babb, Sabbaghan, Davis, & Ashebir

IDEAS 2017 190

Adapting

Decide whether next steps should step back (to address a newly identified critical feature or to

strengthen patterns of variation for one that has already been identified), offer further practice

(to allow independent mastery), or extend ideas for which students have demonstrated success;

consider potential bonus questions for those who are ready to move on before others.

If you encounter difficulty, step back, then proceed with a smaller step or a clearer pattern of

variation; do not attempt to walk students through tasks that require multiple new

discernments.

• Contrast errors with correct responses to highlight the source of error.

• Fill in instructional gaps as needed.

• Continuously extend in clear increments.

• Allow enough practice for students to experience independent mastery.

• Move quickly.

• Include bonuses for all; have extra bonuses for quick finishers.

Connecting

Vary multiple critical features simultaneously (i.e., fuse); solve problems that combine

multiple understandings that have been previously discerned; ask students to extend variation

of particular features; ask students to manage variation by working systematically

• Ensure separated components are put back together. What is the big idea(s) students

should take away from the lesson?

• Draw attention to connections (including logical relationships) between key lesson

components and between lessons.

• After individual variables have been varied one at a time, vary more than one at a time.

• Ask students to generate bonus questions (within clear parameters); reflect on what makes

them effective / ineffective.

• Ask students to solve problems that involve multiple ideas that have been previously

discerned.

• Teach students to manage variation by working systematically and attending to the patterns

and relationships that emerge as they do so.