Examining the Creation of Video Podcasts to Improve the Quality of Mathematical Explanations for Pre-Service Teachers

Abstract

One of the primary skills required by secondary school mathematics teachers is to provide effective explanations to their students. Using Kay’s (2014) theory-based framework for creating instructional video podcasts, this study explored the quality of explanations embedded in mathematical instructional videos created by 37 pre-service teachers (female=26, male =11). Four criteria were assessed for potential strengths and opportunities for growth of mathematical explanations: (i) establishing context, (ii) explanation heuristics (iii) minimizing cognitive load, and (iv) engagement. With respect to context half of the students struggled with identifying key elements of their problem and providing an effective problem label. Regarding effective explanations, a majority of students were able to present and articulate steps required to solve problems, but many failed to use appropriate problem-solving strategies, use visual supports, offer tips for solving problems and identify potential errors. For minimizing cognitive load, most students presented problems in a well-organized clear layout, but many students needed to improve with respect to writing neatly, visually highlighting key areas, and listing key terms and formulas. Finally, with respect to engagement, the majority of students presented their explanations at a reasonable pace, but only half used a clear, conversational voice or offered concise explanations.

Full text

Examining the Creation of Video Podcasts to Improve the Quality of
Mathematical Explanations for Pre-Service Teachers
Robin Kay
University of Ontario Institute of Technology
Oshawa, Canada
robin.kay@uoit.ca
Robyn Ruttenberg-Rozen
University of Ontario Institute of Technology
Oshawa, Canada
Robyn.Ruttenberg-Rozen@uoit.ca
Abstract: One of the primary skills required by secondary school mathematics teachers is to provide
effective explanations to their students. Using Kay’s (2014) theory-based framework for creating
instructional video podcasts, this study explored the quality of explanations embedded in mathematical
instructional videos created by 37 pre-service teachers (female=26, male =11). Four criteria were assessed
for potential strengths and opportunities for growth of mathematical explanations: (i) establishing context,
(ii) explanation heuristics (iii) minimizing cognitive load, and (iv) engagement. With respect to context
half of the students struggled with identifying key elements of their problem and providing an effective
problem label. Regarding effective explanations, a majority of students were able to present and articulate
steps required to solve problems, but many failed to use appropriate problem-solving strategies, use visual
supports, offer tips for solving problems and identify potential errors. For minimizing cognitive load, most
students presented problems in a well-organized clear layout, but many students needed to improve with
respect to writing neatly, visually highlighting key areas, and listing key terms and formulas. Finally, with
respect to engagement, the majority of students presented their explanations at a reasonable pace, but only
half used a clear, conversational voice or offered concise explanations.
Introduction
Providing step-by-step explanations of specific procedural problems is an essential skill for secondary school
pre-service teachers, particularly in more formal subject areas such as mathematics or science (Atikinson et al.,
2000). Kirschner et al. (2006) provide substantial evidence that direct instruction through the use of well-explained
worked examples is particularly effective when students have a limited understanding of concepts to be learned.
However, actually developing those explanation skills, especially within pre-service mathematics teacher education,
is a complex process (Kay, 2014). According to Kay (2014), effective explanation of worked-examples involves at
least four areas: establishing the context of the problem, explanation heuristics, minimizing cognitive load, and
engaging students. Establishing context includes providing a clear problem label (Bransford, 1994), explaining what
the problem is asking (Willingham, 2009), and identifying the type of problem being solved (Ball & Bass, 2000).
Providing effective explanations includes breaking a problem into meaningful steps (e.g. Polya, 2004; Mason,
Burton & Stacey, 2010), explaining the reasoning for each step, and using visual supports (Atikinson et al., 2000;
Clark & Mayer, 2008; Renkl, 2005). Minimizing cognitive load encompasses factors such as presenting problems in
a well-organized clear layout, writing clearly, and drawing students’ attention to key aspects of the problem using
visual highlighting (Clark & Mayer, 2008; Willingham, 2009). Finally, engaging students while explaining worked-
examples refers to using a clear, personalized voice and proceeding at a pace that is suitable for learning (not too
fast, not too slow) and minimizing distractions (Atkinson et al., 2005; Clark & Mayer, 2008; Kester et al., 2006).
The use of video-podcasts to explain worked-examples has been examined by researchers, but not in detail
(Crippen & Earl, 2004; Kay, 2014; Loomes et al. 2002). The format, though, is ideal for critically investigating the
quality of explanations. The purpose of this study was to explore video podcast explanations of grade 7 and 8
mathematics concepts to identify areas of strength and challenge for pre-service teachers.
Method
Participants
-1590-
EdMedia + Innovate Learning 2019 - Amsterdam, Netherlands, June 24-28, 2019

Thirty-seven pre-service teachers (female=26, male=11), enrolled in an eight-month Bachelor of Education
program participated in this study. All pre-service teachers specialized in the teaching of mathematics for grades 7 to
12 (intermediate/senior level). English was the second language for 32% (n=12) of the participants.
Data Collection
The Instructional Video Evaluation Scale (IVES) was used based on Kay’s (2014) framework. The IVES was
comprised of four themes: establishing context (n=3 items), creating effective explanations (n=7 items), minimizing
cognitive load (n=4 items) and engagement (n= 5 items).
Procedure
All pre-service teachers were trained for two hours on how to use the software (Camtasia) and hardware
(laptops and Wacom tablets) to create instructional videos covering mathematics concepts from grades 7 or 8. They
were then given a detailed description of the criteria for each of the 10 IVES to help guide the creation of their
instructional mathematics videos. All students had 3 to 4 weeks to design and create their 4-6-minute video
podcasts. Key mathematics areas covered by the video podcasts included numbers and number sense, geometry and
spatial sense, patterning and algebra, data management and probability. The video podcasts varied in length from
111 to 373 seconds with a mean of 234 seconds (SD= 73.7). After the podcasts were submitted, they were evaluated
using the IVES. Each item in the IVES scale was scored on a three-point scale (0=No, 1=Sort of, 2=Yes) assessing
whether a particular quality of explanation was achieved in the video podcast.
Research Question
The main research question addressed in this study was: “What strengths and challenges in developing effective
mathematics explanations do pre-service mathematics teachers demonstrate in their video podcasts of grade 7 and 8
mathematics content?” Specifically, the IVES was used to explore four themes of explanation: context, explanation
qualities, cognitive load, and engagement.
Results
Establishing Context
The average context item score was 1.52 (SD=0.55), the highest of the four IVES themes. Seven out of ten
students were able to articulate the context and type of problem addressed in the video, six out of ten were
successful at noting the key elements for solving the problem, but only half the students fully achieved the criteria of
correctly labelling the problem (Table 1).
Table 1. Instructional Video Scale - Establish Context Items
Item Mean (SD) 1Item Fully
Achieved
Item Not2
Fully Achieved
Context and type of problem articulated 1.59 (0.64) 68% 32%
Key elements of problem explained 1.54 (0.61) 60% 40%
Clear problem label 1.46 (0.65) 54% 46%
1 Based on a 3-point Likert scale (0=No, 1=Sort of, 2=Yes)
2 Combination of “No” and “Sort of” responses
Creating Effective Explanations
The average explanation item score was 1.27 (SD=0.52), the lowest among the four IVES themes.
Approximately six out of 10 students were successful at showing and explaining all the key steps in their
mathematics problems and using the correct mathematical conventions. About half the students were able to offer a
suitable strategy or tip for solving problems. Only one-quarter of the students offered visuals to support their
explanations or noted potential errors one might make when solving the problem addressed in the video (Table 2).
Table 2. Instructional Video Scale – Effective Explanation Items
Item Mean (SD) 1Item Fully Item Not2
-1591-
EdMedia + Innovate Learning 2019 - Amsterdam, Netherlands, June 24-28, 2019

Achieved Fully Achieved
Show all key steps 1.59 (0.60) 65% 35%
Explain reasoning behind each step 1.49 (0.65) 57% 43%
Correct mathematics conventions used 1.46 (0.69) 57% 43%
Use appropriate strategy to solve problem 1.41 (0.64) 49% 51%
Offer tips for solving problems 1.35 (0.75) 51% 49%
Visuals used to support explanation 0.89 (0.74) 22% 78%
Noting potential errors that could be made 0.70 (0.88) 27% 73%
1 Based on a 3-point Likert scale (0=No, 1=Sort of, 2=Yes)
2 Combination of “No” and “Sort of” responses
Minimizing Cognitive Load
The average explanation item score was 1.43 (SD=0.42), the second highest among the four IVES themes.
Eight out of ten students provided a clear, organized layout for presenting their problems. Half the students had
legible writing and visually highlighted key points in the video explanations. Only one-third of the students listed
key supportive elements like key terms or formulas needed to solve the problems.
Table 3. Instructional Video Scale - Cognitive Load Items
Item Mean (SD) 1Item Fully
Achieved
Item Not2
Fully Achieved
Clear, organized layout of problem 1.59 (0.60) 81% 19%
Readability of writing 1.43 (0.60) 49% 51%
Visually highlighting key points 1.41 (0.69) 51% 49%
Listing supportive elements (terms, formulas) 0.70 (0.88) 32% 68%
1 Based on a 3-point Likert scale (0=No, 1=Sort of, 2=Yes)
2 Combination of “No” and “Sort of” responses
Engagement
The average explanation item score was 1.37 (SD=0.52), the third highest among the four IVES themes. Six
out of ten students limited distracting behaviour (e.g., saying “uhm” too often, clearing throat, poor sound) and
proceeded at a pace that was effective for learning a new concept. Half the students were successful at using a clear,
engaging conversational voice to present concepts and provide an explanation that was not too long or too short.
Table 4. Instructional Video Scale – Engagement Items
Item Mean (SD) 1Item Fully
Achieved
Item Not2
Fully Achieved
Limiting distractions 1.46 (0.80) 65% 35%
Effective pace for learning 1.43 (0.80) 62% 38%
Clear voice 1.46 (0.69) 54% 46%
Appropriate length of explanation 1.32 (0.78) 51% 49%
Engaging conversational voice 1.24 (0.80) 46% 54%
1 Based on a 3-point Likert scale (0=No, 1=Sort of, 2=Yes)
2 Combination of “No” and “Sort of” responses
Discussion
Four areas of potential development in mathematical explanation skills for pre-service secondary teachers were
assessed in this study: establishing context, creating effective explanations, minimizing cognitive load, and
engagement. Regarding establishing context, in their video podcasts, most pre-service teachers were able to
communicate the type of problem but struggled to a certain extent with articulating the key elements required to
solve the problem and providing a clear label of the problem. Pre-service teachers are only just beginning to unpack
their previously automatized mathematical knowledge in order to make their knowledge useful for teaching (Ball &
Forzani, 2009). In other words, the more automatized the knowledge (in this case grade 7 and 8 problems) the harder
it is to unpack and communicate the required steps to explain the mathematics concepts. Pre-service teachers might
-1592-
EdMedia + Innovate Learning 2019 - Amsterdam, Netherlands, June 24-28, 2019

need more direct guidance with elementary mathematics problems, or they may need to observe students trying to
solve these problems to better understand which elements are important for naïve or new learners (Santagata &
Bray, 2015). Not being able to provide a clear label may be related to new teachers not having an evolved schema of
how to organize and categorize problems. Without completely unpacking their own knowledge, beginning teachers
are able to solve the problems, but may not understand the big picture and therefore provide effective guiding
descriptions. Further research, perhaps in the form of interviews could be used to understand the challenges that
preservice teachers have with establishing context.
The difficulty that pre-service teachers experienced with respect to selecting an appropriate strategy and
offering tips to solve a problem in their podcasts highlights the need for teacher education programs to spend more
time explicitly focusing on the connections between problem-solving and making thinking explicit. Once the
connections are made, pre-service teachers can then be supported in converting that experience into providing
effective explanations of the problems for students. Pre-service teachers need to be cognizant of the difference
between solving and explaining problems. Additionally, visuals were rarely used to support explanations in the
podcasts. Pre-service teachers may be able to solve grade 7 and 8 mathematics problems, quite easily, without
visual aids, but clearly, they need instruction on the effectiveness of how this type of support might improve the
quality of their explanations (Arcavi, 2003). Finally, only one-third of preservice teachers were able to note potential
errors. Again, the advanced mathematical knowledge that pre-service teachers have, especially when solving
relatively straightforward grade 7 and 8 problems, probably undermines their ability to identify potential errors –
they simply do not make these errors. Consequently, mathematical error analysis for the purpose of creating
effective explanations is another useful activity that might be included in teacher education programs.
For cognitive load, most pre-service teachers presented problems in a clear, organized format. Readability of
writing, highlighting key points, and writing down supportive elements such as the definition of terms or formulas
proved to be more challenging. These are relatively easy areas to correct, and at the same time could have a
significant impact on the learning of their students, specifically by not overloading students.
Finally, with respect to the engagement of explanations, most students were relatively proficient at limiting
distracting behaviours and explaining at a pace that was neither too fast nor too slow. However, using a clear and
engaging conversational tone was more challenging. Voice may be more important in a video explanation compared
with a face-to-face setting, however, using a more personalized tone is likely more effective, regardless of the
environment. Changing voice may be one of the more challenging skills to develop with preservice teachers.
Overall, the IVES appeared to be an effective metric for analyzing video podcasts created by preservice
teachers and identifying potential opportunities for improvement. Future research might examine whether new
teachers can be taught to improve the quality of their explanations based on the feedback provided by the IVES.
References
Arcavi, A. (2003). The role of visual representations in the learning of mathematics. Educational Studies in
Mathematics, 52(3), 215-241.
Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles
from the worked examples research. Review of Educational Research, 70(2), 181-214.
Atikinson, R. K., Mayer, R. E., & Merrill, M. M. (2005). Fostering social agency in multimedia learning: Examining
the impact of an animated agent's voice. Contemporary Educational Psychology, 30(1), 117-139. doi:
10.1016/j.cedpsych.2004.07.001
Ball, D.L., & Bass, H. (2000). Interweaving content and pedagogy in teaching and learning to teach: Knowing and
using mathematics. In J.Boaler (Ed.) Multiple perspectives on mathematics of teaching and learning . (pp. 83-
104). Westport, Conn.: Ablex Publishing.
Ball, D.L., & Forzani, F.M. (2009). The work of teaching and the challenge for teacher education. Journal of
Teacher Education, 60(5), 497-511.
Bransford, J.D., Brown, A. L. and Cocking. R. C. (2000). How people learn: Brain, mind, experience, and school:
Expanded edition. National Academies Press.
Clarke, R. C., & Mayer, R. E. (2008). E-Learning and the science of instruction. San Francisco: Pfeiffer.
Crippen, K. J., & Earl, B. L. (2004). Considering the effectiveness of web-based worked example in introductory
chemistry. Journal of Computers in Mathematics and Science Teaching, 23(2), 151-167.
Kay, R. H. (2014). Developing a framework for creating effective instructional video podcasts. International
Journal of Emerging Technologies in Learning (iJET), 9(1), 22-30.
-1593-
EdMedia + Innovate Learning 2019 - Amsterdam, Netherlands, June 24-28, 2019

Kester, L., Lehnen, C., Van Gerven, P. W., & Kirschner, P. A. (2006). Just-in-time, schematic supportive
information presentation during cognitive skill acquisition. Computers in Human Behavior, 22(1), 93-112.
Renkl, A. (2005). The worked-out example’s principle in multimedia learning. In R.E. Mayer (Ed.) The Cambridge
handbook of multimedia learning, (pp. 229-245).New York: Cambridge University Press.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An
analysis of the failure of constructivist, discovery, worked example, experiential, and inquiry-based teaching.
Educational Psychologist,
Loomes, M., Shafarenko, A., & Loomes, M. (2002). Teaching mathematical explanation through audiographic
technology. Computers & Education, 38(1-3), 137-149. 41(2), 75-86. doi: 10.1207/s15326985ep4102_1
Mason, J. Burton, L. & Stacey, K. (2010). Thinking mathematically. Essex: Pearson.
Polya, G. (2004). How to solve it. Expanded Princeton Science Library Edition. Princeton: Princeton University
Press.
Santagata, R., & Bray, W. (2016). Professional development processes that promote teacher change: the case of a
video-based program focused on leveraging students’ mathematical errors. Professional development in
education, 42(4), 547-568.
Willingham, D. T. (2009). Why don't students like school? A cognitive scientist answers questions about how the
mind works and what it means for the classroom. San Francisco, CA: John Wiley & Sons.
-1594-
EdMedia + Innovate Learning 2019 - Amsterdam, Netherlands, June 24-28, 2019