Content uploaded by Robin Holding Kay
Author content
All content in this area was uploaded by Robin Holding Kay on Nov 02, 2018
Content may be subject to copyright.
Developing Guidelines for Creating Effective Mathematics Video
Podcasts for Secondary School Students
Robin Kay
UOIT
Canada
robin.kay@uoit.ca
Abstract: This paper presents comprehensive, evidence-based guidelines for creating and designing video
podcasts for mathematics at the secondary-school level. Sixteen design guidelines, organized into four
categories (establishing context, providing effective explanations, minimizing cognitive load, and engaging
students), were used to develop 59 videos and tested on 856 secondary school students. Overall, the majority
of students noted that the video podcasts were useful and helped them understand mathematics better. With
respect to establishing context, the evidence suggested that problem selection was appropriate and video
podcasts were clear, straightforward, and detailed. Regarding the quality of explanations, many students
commented on the effectiveness of the step-by-step presentation of solutions and the use of visuals to support
learning. Students agreed that video podcasts were easy to read, but did not directly comment on other
guidelines related to cognitive load. In addition, students noted that video podcasts were engaging and better
than using textbooks. They also enjoyed working on the interactive student-problems. Finally, significant
gains were observed in the self-rated knowledge categories evaluated. It is concluded that the guidelines
proposed in this study are a reasonable starting point for pre- and in-service mathematics teachers to create
effective video podcasts.
Introduction
While, limited research has been conducted on the creation, design and use of instructional video-podcasts, the
use of worked-example video podcasts has increased markedly since 2005 and the launch of You Tube ("You
Tube", 2011). As of January 2012, four billion YouTube videos, on average, were streamed each per day (Limer,
2012). Regularly, used for entertainment purposes, YouTube is also an open source for thousands of worked-
example video podcasts. Furthermore, portals like the Khan Academy, exclusively designed to distribute worked-
example video podcasts, are extremely popular among students (see http://www.khanacademy.org). Nonetheless,
limited attention has been focused on improving the effectiveness of worked-example video podcasts. If students
are going to use this type of learning media, it is important to identify characteristics that effect their impact on
learning. The purpose of the following study was to develop and assess a set of evidence-based guidelines for
designing worked-example video podcasts in mathematics.
Literature Review
To pinpoint key guidelines that might influence the effectiveness of worked-example mathematics videos, three
key areas of research were consulted: written worked examples (Atikinson et al., 2000; Clark & Mayer, 2008; Renkl,
2005; Kirschner, Sweller, & Clark, 2006; Zhu & Simon, 1987), multimedia learning (e.g. Clark & Mayer, 2008;
Mayer, 2005), and how people learn (e.g., Bransford, Brown, & Cocking, 2000; Sawyer, 2006). Based on an
comprehensive review of this research, 16 guidelines were identified as potentially significant in designing effective
mathematics video podcasts. These guidelines were organized under four main categories: establishing context,
creating effective explanations, minimizing cognitive load, and engaging students. Each category and its sub-
components are presented in Table 1.
Purpose of the Study
The evidence-based guidelines for creating and design video podcasts in Table 1 were used to develop 59 video
podcasts that focused on pre-calculus mathematics concepts. Two research questions were used to assess
effectiveness of the proposed video podcast guidelines:
1) What were student attitudes toward worked-example video podcasts? (survey data and comments)
2) How did student knowledge change as a result of using video podcasts? (self-report data)?
-2692-
SITE 2015 - Las Vegas, NV, United States, March 1-6, 2015
Table 1. Guidelines for Creating Video Podcasts for Mathematics
Establishing the Context
1. Problem Type: An appropriate problem is chosen for the concept being presented (Ball & Bass, 2000; Green,
1997; Schoenfeld, 1992; Willingham, 2009)
2. Clear Problem Label: The problem is clearly labelled and displayed at the beginning of the clip (Bransford,
Brown, & Cocking, 2000; Catrambone, 1994)
3. Background Information: The context and type of problem is clearly articulated at the beginning of the clip
(Bransford, Brown, & Cocking, 2000; Willingham, 2009)
4. Explain Key Elements: Key elements are clearly articulated before trying to solve it (Willingham, 2009; Clark &
Mayer, 2008)
Creating Effective Explanations
5. Meaningful Steps: Problem is broken down into meaningful chunks (Catrambone 1994; Clark & Mayer, 2008;
Kirschner et al., 2006; Mayer, 2005; Renkl, 2005)
6. Explain all steps: The reason for conducting each step is explained (Renkl, 2005)
7. Use of Visuals: Diagrams /pictures/tables used in the clips helped organize /clarify / illustrate key aspects of the
problem ((Atkinson, Derry, Renkl & Wortham, 2000; Clark & Mayer, 2008)
Minimizing Cognitive Load
8. Readability: The writing in the clips is easy to read (Chandler & Sweller, 1991; Sweller, 1988)
9. Write down key information: The important elements (terms /definitions /formulas/ procedures) are written down
as needed (Chandler & Sweller, 1991; Clark & Mayer, 2008)
10. Layout: The layout of the clips is easy to follow (Chandler & Sweller, 1991; Sweller, 1988; Clark & Mayer,
2008)
11. Highlighting: Key areas of the problem are visually emphasized (Chandler & Sweller, 1991; Sweller, 1988;
Willingham, 2009)
Engagement
12. Engaging Voice: The tone of the voice is engaging (Beck, McKeown, Sandora, Kucan, & Worthy, 1996; Clark &
Mayer, 2008; Moreno & Mayer, 2000, 2004)
13. Pace: The pace of the clip is good for learning (Chandler & Sweller, 1991; Sweller, 1988; Willingham, 2009)
14. Length of Clip: The clip is an appropriate length (Medina, 2008; Tapscott, 2009; Renkl, 2005)
15. Distractions: There were no behaviours/habits that would distract a student (Chandler & Sweller, 1991; Clark &
Mayer, 2008; Sweller, 1988)
16. Student problem: Student worked on their own problem while listening to the explanation of a teacher problem
(Bruner, 1986; Trafton & Reiser, 1993; Vannatta & Beyerbach, 2000)
Method
Sample
The student sample consisted of 856 (603 males, 253 females) secondary-school graduates from a large,
metropolitan area of over three million people. Students reported high school grades of 50-59% (4%, n=34), 60-69
(13%, n=107), 70-79 (36%, n=309), 80-89 (35%, n=295) and 90+ (13%, n=101).
Procedure
Students were sent a link to the video podcast repository three weeks prior to a pre-calculus diagnostic test.
After they received their results from the diagnostic test, they were asked to fill in a 10 to 15 minute survey
inquiring about their use of and attitudes toward video podcasts. Participation in this study was voluntary and
anonymous.
Data Sources
Survey Data. Students were asked to rate overall usefulness and six key features of video podcasts they used.
The internal reliability of the scale was 0.83. However, each item in the scale was analyzed independently to
acquire a more detailed understanding of student attitudes toward video podcasts.
Student knowledge. Students were asked to self-assess their pre-calculus knowledge before and after using the
video podcasts in five areas: working with functions, solving equations, linear functions, exponential and
-2693-
SITE 2015 - Las Vegas, NV, United States, March 1-6, 2015
logarithmic functions, trigonometric functions. This approach to assessing performance, although less precise than
an actual test, was used to preserve anonymity and authenticity of the survey data. Kuncel, Crede, & Thomas (2005)
conducted a meta-analysis and concluded that self-reported grades, in general, predict outcomes that at similar to
actual grades received.
Results
Student Attitudes Toward Video Podcasts
General impact. Overall, a majority of students rated the video podcasts as being useful (n=130; 29%) or very
useful (n=203; 45%). Students also appraised video podcasts very highly with average scores ranging from 3.9 to
4.4 on a five-point Likert scale (1=Strongly Disagree, 5 = Strongly Agree).
Establishing context. Student comments suggested that the problem selection for the video podcasts was
effective. These problems helped them review (n=22 comments) or remember (n=11 comments) previously
forgotten pre-calculus concepts, address specific gaps in knowledge (n=23 comments), or solve problems in which
they were having some difficulty (n=19).
Explanation. In addition to student comments about the overall quality of explanations and increased
understanding, students frequently noted the effectiveness of the step-by-step explanations. This was the guideline
remarked on the most (n=75 comments). Students also responded positively to the use of diagrams and other visuals
to help organize and/or clarify key components of a problem (n=35 comments).
Minimizing cognitive load. Nearly 90% of the students agreed or strongly agreed that the handwriting in the
video podcast was clear and easy to follow. Nevertheless, students did not directly comment on cognitive load
guidelines such as writing down key information, layout, or highlighting. Indirect evidence in the form of numerous
comments about step-by-step explanations indicated that a number of students were not overwhelmed by cognitive
load issues watching video podcasts.
Engagement. Seventy-five percent of the students agreed or strongly agreed that using video podcasts was
superior to using textbooks and that helpful learning tips were provided. Almost two thirds of the students agreed
or strongly agreed that they enjoyed doing the student problems. In addition, open-ended responses suggested that
students liked working on the student problem provided in each video podcast (n=25 comments).
Video Podcasts and Student Understanding
Students self-assessed five areas of pre-calculus knowledge before and after a two week period of using video
podcasts. Paired t-tests revealed significant gains in all five pre-calculus knowledge categories evaluated. The
effect sizes of these gains, based on Cohen’s d, ranged from 0.30 to 0.53 and are considered to be moderate
Discussion
Student Attitudes Toward Video Podcasts
Establishing context. Many students in this study responded positively to the problem selection claiming that
the video podcasts were successful in helping them remember or review, previously forgotten topics. Survey data
and student comments strongly suggested that worked-example video podcasts were high quality and easy to follow.
If the context of video podcasts were no sufficiently established, it is unlikely students would have rated ease of use
and quality so highly. However, this is a untested assumption and future research needs to ask this questions of
students directly.
Creating effective explanations. The data from this study suggested many students thought step-by-step
explanations were effective. This result was predicted by previous studies (e.g., Catrambone 1994; Clark & Mayer,
2008; Kirschner, 2006; Mayer, 2005; Renkl, 2005), but has not been explicitly confirmed with worked-example
video podcast examples. In addition, visual aids offered important learning support. As well, students preferred
dynamic visualization of a problem in a video format to the static presentation of a text-based format. This finding
is consistent with previous research stating that clear visuals are beneficial in the problem solving process (e.g.,
Atkinson et al., 2000; Clark & Mayer, 2008; Mayer et al., 1996).
Minimizing cognitive load. Most students agreed that the writing in the video podcasts was clear, although a
direct link to cognitive load was not established. One could argue that extraneous cognitive load was minimal given
that a majority of students judged video podcasts to be useful, clear, easy to follow, and effective at helping them
understand and learn previously forgotten pre-calculus concepts. If students were experiencing excessive
extraneous cognitive load, they probably would have remarked that the video podcasts were difficult to understand,
hard to follow, rushed, or confusing. No such comments were made. Nonetheless, more research is needed to
directly examine cognitive load and the effectiveness of worked-example video podcasts.
-2694-
SITE 2015 - Las Vegas, NV, United States, March 1-6, 2015
Engagement. Student comments design guidelines such as voice, pace, length of clip, and distractions were
sparse. However, nearly two thirds of the students enjoyed working through the student problem as the video
podcast was being presented. Other evidence suggesting that students were engaged included direct comments
about the video podcasts keeping their attention and interest. Additionally, 75% of the students thought working
with video podcasts was better than using textbooks. As was the case when analyzing student attitudes regarding
cognitive load, the evidence does not explicitly support the impact of specific design guidelines used to increase
engagement, with the exception of the student problem.
Video Podcasts and Student Understanding
Significant increases for all five categories of pre-calculus knowledge were observed. This finding is consistent
with student beliefs that that the worked-example video podcasts had a significant impact on their learning. Gains in
understanding and knowledge are also aligned with previous research on the impact of lecture-based video podcasts
(e.g., Alpay & Gulati, 2010; Crippen & Earl, 2004). It is important to realize that any conclusions about knowledge
gains are somewhat limited because the data collected was based on self-assessment. Furthermore, it would be
naive to assume that the only influence on learning was the use of worked-example video podcasts. Students may
have consulted textbooks, other web- based resources, and friends to augment their knowledge of pre-calculus
concepts.
Limitations and Future Research
This study examined and evaluated evidence-based guidelines for designing and creating mathematics-based
video podcasts. Measures to ensure the quality of the analysis included comprehensive review of the literature to
identify video podcasts design guidelines, collecting data from a large sample, and using multiple data collection
tools. Nonetheless, several important limitations are worth exploring to guide future research.
First, actual student behaviours and reactions while using worked-example video podcasts should be examined
to determine the intended impact of specific design guidelines. For example, think-aloud protocols, where a student
talks out loud while using a video podcasts, could provide more detailed information on what students attend to
while solving problems. In addition, interview or focus groups data might help provide useful information on which
design guidelines use are associated with negative or positive outcomes. Second, data collection tools on the
attitudes of students toward video podcasts should be refined to target specific design guidelines. Finally, formal
pre- and post-tests on pre-calculus would provide more rigorous data on the impact of video podcasts on student
learning.
References
Alpay, E., & Gulati, S. (2010). Student-led podcasting for engineering education. European Journal of
Engineering Education, 35(4), 415-427. doi: 10.1080/03043797.2010.487557
Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles
from the worked examples research. Review of Educational Research, 70(2), 181-214.
doi:10.3102/00346543070002181
Ball, D. L., & Bass, H. (2000). Interweaving content and pedagogy in teaching and learning to teach: Knowing and
using mathematics. In J. Boaler (Ed.), Multiple perspectives on the teaching and learning of mathematics (pp.
83-104). Westport, CT: Ablex.
Beck, I. McKeown, M. G. Sandora, C. Kucan, L. & Worthy, J. (1996). Questioning the author: A year-long
classroom implementation to engage students in text. Elementary School Journal, 96(4), 385-414.
Bransford, J. D., Brown, A. L., & Cocking, R. C. (2000). How People Learn: Brain, Mind, Experience, and School.
Washington, DC: National Academy Press.
Bruner, J. (1986). Actual minds, possible worlds. Cambridge, MA: Harvard University Press.
Catrambone, R. (1994). The effects of labels in example on problem solving transfer. In A. Ram, & K. Eiselt (Eds.),
Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society, (pp. 159-164). Hillsdale, NJ:
Erlbaum.
Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8(4),
293–332. doi: 10.1207/s1532690xci0804_2
Clark, R. C., & Mayer, R. E. (2008). E-Learning and the Science of Instruction. San Francisco: Pfeiffer.
Crippen, K. J., & Earl, B. L. (2004). Considering the effectiveness of web-based worked-example in introductory
chemistry. Journal of Computers in Mathematics and Science Teaching, 23(2), 151-167.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction foes not work: An
-2695-
SITE 2015 - Las Vegas, NV, United States, March 1-6, 2015
analysis of the failure of constructivist, discovery, worked example, experiential, and inquiry-base teaching.
Educational Psychologist, 41(2), 75-86. doi: 10.1207/s15326985ep4102_1
Kuncel, N. R., Crede, M., & Thomas, L. L. (2005). The validity of self-reported grade point averages, class ranks,
and test scores: A meta-analysis and review of the literature. Review of Educational Research, 75(1), 63-82.
Limer, E. (2012, January). YouTube hits 4 billion pageviews per day, just keeps growing. [Web log post].
Geekosystem. Retrieved from http://www.geekosystem.com/youtube-4-billion-pageviews/
Mayer, R. E. (Ed.) (2005). Cambridge Handbook of Multimedia Learning. New York: Cambridge University Press.
Mayer, R. E. (2005). Principles for managing essential processing in multimedia learning: Segmenting, pretraining,
and modality principles. In R. E. Mayer (Ed.), The Cambridge Handbook of Multimedia Learning. (pp. 169-
182). New York: Cambridge University Press.
Mayer, R. E., Bove, W., Bryman, A., Mars, R., & Tapangco, L. (1996). When less is more: Meaningful learning
from visual and verbal summaries of science textbook lessons. Journal of Educational Psychology, 88(1), 64-
73. doi: 10.1037/0022-0663.88.1.64
Medina, J. (2008). Brain Rules. Seattle, WA : Pear Press.
Moreno, R., & Mayer, R. E. (2000). Engaging students in active learning: The case for minimizing irrelevant
sounds in the design of multimedia instructional messages. Journal of Educational Psychology, 92(4), 724-
733. doi: 10.1037/0022-0663.92.4.724
Moreno, R., & Mayer, R. E. (2004). Personalized messages that promote science learning in virtual environments.
Journal of Educational Psychology, 96(1), 165-173. doi: 10.1037/0022-0663.96.1.165
Renkl, A. (2005). The worked-out examples principle in multimedia learning. In R. E. Mayer (Ed.) The Cambridge
Handbook of Multimedia Learning. New York: Cambridge University Press.
Sawyer, R. K. (Ed.) (2006). Cambridge Handbook of the Learning Sciences. New York: Cambridge University
Press.
Schoenfeld, A. H. (1992). Learning to think mathematically: Problem solving, metacognition, and sense-making in
mathematics. In D. Grouws (Ed.), Handbook for Research on Mathematics Teaching and Learning (pp. 334-
370). New York: MacMillan.
Sweller, J. (1988). Cognitive load during problem solving: effects on learning. Cognitive Science, 12, 257–285.
Tapscott, D. (2009). Grown up digital. Toronto, Ontario: McGraw Hill.
Tarmizi, R. A., & Sweller, J. (1988). Guidance during mathematical problem solving. Journal of Educational
Psychology, 80, 424-436. doi: 10.1037/0022-0663.80.4.424
Trafton, J. G., & Reiser, B. J. (1993). The contributions of studying examples and solving problems to skill
acquisition. In M. Poison (Ed.), Proceedings of the Fifteenth Annual Conference ~ the Cognitive Science
Society (pp. 1017-1022). Hillsdale, NJ: Erlbaum.
Vannatta, R. A., & Beyerbach, B. (2000). Facilitating a constructivist vision of technology integration among
education faculty and preservice teachers. Journal of Research on Computing in Education, 33(2), 132-148.
Willingham, D. T. (2009). Why don't students like school? San Francisco: Jossey-Bass.
YouTube (2001). In Wikipedia. Retrieved from http://en.wikipedia.org/wiki/YouTube .
-2696-
SITE 2015 - Las Vegas, NV, United States, March 1-6, 2015
