Content uploaded by Matthew J Koehler
All content in this area was uploaded by Matthew J Koehler on Feb 18, 2014
Content may be subject to copyright.
If a picture is worth a thousand words is video worth a
million? Differences in affective and cognitive processing of
video and text cases.
Aman Yadav*, Ph.D.
Department of Educational Studies, Purdue University
Michael M. Phillips, Ph.D.
School of Psychological Sciences, University of Northern Colorado
Mary A. Lundeberg, Ph.D.
Matthew J. Koehler, Ph.D.
Educational Psychology and Educational Technology, Michigan State University
Katherine Clouse, Ph.D.
School of Teacher Education & Leadership, Radford University
Kathryn H. Dirkin, Ph.D.
Department of Teacher Education, Central Michigan University
*Corresponding Author Address: 100 N. University Street, West Lafayette, IN 47907,
phone: 765.496.2354, fax: 765.496.1228. Email: email@example.com
In this investigation we assessed whether different formats of media (video, text, and
video+text) influenced participants’ engagement, cognitive processing and recall of
non-fiction cases of people diagnosed with HIV/AIDS. For each of the cases used in
the study, we designed three informationally-equivalent versions: video, text, and
video+text. Thirty participants experienced one version in each format, thought aloud
as they read or viewed the case, discussed their reactions to the stories during an
interview, and completed an affective and engagement survey. Participants were
again interviewed six weeks later to assess their memory for the cases. Results from
protocol analysis indicate that the video and video+text versions of the stories led to
higher levels of both engagement and sympathy with the characters, and recall of
particular information; however, interactions between medium and content were
important. We argue the main benefit of video lies in engaging students emotionally
in the content.
Keywords: Cognitive Processes/Development; ANOVA/MANCOVA; Factor
Analysis; Survey Research; Learning Processes/Strategies
Aman Yadav’s research focuses on the affordances of video cases for preservice
teacher education and problem-based learning in Science, Technology, Engineering,
and Mathematics (STEM) disciplines. Michael Phillip’s research focuses on the
investigation of instructional practices and tasks that nurture student motivation. Mary
Lundeberg's research interests include problem-based and case-based pedagogy in
teacher education and science, interactive multimedia environments, scientific
literacy, and cultural and gender influences in confidence. Matthew Koehler’s
research and teaching focus on understanding the affordances and constraints of new
technologies; the design of technology-rich, innovative learning environments; and
the professional development of teachers. . Katherine Clouse’s research focuses on
informational comprehension in the primary grades. Kathryn Dirkin’s current
research focuses on using problems of practice to support technology integration in
Evaluating the effectiveness of video as an instructional medium has proven to
be complicated and somewhat contentious (e.g., see the debate by Clark, 1983, 1994;
Kozma,1994). In general, we concur with Mayer et al. (2009) that “media do not
cause learning, methods cause learning” (p. 53), and a promising area of research is to
examine the ways in which instructional technologies can be combined with
instructional methods to produce cognitive and affective processing during learning
(e.g., Mayer & Wittrock, 2006). Early studies often showed no benefit for video,
mixed results, or idiosyncratic findings, leading Clark (1983) to restate a similar claim
that had been made by others that “media do not influence learning under any
conditions” (p. 445). Researchers who disagree with Clark might concede, at the very
least, that media benefits are not simple to identify, and that media effects interact
with other educational factors (e.g., contexts, goals, social processes, etc.) in
complicated ways. Kozma (1994), for example argues for this contextual stance by
considering media to have affordances that “… interact with cognitive and social
processes” (p. 11).
Cognitive theory of multimedia based on dual coding theory
Paivio’s (1990) dual coding theory provides an initial framework for
understanding how multimedia information is processed. Paivio’s theory posits two
separate symbolic systems – one system is attuned to verbal information including
auditory processing and language and the other non-verbal system includes imagery,
visual and spatial processing. Dual coding goes beyond making a distinction between
separate processing of verbal and visual information, it suggests there is little
competition for resources when presenting visual and auditory information together,
so that multimedia representations have more powerful educational affordances
(Paivio, 1990). Additionally, dual coding theory conceptualizes affective and
emotional reactions within the nonverbal system since affective responses, by
definition, are nonverbal (Sadoski, Goetz, & Rodriguez, 2000). Clark and Paivio
(1991) asserted that pictures elicit higher emotional reaction than words.
Video with its visual and verbal codes might be a more effective and powerful
medium for delivery of instructional material compared to a single representation of
just verbal code. According to the cognitive theory of multimedia developed by
Mayer and his colleagues (2001), information is easier to process in multimedia
environments with on-screen narration and images. Pictorial information and
narration allows for the parallel processing of information via visual and verbal inputs
(Moreno & Mayer, 1999).
Affective processing of media
Visual imagery, such as pictures and video generally elicit more affective
responses as compared to written words (Clark & Paivio, 1991; Machula, 1978). For
example, Koehler, et al. (2005) examined how equivalent text and video stories
influenced participants’ affective and emotional engagement. Results suggested that
participants in the video condition found it to be more engaging and interesting than
their peers in the text condition. Specifically, video showing emotional interactions
led to participants’ higher emotional engagement. The authors argued that one
benefit of video lie in the affective realm to capture one’s interest and engagement;
however, there is a complex interaction between media format and the content it
delivers. Specifically, Koehler and colleagues suggested that video is engaging for
only certain types of stories, such as human-interest narrative, but not for artistic or
informational presentations, such as a talking-head lecture. Machula also found that
media and content interact to influence one’s experience, with video producing
affective responses in some stories and not others.
Video with its nonverbal cues has the potential to emotionally engage
students. For example, Birdwhistell (1970) found that 55% of what one
communicates to another is conveyed via nonverbal communication, such as body
posture and movement, 30% using tone of voice, timber, tempo, and volume of
speech, and only 7% via words. Mayer, Heiser, and Lonn (2001) also found that
using video (with irrelevant information) to elicit emotional interest in an explanation
primed students to pay attention to supplemental information rather than structurally
Cognitive processing of media
A few notable attempts have been made to investigate media effects while
controlling for equivalent content. For example, in a study by Baggett (1979),
revisions to a text version of the dialogue-less movie The Red Balloon were made
until participants could match episodes in the film with passages from the story and
vice versa. The resulting structurally-equivalent forms of the story (the video and text
versions) were used in a different study of memory. Using a cued recall approach,
they found that immediately upon finishing the story, there was no effect for media –
both forms of the story (text and video) led to similar patterns of recall. However, an
analysis of delayed recall (seven days later) revealed much better performance for
participants who had viewed the video version of the story. Interpretation of these
findings is based upon the work of Kintsch and van Dijk (1975), who demonstrated
that stories told in words invoke schemas for processing, storing, and organizing
information to come later in the story. Baggett (1979) argued that a similar schema-
driven process exists for stories told in video, however, these schemas differ from
those used to process stories in text.
Presenting the same information in on-screen text and video may produce a
redundancy effect because the cognitive load placed on the visual channel by the text
and video have the potential to hinder processing of the information (Sweller &
Chandler, 1994). However, there is limited research that has utilized participant
verbal reports to elicit their cognitive processing and affective responses to media.
In the present study, we were interested in studying students’ level of
engagement and affective impact of the cases with the different media formats (text,
video, or video+text). We utilized verbal protocol analysis, which has led to important
insights into how struggling and sophisticated readers process texts, reasoning that
asking people to think aloud as they experienced different media might provide
insight into students’ cognitive processing strategies and affective responses
(Lundeberg, 1987; Pressley & Afflerbach, 1995).
Our research questions were: (1) Does the media or case affect students’
cognitive processing strategies? (2) Does the media or case affect students’ emotional
reactions? (3) Do either media or case affect students’ six-week recall? Emotional
responses may motivate prosocial, moral behaviors (Pizarro & Salovey, 2002), and if
video cases of people diagnosed with HIV/AIDS elicit more emotion than equivalent
text version, we would expect video cases to produce more engagement and delayed
recall. We did not know what the effects of giving participants both text and video
versions of the case simultaneously would produce.
Thirty undergraduate students who were all education majors at a large Mid-
western university took part in the study and were either paid for their time or
received extra credit in their course work. Participants included seven males and 23
females who were freshmen (n=17), sophomores (n=8), juniors (n=4), and one senior.
The majority of the participants were Caucasian (n=24), with three African
Americans, one Asian, one Hispanic, and one Caribbean African. The average age of
the participants was 19 years, ranging from 17 to 32 years of age.
We chose video that contained a number of personal narratives about
HIV/AIDS. The stories are a mix of narration by a focal person and an unseen
narrator. The cases were being developed for future use in biology courses, therefore
we attempted to find cases that were ecologically valid and not contrived. The
personal narratives were three individuals that had been diagnosed with HIV and were
willing to share their experiences. The narratives used in the study were as follows:
• Lisa is a Caucasian, heterosexual female from an affluent background. Lisa
was infected by her boyfriend and received immediate expert care and
medications that controlled the spread of HIV in her body. She later married
and had three HIV negative children by having unprotected sex when her viral
load was low. She has not reported serious medical problems related to HIV.
• Doug is a Caucasian, homosexual male from a middle class background who
grew in Southern California. He “came out” when he was 20 and moved to
San Francisco. While Doug was well aware of the risks of HIV, and knew
how to protect himself, he had unprotected sex and contracted HIV. He found
out he was HIV positive when he became seriously ill. Doug’s case focuses on
his poor decision-making and how that led to him contracting HIV. For
example, Doug says, “[having unprotected sex] was stupid, you know, I mean
I just can’t believe that I did that”. Doug closes his story by stating the
importance of disclosing if one is HIV positive and that he is educating young
people about HIV.
• Catrice is an African-American, heterosexual woman from a low socio-
economic background. She grew up in a “Bible Belt” in the south. The risks
of HIV were never discussed in her home or at school. She contracted HIV
when she was 17 from a popular, promiscuous boy at school. After being
diagnosed, Catrice reported going into a “deep denial” with her friends and
family. She went untreated for four years. During that time Catrice became
pregnant and she lost a lot of weight due to the virus. The case closes by
stating that three months after she gave birth, preliminary tests showed that
her daughter might have been exposed to the HIV virus.
Three different media versions – text, video, and video+text – were created for
each of the three cases (i.e., Lisa, Doug, and Catrice). Text versions of the cases were
created by transcribing the videos and making subtle changes to maintain the overall
essence of the story. Any verbal patterns (e.g., uh’s, like’s, etc.) that led to unclear
transcripts were removed and quotes from the focal person(s) in the videos were
included. All three media types were presented in two segments on webpages (word
length and readability statistics for the texts and time length for videos are
summarized in Table 1 and an example of a screenshot are provided in Figure 1).
------------------------ Insert Table 1 here ------------------------
------------------------ Insert Figure 1 here ------------------------
All three formats (text, video, and video+text) were segmented at the same point in
the case and participants had to click a button to continue watching/reading the
remaining narrative. For the text version of the cases, the text was presented in
paragraph form with a still picture of the individual from the personal narrative
juxtaposed to the left of the text. For the video+text version of the cases, the video
was juxtaposed to the left of the text narrative (i.e., the same text versions from the
text only format) and included audio as well. Both the text and video+text versions
were presented in a way that allowed participants to read the material at their own
pace and were segmented into two sections. For the video+text versions, participants
were given no direct instructions to use both media.
All of the participants experienced three cases. We counterbalanced each
combination of media (i.e., text, video, and video+text) and case (i.e., Lisa, Doug, and
Catrice) so that each media x case was presented in the first, second or third position,
participants were then randomly assigned to one of the 36 potential sequences (e.g.,
LisaVideo CatriceVideo+Text, DougText; CatriceText, DougVideo, LisaVideo+Text; and so forth).
This was to assure that each media and case condition was equally likely to be
presented first, second or third in the overall participant pool. Thus, each participant
experienced every case and every media, but in different combinations and in a
different order. The six sequences not used, based upon the random assignment of 30
participants to the 36 possible conditions, were: 1) DougT LisaV+T CatriceV, 2)
CatriceV DougV+T LisaT, 3) LisaT DougV CatriceV+T, 4) DougV+T CatriceT LisaV, 5)
DougT CatriceV LisaV+T, and 6) DougV+T CatriceV LisaT.
Participants completed a background survey designed to gather demographic
information, specifically, participants’ age, gender, ethnicity, and year in school.
After participants completed the background survey, the experimenter introduced the
media using a training video (i.e., a different HIV/AIDS story not used in the study)
for the participants to familiarize themselves with how to stop the video to discuss
what they were thinking or feeling while viewing the case. A training video was used
to ensure participants were comfortable and knew how to stop the video in order to
discuss their thoughts. We also trained students to think aloud when reading the text;
specifically, we modeled how students could stop reading to share their thoughts with
the researcher (students were not forced to do this and the discussion was driven by
Participants were then instructed to begin the first of the three cases and to
think aloud as they experienced it. All think-aloud data were audiotaped and
transcribed. With the narratives being segmented into two parts, the interviewer asked
participants what they were thinking and feeling about the narrative and prompted for
further clarification as needed between the segments and at the end of each case, even
if the participant had stopped during the case or not. Participants experienced all
three narratives using the same routine. These think-alouds were used to assess
cognitive strategies used by participants to process the cases. Think aloud protocol
has been used previously to assess cognitive strategies (Lewalter, 2003). The
researchers were trained in the think-aloud procedure regarding how to prompt
participants to begin discussion, clarify their responses, and elaborate on their
thoughts utilizing similar practice materials.
After each case, students also completed an affective/engagement
questionnaire. The questionnaire consisted of 23 items using a semantic differential
(Osgood, Suci, & Tannenbaum, 1957) with opposing affective adjective pairs.
------------------------ Insert Table 2 here ------------------------
Fourteen of the items were in response to the prompt, “The stories made me
feel …” (e.g., excited/bored, emotional/unemotional, informed/uninformed,
interested/uninterested, etc.) and the other nine were in response to “The cases were
…” (e.g., interesting/not interesting, realistic/unrealistic, sympathetic/unsympathetic,
informative/not informative). The affective/engagement survey was adapted from
Koehler, et al. (2005) and was used as the basis to explore the macro-level questions
of media and story interactions since this instrument was common to all stories.
Participants were interviewed again, by the same researcher, after 6 weeks to
assess their recall of the three narratives experienced during the first session.
Participants were asked what they could recall about the three narratives; specifically
asking them what they remembered about each story and what stood out for them.
Participants were also asked to recall how the person became infected. We utilized
this free recall rather than a “standardized” recall, as it has been found that free recall
allows participants to recall information as they have encoded it and not restrict their
recall by imposing an external structure (Mandler, 1967; Reffel, 1997).
We analyzed participants’ think-aloud protocols when they experienced the
cases, their six-week recall of the cases, and responses to the affective/engagement
survey given after every case. Data from all 30 participants were used for the
affective/engagement survey and the six-week recall. Due to the amount of data
generated by each individual during the think-aloud protocols, 15 participants were
randomly selected for the protocol analysis from the 30 participants using a random
number generator. The transcripts were not selected based on content or
representativeness for any particular theme or participants’ demographic information.
Of the 15 students whose verbal protocols we analyzed, eight were female
Caucasians, three were male Caucasians, three were African American females and
one Caribbean-African male.
Think-aloud analysis. For the think-aloud protocols, we used Grounded
Theory (Strauss & Corbin, 1990) to develop a coding scheme. First we reviewed the
15 individual transcripts and identified appropriate categories. Upon completion of
the first attempt for creating a coding scheme, we examined the transcripts a second
time to see if any new categories could be identified. The first round of developing a
coding scheme resulted in a very fine-grained analysis with two major categories,
emotion and cognition, with 20 subcategories emerging for these themes. For the
finalized coding scheme we collapsed many of the subcategories.
The finalized coding scheme resulted in the following six overarching
categories: 1) Cognitive Processing; 2) Cognitive Disconnect; 3) Positive
Evaluations; 4) Judgments; 5) Prior Knowledge; and 6) Emotions. See Table 3 for
examples and descriptive statistics regarding the categories of comments.
------------------------ Insert Table 3 here ------------------------
With the final coding scheme, two researchers coded the 15 transcripts. The inter-
rater reliability between two researchers was 79% agreement for all 15 protocols.
When discrepancies in coding occurred, consensus was reached for the particular
comment in question by consulting with a third researcher.
The six categories (Cognitive Processing, Cognitive Disconnect, Positive
Evaluation, Judgments, Prior Knowledge and Emotion) were analyzed for differences
using a multivariate analysis of variance (MANOVA) with two factors – media (text,
video, video + text) and case type (Lisa, Doug and Catrice). Field (2009)
recommended that the traditional approach is to complete a follow-up univariate
analysis for each dependent variable of a MANOVA. Hence, we conducted a follow-
up univariate analysis (See Table 8).
Recall analysis. The six-week recall interview responses were transcribed, and
analyzed for the facts participants could recall about each case. Two researchers
independently coded each transcript for the absence or presence of facts from the
cases. Inter-rater reliability was 79% agreement, and disagreements were resolved by
The themes were collapsed into four overarching categories: Description of
the individual (description of the individuals in the three cases); Family of the
individual (description of the family of the individual); Contraction (recalling how
the individual contracted HIV); and, Medication (recalling what the individual was
doing for medication). These four categories were used as dependent variables in the
final analysis using univariate analysis of variance (ANOVA) with two factors: media
(text, video, video+text), and case type (Lisa, Doug and Catrice). However, we
adjusted the probability level (!=0.05) to guard for inflated experiment-wise (type-I)
error rate. Since we used four ANOVAs, the Bonferroni adjustment resulted in the
alpha value of 0.0125.
Survey analysis. The affective/engagement survey was analyzed using a
principal components factor analysis with varimax rotation on ranked responses to the
23 items to identify patterns in the responses, and to reduce the number of dimensions
of comparison in the analysis (for purposes of increasing interpretability and
controlling overall experimental-wise error rate). To generate the factors, we used
Kaiser’s recommendation of Eigen value over one and also analyzed the scree plot,
which suggested the same number of factors – seven. In order to interpret specific
factors, we only assigned items with factor loadings above |0.40| and if an item had
loadings on two factors greater than |0.40|, it was assigned to the factor with highest
loading. The resulting factor scores were analyzed in a blocking design (using the
participants repeated measures as blocks) in a three-way ANOVA with media (text,
video, video + text), case (Catrice, Doug, Lisa), and participants as factors. Since we
used seven ANOVAs, we adjusted the probability level for inflated experiment-wise
(type-I) error rate, which resulted in the alpha value of 0.007.
The principal component analysis with varimax rotation produced seven
factors. The Kaiser-Meyer-Olkin measure suggested that the sample was adequate,
KMO=0.74, which is considered good (Field, 2009). Bartlett’s test of sphericity,
2(253) = 807.54, p<.0001, indicated that correlations between items were sufficiently
large for principal component analysis. These seven factors were produced, and can
be easily interpreted based upon the adjective pairs most heavily loaded for each
factor, as described in the Table 4 below.
------------------------ Insert Table 4 here ------------------------
Table 5 shows descriptive statistics and Table 6 shows ANOVA statistics for the
------------------------ Insert Table 5 here ------------------------
------------------------ Insert Table 6 here ------------------------
Using three-way ANOVAs (Media x Case x Participant), a significant main
effect was found for media on the engagement factor F(2,52)= 10.396, p<.000,
!p2=0.29. A pairwise comparison analysis revealed that the text condition differed
significantly in terms of engagement from the video (p=0.001) and video+text
conditions (p=0.001). Participants felt more engaged in the video and video+text
conditions than they did in the text only condition. A main effect was also found for
case for the positive affect factor F(2, 52)=6.702, p=0.003, !p2=0.21. A pairwise
comparison suggested that participants felt more positive affect towards Lisa
(p=0.002) and Doug’s (p=0.005) case as compared to Catrice’s case.
Overall, participants found that using video as a medium for presenting a case
was more engaging than presenting the case using text. Presenting the cases in a
video format seemed to make it easier for students to invest their attention and their
emotions in the cases being presented. However, participants felt that Lisa’s story
was more realistic compared to Doug’s or Catrice’s story.
Think-aloud Protocol Analysis
A multivariate analysis of variance (MANOVA) was conducted to investigate
potential differences between the independent variables (media and case type) on the
six dependent variables (Cognitive Processing, Cognitive Disconnect, Positive
Evaluation, Judgments, Prior Knowledge and Emotion), controlling for participants.
We hypothesized that participants’ media preference would have an influence on how
they processed the content. With participants reporting a preference for either the
video or video+text condition during the exit interviews during session one, we
expected our hypothesis to be supported by the think-aloud data. Even though the
overall omnibus MANOVA results were significant, the main effect for media type
using Hotelling’s trace was not, F(12, 58) = .905, p = 0.547, !p2 = .158 (See Table 7
for means and Table 8 for MANOVA statistics).
------------------ Insert Table 7 here ------------------
------------------ Insert Table 8 here ------------------
With all of the narratives from the same genre and about the same topic—
people’s stories about contracting HIV and their reaction to this disease – we did not
expect a difference to be found on the six dependent variables for case type.
However, the results for the overall MANOVA revealed a main effect [F(12, 58) =
4.074, p = .000, !p2 = .457] for case type. Results for the overall MANOVA also
revealed a significant main interaction effect between media and case [F(24, 114) =
2.55, p < .001, !p2 = .349].
On a follow-up investigation of the case type main effect, we found that two
factors were significant for case type—Positive Evaluation [F(2, 35) = 13.672, p <
0.001, !p2 = .439] and Judgments [F(2, 35) = 3.382, p < 0.05, !p2 = 0.162]. In
addition, a non-significant trend for Cognitive Processing was found [F (2, 35) =
3.074, p < .059, !p2 = .149]. Post hoc analyses were conducted for the two factors,
Positive Evaluation and Judgments, to evaluate where the differences occurred
between the three cases – Lisa, Doug, and Catrice.
The post hoc t-tests for Positive Evaluation by case type revealed significant
differences between all of the cases (Lisa > Doug > Catrice; Lisa > Doug, t(28) =
2.907, p = .007; Lisa > Catrice, t(28) = 4.490, p < .000; Doug > Catrice, t(28) =
3.434, p = .002). In other words, students made more positive evaluations when
experiencing Lisa’s case than they did with Doug’s case, who in turn experienced
more positive evaluations than did Catrice. Students seem to identify with Lisa’s case
and reported being happy that her treatment was working and that she experienced
successful outcomes. Typical Positive Evaluations for Lisa included, “it’s good that
she went on with her life”, and “I’m happy her kids are HIV negative”. Doug
represented more of a neutral case; however, several students made Positive
Evaluations regarding his decision to educate people about HIV infection. For
instance one student stated, “I think it is nice that he is helping people. It’s a pretty
noble cause to undertake”.
Perhaps understandably then, the results of the post-hoc t-tests for Judgment
by case type (No significant difference between Lisa and Doug; Lisa < Catrice, t(28)
= 2.334, p = .027; Doug < Catrice but not at a significant level, t(28) = 1.994, p =
.056) indicated that students were much more likely to evaluate Catrice negatively
over the other two characters, Lisa and Doug, due to her perceived poor decision-
making. This contrast can be seen in the following response, “It is just kinda stupid
how some people are just in denial about when they are pregnant or they are just
waiting for something to happen before they take care of themselves.”
However, with Catrice’s case we also observed a tendency for participants to
remark on the lack of education that high school students receive regarding proactive
behaviors against HIV. On average students made almost double the negative
judgments when experiencing Catrice’s case over the other two cases (1.53 comments
on average compared to 0.73 for Lisa and 0.87 for Doug). Whereas, many of the
judgments made during Lisa’s case had to do with the ability to purchase medication
due to inequality of socio-economic status among HIV patients. Thus, we found that
the individual cases mediated the effects of video. Even though the cases were all the
same genre, they affected students’ emotions differently.
Table 9 and 10 show detailed descriptive and inferential statistics for the recall
------------------------ Insert Table 9 here ------------------------
------------------------ Insert Table 10 here ------------------------
The ANOVA results suggested no significant media differences on the four
categories, Description of the individual, F(2,38) = 1.768, p = .184, !p2=0.085;
Family of the individual, F(2,18) = 2.661, p = .097, !p2= .228; Contraction, F(2,38) =
0.288, p = .561, !p2= .030; Medication, F(2,38) = 3.350, p = .046, !p2= .150. For case
type, results suggested that two of the four recall categories were significant: Family
of individual in the case, F(1, 18) = 39.806, p < .000, !p2= .689; and Medication,
F(2,38) = 5.507, p = .008, !p2= .255.
It was not a surprise that case was a significant factor for participants being
able to recall about Family of the individual in the case. Participants recalled the
most information about Lisa and her family (M=2.57, SD=1.72) as compared to
Catrice (M=0.48, SD=0.89). Lisa’s case was unique with regards to family as she met
her husband after she had contracted HIV and they got married. The case further
describes how they had three HIV negative kids in spite of Lisa being HIV positive.
The case also described that Lisa’s dad was a doctor and hence was able to provide
her with the best care possible. These aspects of the case about Lisa and her family
helped students recall more details about her family. This is highlighted by one
participant’s comments when recalling Lisa’s case, “And then she [Lisa] met another
guy, who, they wanted to get married and they wanted to have kids but, she was uh
concerned about her kids getting HIV. So anyways, they got married [and had] three
kids who didn’t get the virus [were not HIV positive].”
Similar to the findings for media type, case type was also a significant factor
for the Medication category. Results indicated that participants recalled more things
about Lisa’s medication/treatment (M=0.87, SD=1.18) as compared to Doug (M=0.09,
SD=0.28) and Catrice (M=0.52, SD=0.84). This was again not surprising as Lisa’s
case depicted her as being from a wealthy family and her dad as a doctor; hence, she
able to afford the best medication available. Another hypothesis for why participants
recalled more about Lisa’s medication is that even though she was HIV positive, she
had three kids who were not. In addition, her husband remained HIV negative even
after having unprotected sex with Lisa in order to conceive their three children. This
is highlighted by one participant’s comments, “the second one [Lisa] I think she was a
married, she was married woman and had kids and her kids didn’t have HIV, her dad
was a doctor… she was like better off, she was pretty wealthy and her dad got her lots
of good treatment and everything, and she was really able to get it under control”.
Similarly another participant said, “So anyways, they got married [and had] three kids
who didn’t get the virus. I just remember her…[talking about] taking medicine to
control the virus to increase her T-cells in fighting the infection.” Thus, students
recalled different kinds of information from the three cases and these differences
appeared connected to the differences in emotional reactions produced by these cases.
Video contexts strengthened the information recalled, depending on the context of the
Results from this study suggest that video is more powerful than text in
engaging students; both mediums have similar cognitive affects in terms of cognitive
processing and recall; finally, the interaction between medium and the content it
delivers is important. Participants in this study were more engaged when
experiencing the cases in the video or video+text condition as compared to the text
condition. Participants also felt more sympathetic and thought the video cases were
more realistic than the text cases. We also found that media did not significantly
influence participants’ cognitive processing and cognitive disconnect or recall of the
description of individual and family in the cases.
Video More Powerful Than Text
Results from this study suggest that video (or video+text) cases were more
powerful than the text cases in the affective realm. Specifically, the engagement
questionnaire suggested that participants felt video was more engaging than text, and
video also elicited more emotional feelings towards the focal person in the cases
presented. The data also suggested that participants found video to be a more
powerful medium as it “brought to life” the cases and made them more “realistic” as
compared to the text cases, which were “like reading a newspaper”. This finding has
important implications for influence on students’ lives and promoting prosocial
behaviors. Previous research by Cody and Lee (1990) has suggested that when
students watch an emotional video, they are less likely to engage in risky behaviors.
They examined the influence of viewing an emotional video about skin cancer on
students’ beliefs, knowledge, and skin protection behavior as compared to
informational video and a control video. Data was collected in a pretest-posttest
format; a second posttest was also conducted 10 weeks later using questionnaires that
assessed students’ health beliefs and behaviors. Cody and Lee found that students
who watched the emotional video were motivated to use higher skin protection
behaviors as compared to the control video. Even though we did not directly examine
students’ subsequent intentions about protection from HIV/AIDS, our finding in
conjunction with Cody and Lee’s findings that an emotional video can captivate
students’ attention is important given the nature of the disease and the worldwide
attention it receives. Emotions, too, have an impact on conceptual change in science,
and we are just beginning to appreciate the role of “hot cognition” in learning as
Sinatra and colleagues have illustrated (Sinatra, et al., 2009).
Video and Text Have Similar Cognitive Affects
The mediums, however, did not differ in terms of their influence on cognitive
factors. Specifically, the verbal protocol analysis exhibited no difference on
participants’ cognitive processing and cognitive dissonance. Also, the recall
differences between the mediums were not found to be significant. These recall
results were surprising as it was hypothesized that there would be differences between
the mediums for recall of information, since video – with its two separate symbolic
systems – provides little competition for resources (Paivio, 1990). A possible
conjecture for this finding might have been that the information presented via both
text and video was concrete, as opposed to abstract, and was easily encoded twice,
“once in terms of verbal attributes and once in terms of imaginal attributes” (Ashcraft,
1989, p. 219). For example, the category description of individual consisted of
themes like Caucasian, African American, Woman, etc., which could be “recorded
twice in memory, once as a word and once as a visual image, there were two different
ways it could have been retrieved from memory, one way for each code” (Ashcraft, p.
219). Hence, we did not find significant difference between video and text for recall
of concrete information presented in the cases.
One of the crucial findings of this study is that differences between stories did
occur and there was significant interaction effect between media and case for the
verbal protocol analysis. Specifically, the story significantly influenced participants’
positive evaluation and judgment of the individuals portrayed in the story. These
results were not particularly surprising as each case presented a unique story about the
focal person and showcased different situations depending on the individual. For
example, participants made more positive evaluation statements for Lisa, while
making negative judgment statements for Catrice. These two cases were quite
different as Lisa’s case showcased her as being “responsible” where she took
medication when she found out she was HIV positive. Lisa met her husband and told
him about her illness and they had three children who were all HIV negative. Catrice,
on the other hand, remained in denial for four years after she found out she was HIV
positive, got pregnant, and her child might have been exposed to HIV. Hence, it was
not a surprise that case was a significant factor in participants’ positive evaluation and
judgment of the individuals portrayed in these cases.
Case was also a significant factor for recall of individual’s family and
medication they took. In recalling about the family of the individual in the case,
participants recalled most about Lisa’s family; specifically, recalling how Lisa was
married and had kids, who were all HIV negative. Participants were surprised at this,
as one participant reported, “the thing I remember about that is that they were um,
they were doing unprotected sex and the dad didn’t get infected. He was negative,
and then he didn’t get infected, which was shocking to me”. It is possible that this
information presented in the case stood out for the participants and they were more
likely to recall it as it was something they did not know about previously. Finally,
participants’ recall regarding individuals receiving medication/treatment was greatest
for Lisa. This could be because participants were able to recall that Lisa’s dad was a
doctor and she received the best care, which allowed her to get pregnant without
having her husband and kids being infected with HIV. Furthermore, the majority of
the participants were able to recall that taking medication helped Lisa increase her T-
cell count to fight the infection. These two facts could have potentially supported or
challenged participants’ already known information (or misinformation), they were
able to recall it more easily.
Our purpose in this study was to examine both cognitive and affective
differences between text, video, and video+text by using verbal protocol analysis,
interviews, and survey. There are three main findings from this study as discussed
above: video is more powerful than text in the affective/engagement realm; video and
text have similar cognitive effects as indicated by the cognitive processing, cognitive
dissonance, and recall of information; finally, content matters as stories significantly
influenced participants’ evaluation, judgment, and recall. Results also suggest that
the interaction between medium and story was more complex and it was not simply a
matter of whether video is better than text. Our findings support recent research on
differences between video and text (See Koehler, Yadav, Phillips & Cavazos-Kottke,
2005). Particularly, we support Koehler and colleagues view that even though video
as a medium does not influence student achievement, it influences viewers in terms of
engagement and affective change “in ways that reading a text does not” (p. 269).
The lack of relevance is one of the greatest barriers for learning and is the
reason why some students are not interested and lack motivation in learning
(Aikenhead, 2006; Brophy, 1999, 2004). Interest plays an important role in learning
as Dewey (1916) stated citing an American humorist, “It makes no difference what
you teach a boy so long as he doesn’t like it” (p. 134). Piaget (1981, as citied in Hidi,
1990) also argued that both affective and cognitive components are equally important
for intellectual functioning. Hence, one main benefit of video lies in engaging
students in the content and making it interesting. However, more research needs to be
conducted to examine the impact of video in a classroom setting. A second benefit of
video may occur in instructional contexts when students are encouraged to make
judgments. In this study, participants recalled more under both, negative judgments
(i.e., with Catrice’s case) and positive judgments (i.e., with Lisa’s case). Given that
HIV/AIDS has become a global problem, it is imperative to increase students’
awareness and literacy of this infectious disease. HIV infections have been increasing
dramatically worldwide; HIV infections have increased by 3.5 million between 2001
and 2007, and in the U.S. it has risen by 200,000 over the same period of time, with
the bulk of the increase in the 15-49 age groups (UNAIDS, 2008). Media, such as
presenting emotional and informational video has the potential to increase awareness
of HIV/AIDS and promote positive behaviors among the target age groups 15-49,
which has seen the most dramatic increase of this disease.
Limitations and Future Research
One limitation of this study is that it was not conducted in a classroom
context, and thus we were unable to examine the effects of media combined with
instructional method. Another limitation of this study was the overrepresentation of
Caucasian females, however, this demographic is representative of the population we
sampled from—education majors (Jennings, 2007). Because this was conducted as a
lab study with a large number of Caucasian females, our results are limited in terms of
The current research did not examine how participants’ background
influenced their affective response to the cases as well as the medium. Future research
needs to investigate how participants’ background, such as preference for medium,
experience with content of the story itself, etc. might impact their emotional and
cognitive responses. Additionally, future research also needs to investigate whether
order-of-media changes participants’ engagement with the content and the medium.
Because the cases were being developed for future use in biology courses, we
attempted to find cases that were not contrived and had a feeling of authenticity.
However, in this process there were many details across cases that were not
consistent. Future research should explore whether or not there are differences
between media when content is held consistent across cases.
Aikenhead, G. S. (2006). Science education for everyday life: Evidence-based
practice. New York: Teachers College Press.
Ashcraft, M. (1989). Human Memory and Cognition. Glenview, IL: Scott, Foresman
Baggett, P. (1979). Structurally equivalent stories in movie and text and the effect of
the medium on recall. Journal of Verbal Learning and Verbal Behavior, 18,
Birdwhistell, R.L. (1970). Kinesics and context: Essays on body motion
communication. Philadelphia: University of Pennsylvania Press.
Brophy, J. (1999). Toward a model of the value aspects of motivation in education:
Developing appreciation for particular learning domains and activities.
Educational Psychologist, 34, 75-85.
Brophy, J. (2004). Motivating students to learn (2nd ed.). Boston, MA: McGraw Hill.
Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Educational
Psychology Review, 3, 149-210.
Clark, R. E. (1994). Media will never influence learning. Educational Technology
Research and Development, 42(2), 21-29.
Clark, R. E. (1983). Reconsidering Research on Learning from Media. Review of
Educational Research, 53(4), 445-459.
Cody, R., & Lee, C. (1990). Behaviors, beliefs, and intentions in skin cancer
prevention. Journal of Behavioral Medicine, 13(4), 373-389.
Coomey, S.M., & Wilczenski, F.L. (2005). Implications of technology for social and
emotional communication. Journal of Applied School Psychology, 21(1), 127-
Dewey, J. (1916). Democracy and education: An introduction to the philosophy of
education. New York: MacMillan.
Field, A. (2009). Discovering statistics using SPSS. Thousand Oaks, CA: Sage
Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review
of Educational Research, 60(4), 549-572.
Jennings, T. (2007). Addressing diversity in US teacher preparation programs: A
survey of elementary and secondary programs’ priorities and challenges from
across the United States of America. Teaching and Teacher Education, 23,
Koehler, M.J., Yadav, A., Phillips, M. M., Cavazos-Kottke , S.C. (2005). What is
video good for? Examining how media and story genre interact. Journal of
Educational Multimedia and Hypermedia, 14(3), 249-272.
Kozma, R. B. (1994). Will media influence learning? Reframing the debate.
Educational Technology Research & Development, 42(2), 7-19.
Kintsch, W., & van Dijk, T. (1975). Recalling and summarizing stories. Language,
Lewalter, D. (2003) Cognitive strategies for learning from static and dynamic visuals.
Learning and Instruction, 13(2), 177-189.
Lundeberg, M.A. (1987). Metacognitive aspects of reading comprehension: Studying
understanding in legal case analysis. Reading Research Quarterly, 22(4), 407-
Machula, R. S. (1978). Media and Affect: A Comparison of Video-Tape, Audio-Tape,
and Print. Journal of Educational Technology Systems, 7(2), 167-185.
Mandler, G. (1967). Organization and memory. In K. W. Spence & J. T. Spence
(Eds.), The psychology of learning and motivation: Advances in research and
theory (pp. 327-372). New York: Academic Press.
Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia
learning: When presenting more material results in less understanding.
Journal of Educational Psychology, 93(1), 187-198.
Mayer, R. E. & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia
learning. Educational Psychologist, 38, 43-52.
Mayer, R. E., & Wittrock, M. C. (2006). Problem solving. In P. A. Alexander & P. H.
Winne (Eds.), Handbook of educational psychology (pp. 287–304). Mahwah,
Mayer, R.E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., Bulger,
M., Campbell, J, Knight, A, & Zhang, H (2009). Clickers in college
classrooms: Fostering learning with questioning methods in large lecture
classes. Contemporary Educational Psychology, 34, 51–57.
Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The
role of modality and contiguity. Journal of Educational Psychology, 91(2),
Paivio, A. (1990). Mental Representations: A Dual Coding Approach. Oxford
Piaget, J. (1981). Intelligence and affectivity: Their relationship during child
development. Annual Reviews Monograph, Palo Alto, CA: Annual Reviews.
Pizarro, D.A., & Salovey, P. (2002). On being and becoming a good person: The role
of emotional intelligence in moral development and behavior. In J. Aronson
(Ed.), Improving academic achievement: Impact of psychological factors on
education (pp. 247-266). San Diego: Academic Press.
Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: The nature of
constructively responsive reading. Hillsdale NJ: Erlbaum.
Osgood, C.E., Suci, G., & Tannenbaum, P. (1957). The measurement of meaning.
Urbana, IL: University of Illinois Press.
Reffel, J. (1997). Cued vs. free recall in long-term memory of the fifty United States.
Current Psychology, 16(3), 308-315. doi: 10.1007/s12144-997-1004-1
Sadoski, M., Goetz, E. T., & Rodriguez, M. (2000). Engaging texts: Effects of
concreteness on comprehensibility, interest, and recall in four text types.
Journal of Educational Psychology, 92, 85-95.
Seymour, E., & Hewitt, N. M. (1997). Talking About Leaving: Why Undergraduates
Leave the Sciences. Boulder, CO: Westview Press.
Sinatra, G.M., Linnenbrink-Garcia, L, Pugh, K.J., Hofer, B. DeLisi, A., Lan, C.F,
Broughton, S.H., Muis, K.R., Franco, G., Kendeou, P.,& Behrens, S.F. (April,
2009). Controversial conceptual change: The role of emotions, interest and
epistemic beliefs. Symposium presented at the American Educational
Research Conference, San Diego, CA.
Strauss, A. & Corbin, J. (1990). Basics of qualitative research: Grounded theory
procedures and techniques. Thousand Oaks, CA: Sage Publications.
Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition
and Instruction, 12(3), 185-233.
UNAIDS: Joint United Nations Programme on HIV/AIDS (2008). Report on the
global AIDS epidemic. Retrieved April 22, 2009 from the UNAIDS Web site:
Table 1: Length of video segments, number words from text, and Flesch Reading Ease
Length for Video
Number of Words
Lisa: Part 1
Lisa: Part 2
Doug: Part 1
Doug: Part 2
Catrice: Part 1
Catrice: Part 2
Table 2: Adjective pairs and factor loadings
Troubled - Untroubled
Emotional - Unemotional
Uncomfortable - Comfortable
Not confused - Confused
Sad - Happy
Optimistic - Pessimistic
Frustrated - Not frustrated
Passive - Active
Disengaged - Engaged
Bored - Interested
Interesting - Not interesting
Unsympathetic - Sympathetic
Convinced - Unconvinced
Sympathetic - Unsympathetic
Not informative - Informative
Unemotional - Emotional
Thought Provoking - Not
Relevant - Irrelavant
Informed - Uninformed
Knowledgeable - Not
Realistic - Unrealistic
Clear - Confusing
Biased - Unbiased
Table 3: Examples and Descriptive Statistics for Think Aloud Categories
M, SD &
• Prediction: “When it said that this would be the last place that
you would expect to find it, I knew she was probably going to
have AIDS and it was probably from the guy she had slept
• Summary; “He knew the other partner and trusted him and
basically he got infected.”
• Inference: He made his decision and I guess he realizes that
now, and now he is just trying to make the best out of his
• Monitoring: “I don’t know if they were married or just a
• Characters’ actions & thoughts: “They talked about condoms
but just didn’t use them, that was really stupid.”
• Society & Education: “She should have been informed about
it, like in school or something.”
• “That’s awesome that he loves her so much that he can get
past that part that she was infected.”
• “Well for her she is lucky because it is covered by health
• Disagreement: “He says, ‘especially the gay community,’
how they should be aware of it. It’s not just the gay
community though. Everyone should be aware really, not just
the gay people.”
• Questioning: “When is back then in this story?”
• Cognitive Dissonance: “It just struck me as odd to see, when I
read, that women, like Lisa, were not suppose to become
infected by the AIDS virus.”
• Text-to-World: “We can relate his story to someone, who is
like Magic Johnson, who is ten years ago, who came forward
and said that he was infected.”
• Text-to-Self: “But my aunt who died, she had some type of
infectious disease, like a viral that can only be transmitted
through the blood.”
• Personal Beliefs: “You can’t count on everybody’s parents to
teach their kids about sex and everything.”
• Double Coded with Emotion: “It is just kinda stupid how
some people are just in denial about when they are pregnant
or they are just waiting for something to happen before they
take care of themselves.”
• “I feel bad for her.”
• “That’s sad, that he wouldn’t protect himself.”
Table 4: Results of Factor Analysis
% of Variance
Adjectives most associated with this factor
I felt: Untroubled, Comfortable, Happy,
I felt: Active, Engaged, Interested
The stories were: Interesting
I felt: Sympathetic, Optimistic, Convinced
The stories were: Sympathetic
The stories were: Informative, Thought
I felt: Informed, Knowledgeable
The stories were: Clear, Realistic
The stories were: Biased
Table 5: Descriptive Statistics for the Survey Factors
Table 6: ANOVA Statistics for the Survey Factors
Media x Case
Media x Case
Media x Case
Media x Case
Media x Case
Media x Case
Media x Case
* Indicates significant values
Table 7: Descriptive Statistics for the Verbal Protocol Categories
Table 8: MANOVA Results from Verbal Protocol Analysis
* Indicates significant values
Table 9: Mean for all the four recall categories
* Indicates significant values
a Participants did not recall any items for Doug’s family
Table 10: ANOVA results for the four recall categories
Description about the
Family of the individual
* indicates significant values at p<0.00125
Figure 1: Screenshot Examples