Content uploaded by Hyun Gu Kang
Author content
All content in this area was uploaded by Hyun Gu Kang on Jul 24, 2014
Content may be subject to copyright.
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
Learning Outcomes in a Stress Management Course:
Online versus Face-to-Face
Kristine Fish
Associate Professor
Department of Kinesiology and Health Promotion
California State Polytechnic University, Pomona
Pomona, CA 91768 USA
kfish@csupomona.edu
Hyun Gu Kang
Assistant Professor
Department of Kinesiology
California State University San Marcos
San Marcos, CA 92096 USA
hkang@csusm.edu
Abstract
The purpose of this study was to compare learning outcomes in a stress management
course delivered in an online environment with those in the traditional, face-to-face
(F2F) classroom. Learning outcomes assessed were exam scores, perceptions relating
to awareness of and ability to handle stress, and self-reported decreases in heart rates
following five relaxation exercises. Impact of age and ethnicity on learning outcomes
was also examined. Online students (n = 56) listened to audio recordings of relaxation
techniques, while F2F students (n = 63) received the same material via on-campus
classroom delivery. Differences in exam scores for two out of three exams were not
statistically significant. F2F students felt more aware of stress compared to online
students, but there were no significant differences in perceived ability to manage stress.
Age and ethnicity were not significant predictors of the preceding factors. No statistically
significant differences were found in heart rate drops following relaxation techniques
with the exception of autogenic training, which resulted in greater heart rate drops in
online students. For this group of students, taking a stress management course online
appeared to be just as effective, and possibly even more effective with learning
relaxation techniques, when compared to a classroom-based approach.
Keywords: stress management education, relaxation techniques, classroom instruction,
distance education, audio recordings, comparative study, learning outcomes
Introduction
Online learning has grown tremendously over the last couple of decades (Allen & Seaman, 2007; D'Orsie
& Day, 2006). Internet-based course offerings have been on the rise among colleges and universities
(Parker, Lenhart, & Moore, 2011). Flexibility of location that online courses afford to busy students
appears to be the predominate reason for the large demand of these courses (Furst-Bowe & Dittmann,
2001; Kramarae, 2001). Potentially, a more compelling justification for administrators attempting to meet
student demand by increasing online offerings is that learning outcomes in online classes in several
academic disciplines have been shown to be just as good as in traditional, face-to-face (F2F) classes
(Zhao, Lei, Yan, Lai, & Tan, 2005). For example, online students achieved learning objectives at the
same levels as students studying F2F in diverse disciplines, such as a medical terminology class
(Somenarain, Akkaraju, & Gharbaran, 2010); a graduate level instructional design course (Johnson,
Aragon, Shaik, and Palma-Rivas, 2000); an undergraduate Spanish course (Salcedo, 2010); a graduate
course on behavior management in a special education teacher credential program (Caywood & Duckett,
2003); or organization behavior, personal finance, managerial accounting, sociological foundations of
education, and environmental studies courses (Schuman & Sims, 1999).
179
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
Learning outcomes are usually assessed by performance on traditional cognitive tasks such as quizzes
and projects, which may be completely appropriate given the learning objectives of the aforementioned
disciplines. The desired learning outcomes and learning objectives of a stress management course,
however, generally do not fit neatly into the cognitive domain of learning. In other words, learning
outcomes not only include cognitive learning objectives but physiological and behavioral outcomes.
Incorporating effective coping techniques (e.g., communication skills, creative problem solving, cognitive
restructuring) and achieving a relaxed physiological state by performing relaxation techniques (e.g., deep
breathing, mental imagery, meditation) are two significant learning objectives of a stress management
course. The authors of the current study propose that the effectiveness of online learning may depend on
traditional cognitively oriented learning objectives.
For several decades, three types of learning have generally been recognized: cognitive (knowledge),
affective (emotions), and psychomotor (physical skills), and are often referred to as Bloom's taxonomy
(Bloom, 1956). The primary learning objectives in a stress management course typically require changes
in: (1) the affective domain of learning (meaning values, attitudes, perceptions about stressors) rather
than the traditional cognitive domain of learning; and (2) the psychosomatic domain of learning (meaning
ability to use thoughts to affect physiology), a domain that is not recognized in Bloom's taxonomy. Without
exception, the primary learning outcome assessed in all of the aforementioned courses fell within the
cognitive domain. This points to the need for investigation into the effectiveness of online learning relative
to traditional, F2F classroom instruction, with varied learning objectives, specifically learning beyond mere
knowledge acquisition or beyond the cognitive domain of learning.
Despite myriad studies documenting the absence of significant differences in learning outcomes between
online learning and F2F delivery, research among a variety of student populations and across a variety of
disciplines would result in a more educated and informed discussion as to which contexts are most
appropriately served by Internet-based education (Frost & Fukami, 1997). Although online course
offerings continue to increase in higher education, and learning outcomes in online classes appear to be
just as good as those in the F2F classroom in many cases, it is still unclear if online learning produces
similar learning outcomes as those produced with traditional classroom instruction in all disciplines. It may
be possible that some courses are more suitable for online learning and others more suitable for
traditional classroom delivery based on the type of learning objectives.
Literature Review
In order to shed light on the continuing online-versus-F2F dilemma, Sitzmann, Kraiger, Stewart, and
Wisher (2006) completed a meta-analysis of 96 research reports that included over 19,000 participants. A
total of 168 courses were examined and included a wide range of subjects such as psychology,
engineering, computer programming, business, and technical writing. Across all studies, online learning
was six percent more effective than F2F classroom delivery for teaching declarative knowledge, but there
was no evidence that it was more effective for teaching procedural knowledge. It should be noted that
online learning and F2F delivery were equally effective for teaching declarative knowledge when similar
instructional methods were used. Online learning was found to be 11% more effective than classroom-
based instruction for teaching declarative knowledge when different instructional methods were used to
deliver the two courses. Thus, unique instructional methods may be responsible for differences in the
effectiveness of online learning relative to F2F classroom instruction. However, the authors caution that
institutions should be careful when considering implementing online learning because its relative
effectiveness may depend on both the intended learning outcomes and the learning conditions.
Again, an important difference between stress management courses and courses in other disciplines is
the emphasis on the affective domain of learning, or the emotional aspects of learning. Sitzmann et al.
(2006) were able to identify only 12 studies, out of 96, that assessed procedural knowledge and even
fewer that assessed affective learning. There were so few studies focused on affective learning that the
researchers could not determine an overall effect size and were left with inconclusive results regarding
the effectiveness of online learning for primarily affective learning outcomes. Thus, it is unclear based on
these results if online learning is more or less effective compared to F2F delivery in courses with affective
learning objectives, specifically for courses pertaining to stress management because of the inherent
emphasis on affective outcomes within such courses.
Another large-scale meta-analysis of distance education was completed using 232 research studies that
included just over 57,000 students with achievement outcomes, over 35,000 students with attitude
outcomes, and just under 58,000 students with retention outcomes (Bernard et al., 2004). One of the
180
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
overall conclusions suggested by the authors is that even though the literature is large, it is difficult to
draw firm conclusions as to what works and what does not work in regard to distance education, which
often includes online learning. However, the distinction between synchronous and asynchronous forms of
distance education was found to moderate effect size as a function of achievement and attitude. There
were instances in which the distance education group outperformed the traditional instruction group by
more than 50%, and there were cases in which the opposite occurred. Bernard et al. (2004) contend that
it is incorrect to say that distance education and F2F learning are equal without examining the sources of
variability in the statistical model. A significant factor that may contribute to this variability is learning
objectives. Thus, the question remains as to the influence that types of learning objectives, specifically
those that extend beyond knowledge acquisition to measurable physiological change, may have on the
effectiveness of online learning relative to F2F.
More research is clearly needed to determine the effectiveness of online learning in comparison to F2F
classroom instruction based on learning objectives, specifically objectives pertaining to the affective
domain of learning as well as psychomotor learning. Furthermore, the literature is extremely limited
regarding comparisons of online learning and F2F delivery within the context of stress management
courses in higher education. Chiauzzi, Brevard, Thurn, Decembrele, and Lord (2008) developed and
tested an online stress management intervention program. Students at six U.S. colleges were
randomized into one of three conditions: an online stress management program, a health information
website, or no intervention. Baseline differences between groups on stress control and health behavior
measures were compared at the outset of the research, and at one, three, and six months after baseline.
Although there were no between-group differences on primary outcome variables, secondary analyses
indicated that participants in the online stress management program were more likely to increase weekly
physical activity, use specific stress management methods, and exhibit decreased anxiety and family
problems. Although online learning may have been effective in this case, there were no comparisons
made with F2F delivery.
Carpenter, Stoner, Schmitz, McGregor, and Doorenbos (2014) developed an online cognitive behavioral
stress management intervention for early stage breast cancer survivors and evaluated its effectiveness.
Intervention and control group participants were assessed at baseline, at 10 weeks, after which only
intervention participants had used the workbook; and at 20 weeks, after which both groups had used the
workbook. Results indicate that at 10 weeks intervention participants showed improved self-efficacy for
coping with their cancer and for regulating negative mood and lower levels of cancer-related post-
traumatic symptoms as compared to the control group. Although results suggest that an Internet stress
management intervention program may be effective for helping patients increase confidence with ability to
cope with stress, there were no comparisons made with the F2F format once again. Furthermore, there
may be significant differences between cancer patients and college students. Thus, making
generalizations based on these results to stress management courses in higher education, and other
courses where learning objectives extend beyond knowledge acquisition, must clearly be done with
caution pending future research.
Fridrici and Lohaus (2009) evaluated an online stress-prevention program among high school students. A
total of 904 adolescents in Grades 8 and 9 were assigned to one of four intervention learning
environments: online training in school, online-training via Internet from home, school-based F2F training,
and a control group without intervention. Before and after the training interval, all adolescents were
questioned about their knowledge regarding stress and coping, their appraisal of stress-evoking
situations, their perceived stress vulnerability, symptoms of stress and coping, and training acceptance
(i.e., self-assessment as to whether or not they had learned something from the training program and
self-assessment as to whether or not they knew better how to cope with stress after the training, often
termed training acceptance).
Regarding knowledge about stress and coping, results indicate a significant increase of knowledge in all
intervention groups with the highest effect size in the online school group, followed by the school-based
F2F group and the online home group. Regarding appraisal of stress-evoking situations, results show that
positive thinking increased mainly in the traditional F2F training group and for participants working online
in school. This raises the question as to whether learning in a traditional, F2F classroom is more effective
than online learning in terms of altering attitudes and perceptions (i.e., the affective domain of learning).
Regarding perceived stress vulnerability and symptoms of stress and coping, results indicate a slight
decrease of vulnerability during the intervention period. However, this decrease was found not only in the
181
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
intervention groups but also in the control group. Regarding symptoms of stress, results indicate a
significant reduction of psychological stress symptoms in the F2F group and in the online-school
condition, but not in the online-home and the control group. Overall, the authors suggest that their results
indicate that a school-based F2F intervention seems to be the most accepted form of stress prevention
for adolescents. Nevertheless, results are conflicting based on the learning objectives: increased
knowledge (cognitive), improved appraisal of stressful situations (affective), perceived stress vulnerability
(affective), decreased symptoms of stress (physiological), and training acceptance (affective). Again,
additional research is needed to determine the effectiveness of online learning relative to F2F delivery in
regards to varied learning objectives.
Regarding ability to implement relaxation techniques following intervention, Ricks, Naquin, Vest, Hurtt,
and Cole (2011) examined students' responses to stress management techniques provided via podcasts
in health and stress management classes. Seven techniques were each practiced for 7 days by listening
to podcasts on an iPod or other MP3 player. Results indicated significant increases in perceived
relaxation in five of the seven techniques. Although this study did not involve students enrolled in an
online course, results indicated that students were able to effectively incorporate relaxation techniques
after listening to recordings of them, as noted by their reported decreased stress levels. However, there
were no comparisons made with delivering the relaxation techniques via F2F classroom instruction.
Problem Statement, Aims, and Research Questions
It appears that the most common learning outcomes assessed with online learning relate to academic
performance in some way (grades, scores, total points, etc.). However, in a stress management class,
students' psychosomatic ability to get into a relaxed state is a much more important learning outcome
than scores on traditional cognitive measures such as exams. Coping techniques and relaxation
techniques are equally important components to any stress management intervention effort. The primary
purpose of this study was to examine relaxation techniques as they affect heart rate. The ability to
perform mental skills in order to affect physiology, specifically the ability to achieve a relaxed state, is a
significant learning outcome in a stress management class and an extremely unique one when compared
to other course objectives.
Heart rate is an easily accessible and accurate physiological measurement that indicates a stressed state
or a relaxed state. The nature of a stress management course is very different from that of a more
traditional course such as math, history, or science where the content and learned skills are well defined
and generalized for all students. Students in a stress management course learn to alter their minds, which
thereby alters their physiology and allows them to get into a relaxed, meditative state. Practicing
relaxation techniques and other types of mental skills requires tapping into creativity and imagination that
is unique and personal to each student. Therefore, the instructor cannot prescribe the exact techniques
that are most effective for each student.
Clearly, additional research is needed to examine varied learning objectives with online learning in
relation to F2F classroom delivery, specifically learning objectives that pertain to affective (i.e., altering
attitudes and perceptions about stressors) and psychosomatic (i.e., using the mind to affect the body)
components as in a stress management course. The primary aim of the present study was to compare
the effectiveness of online learning relative to F2F delivery regarding students' ability to decrease their
heart rate (psychosomatic outcome). Secondary aims were to compare exam scores (cognitive outcome),
awareness of stress and perceived ability to handle stress (affective outcome) with online learning relative
to F2F delivery, and the influence of age and ethnicity on the preceding factors.
With these goals in mind, the following research questions were formulated:
1) What differences exist in heart rate following relaxation techniques between online students and
F2F students?
2) What differences exist in exam scores among online students and F2F students?
3) What differences exist in students' perceptions of awareness of stress among online students
and F2F students?
4) What differences exist in students' perceptions of their ability to handle stress among online
students and F2F students?
5) What differences exist in heart rate drop, exam scores, awareness of stress, and ability to handle
stress as a function of age and ethnicity of students?
182
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
Method
Context
This study compared self-reported outcome data obtained from students enrolled in one of two sections
of an undergraduate, upper-division stress management course taught at a large, public university on the
West Coast of the United States. The course satisfied a general education requirement and therefore
consisted of students from all majors across campus. One section was taught completely in a traditional
F2F format, whereas the other version of the same course was offered online with no direct F2F contact
between the instructor and the students at any time. Both courses were taught by the same instructor,
consisted of the same assignments, and were taught over a 10-week period on a quarter system.
Approval to conduct the research was obtained from the University's Institutional Review Board. Students
were informed that their responses were anonymous and participation in the study was voluntary.
Each week, students enrolled in the online section (n = 56) were required to view and listen to, at their
convenience, audio/visual recordings of lectures and audio-only recordings of relaxation techniques.
Instruction for performing the relaxation techniques was delivered solely via these audio recordings.
Students in the F2F section (n = 63) met twice a week for approximately 100 minutes each session. They
received the same lectures and instruction for performing the relaxation techniques as the online students
but in a F2F format. The sessions included lectures, group discussions, and instructions for performing
relaxation techniques.
The online and F2F course formats differed in the following ways:
1) Online students viewed/listened to recorded lectures, while F2F students listened to live lectures
that they attended in person;
2) Although both online and F2F students were required to respond to critical thinking questions as
part of the course requirements, F2F students discussed the questions in small groups prior to
writing and submitting their responses (This, however, does not relate to the learning outcomes
that were assessed);
3) Although the instructor attempted to deliver the lectures in the F2F classroom verbatim from the
recorded lectures, F2F students asked questions throughout the lecture, which resulted in very
minor modifications from the planned lecture as the instructor responded to these questions;
4) F2F students took their heart rate in the classroom, and online students were instructed to take
their heart rate at home;
5) Online students took the exams using the quiz tool in Blackboard in an unproctored environment
with various limitations imposed (e.g., timed, randomized questions), and F2F students took the
exam on a scantron in the classroom with the instructor serving as proctor;
6) F2F students were required to attend class and were deducted points for absences, whereas
online students were never required to attend class.
The similarities between the two course formats were as follows:
1) The recorded lectures provided to the online students and the live lectures delivered to the F2F
students consisted of the same content and PowerPoint slides;
2) Course requirements were the same for students enrolled in both the online and F2F sections
(e.g., weekly written discussions, weekly book reviews, weekly reactions to coping and relaxation
techniques, and weekly tracking of stressors);
3) The same instructions were given verbatim for performing the relaxation techniques;
4) The same exams were given to students in both sections.
Data Collection
Students in both sections were asked to take their pulse for a 6-second count immediately before and
immediately after engaging in five relaxation techniques that were delivered throughout the 10-week
quarter. The reason for using a 6-second count rather than a 10-second or even a 60-second count was
because of the sensitivity and variability of the heart rate. In other words, heart rate may begin to rise
immediately after ending a relaxation technique, hence a 6-second count may be more accurate than a
183
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
10-second count. Also, 6 seconds is often used for ease of translating into a 1-minute count by simply
adding a zero (or multiplying by 10).
The following five relaxation techniques were assessed:
• Energy Breathing. For this technique, both online and F2F students were required to close their
eyes and imagine being in a supine position (i.e., lying down) with a small flame, which
represented their level of relaxation, burning above their abdominal area. As they took deep
breaths and became more relaxed, they imagined the flame getting larger and larger as their
tension dissipated with each exhalation, and their body became more relaxed.
• Repeated Sounds. For this technique, students were required to choose a mantra, a short phrase
or word, and repeat it multiple times during the exhalation phase of diaphragmatic breathing (i.e.,
deep breathing).
• Mental Imagery. Students were required to close their eyes and listen to a vignette in which they
were taken to a beach. They focused on all five senses (e.g., visual details of their beach
including shapes and colors, wind, sound of waves crashing, sound of seagulls, taste of salt air
on their lips, feel of sand).
• Music Therapy. Students were required to close their eyes and engage in diaphragmatic
breathing while listening to slow-paced instrumental music.
• Autogenic Training. Students were required to close their eyes and focus on feelings of warmth
and heaviness in their feet, legs, arms, shoulders/neck, and head. The phrase "your _____ feel
very warm and heavy" was repeated three times for each body part.
Both online and F2F students were required to take three exams, each worth 50 points. Online students
took the exam on a computer using the quiz tool in Blackboard and were told it was "on their honor" to
refrain from referring to their notes. As an attempt to prevent cheating, the exams were timed (50 minutes
for 50 multiple-choice questions), questions were randomly delivered for each student so that Question 1
for one student may have been Question 20 for another, and the questions were delivered one at a time
where students could not skip ahead or go back after responding to a particular question. Students in the
F2F section took the exams using traditional scantrons during a timed session in the classroom, with the
instructor monitoring the students. F2F students were also required to complete the exam in 50 minutes.
At the end of the quarter, students in both the online and F2F groups were asked to rate their agreement
with each of the following statements on a scale of 1 (strongly agree) to 5 (strongly disagree):
1) After taking this class, I feel more aware of when I'm feeling stressed.
2) After taking this class, I feel better able to handle stress.
Responses were submitted online, via Blackboard.
Data Analysis
The drop in pulse rates (beats per minute) were calculated from pre and post pulse rates and compared
using a t-test between the instruction types. Exam scores were compared using the t-test. Likert scale
responses were compared using both the t-test and Mantel–Haenszel chi-squared test. The effects of
age and ethnicity on the drop in pulse rates were tested as factors of a full factorial analysis of variance
(ANOVA), but since these factors were not significant, the analyses were collapsed to the t-test. All
statistical tests were conducted with a significance level of α = .05.
Results
Participants
Out of a total of 119 participants, 56 were enrolled in the online section and 63 were enrolled in the F2F
section. Demographic comparisons of students can be seen in Table 1.
Research Question 1 questioned if differences exist in the learning outcome as measured by drop in
heart rate following relaxation techniques between online students and F2F students. In order to test
Question 1, a comparison in the differences in drop in heart rate following five relaxation techniques was
completed using a t-test of two means. Results indicate that there were no significant differences in
average drop in heart rate between online and F2F students for four of the relaxation techniques: Energy
Breathing, Repeated Sounds, Mental Imagery, and Music Therapy (see Table 2). Nevertheless,
184
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
significant average drop in heart rate differences between online and F2F students were found for
Autogenic Training, where online students reported greater drops (Table 2). Thus, the findings suggest a
partial affirmative response for Research Question 1. Some differences existed in heart rate following
relaxation techniques among online students and F2F students.
Table 1. Demographic comparisons of students
Group n
Age
Ethnicity
17-20
21-24
25-28
29+
Not Specified
Caucasian
Asian
Latino
Othera
Online
56
2
38
10
5
1
15
22
12
4
F2F
63
7
40
6
4
6
22
14
14
7
Note. F2F = face-to-face.
aIncludes African American, Middle Eastern, Polynesian, others, multi-ethnic identification.
Table 2. Average heart rate drop for each relaxation technique and for all techniques combined
Technique
Onlinea
F2Fb
p
M
SD
M
SD
Energy Breathing
9.5
17.4
6.0
18.5
.3785
Repeated Sounds
6.5
14.3
7.2
17.8
.8681
Mental Imagery
9.0
16.5
7.9
12.3
.7225
Music Therapy
13.5
16.0
8.6
22.0
.2599
Autogenic Training
13.8
16.5
6.3
13.1
.0219
All Techniques (Combined)
7.2
8.7
6.8
10.9
.8267
Note. F2F = face-to-face.
an = 56. bn = 63.
Research Question 2 explored the relationship between exam scores of online and F2F students. In order
to test Question 2, a comparison in the differences in three exam scores was completed using a t-test.
The results indicate that there were no significant differences in exam scores when all three exams were
examined together (see Table 3). When differences among individual exam scores were compared, there
were no significant differences among scores for Exams 1 and 2, but there was a significant difference
among scores for Exam 3 (Table 3). Thus, the findings suggest a partial affirmative response for
Research Question 2. Some differences existed in exam scores among online students and F2F
students.
Table 3. Average scores for each exam and for all exams combined
Exam
Onlinea
F2Fb
p
M
SD
M
SD
Exam 1
41.1
6.0
41.5
5.5
.7423
Exam 2
42.5
4.0
42.1
4.7
.6301
Exam 3
41.1
4.6
39.4
4.4
.0364*
All Exams (Combined)
41.5
4.0
41.0
4.0
.5011
Note. F2F = face-to-face. Exams were scored out of 50 total possible points.
an = 55. bn = 62.
*p < .05.
Research Question 3 examined students' awareness of when they are feeling stressed. In order to test
Question 3, a comparison of the differences in perceived awareness was completed using a t-test of the
mean rating as well as using Mantel–Haenszel chi-squared test for categorical data. The results indicated
that F2F students felt more aware of stress compared to online students (p < .01) (refer to Table 4). Thus,
the findings suggest an affirmative response for Research Question 3. Differences existed in the learning
outcome as measured by awareness of stress among online students and F2F students, where the F2F
students reported better awareness.
185
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
Table 4. Awareness of stress
Group n M SD
Frequencies of Each Ratinga
1
2
3
4
5
Online
54
1.83
0.82
21
23
8
2
0
F2F
57
1.46
0.66
35
19
2
1
0
t(109) = -2.67
p = .0084
χ2 = 0.009
Note. F2F = face-to-face.
a1 = strongly agree, 2 = agree, 3 = neither agree nor disagree, 4 = disagree, 5 = strongly disagree.
Research Question 4 examined if differences exist in students' perceived ability to handle stress among
online students and F2F students. In order to answer Question 4, a comparison in the differences in
perceived ability to handle stress was completed using a t-test and the Mantel–Haenszel chi-squared test.
The results indicate that F2F students reported that they were slightly better able to handle stress
compared to online students (p < .003). However, the difference was not statistically significant (refer to
Table 5). Thus, the findings do not support an affirmative response for Research Question 4. No
differences exist in the learning outcome as measured by ability to handle stress among online students
and F2F students.
Table 5. Ability to handle stress
Group n M SD
Frequencies of Each Ratinga
1
2
3
4
5
Online
54
2.11
0.84
11
30
11
2
1
F2F
57
1.68
0.60
22
31
4
0
0
t(109) = -3.09
p = .0025
χ2 = 0.0029
Note. F2F = face-to-face.
a1 = very well, 5 = not well.
Research Question 5 explored if differences exist between age and ethnicity of students and heart rate
drop, exam scores, awareness of stress, and ability to handle stress. In order to answer Question 5, a
comparison in the differences among these four learning outcomes and age and ethnicity was completed
using univariate ANOVA tests as well as full factorial ANOVA. Relationship between age groups and
ethnicity with awareness of stress and the ability to handle stress were also examined using the Mantel–
Haenszel chi-squared test. The results indicated that there are no significant differences between age
and ethnicity and heart rate drop, awareness of stress, and ability to handle stress (refer to Table 6).
In Exam 2, older students scored lower than younger students, and Latino students scored lower than
Caucasian students according to univariate ANOVA, but this trend did not surface in full factorial ANOVA.
Caucasian students reported greater heart rate drop using the Repeated Sounds technique, but this
effect was seen only in the full factorial model. Although some of these comparisons are statistically
significant, any differences observed do not hold to multiple tests. Thus, the authors conclude that the
findings do not support an affirmative response to Research Question 5. No trends were seen between
age and ethnicity of students and heart rate drop, exam scores, awareness of stress, and ability to handle
stress.
Discussion
Drop in Heart Rate
Results of the current study are somewhat similar to results of Ricks et al. (2011) in that both studies
provided audio recordings of relaxation techniques for participants. A somewhat minor difference between
the two studies was the primary measurement of relaxation: self-reported drops in heart rate in the
current study versus a perceived relaxed state in the study conducted by Ricks et al. Regardless, both
studies found that recordings of relaxation techniques produced a relaxed state. Another important
difference between the two studies was that the current study compared effectiveness of recorded
relaxation techniques specifically within the context of online learning relative to F2F classroom delivery.
Nevertheless, in both studies a relaxed state was induced among students by listening to recordings of
relaxation techniques.
186
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
Table 6. Age and ethnicity
Heart Rate Drop with ... Effect of ...
Univariate
Multivariate
F
p
F
p
Energy Breathing
Age
F(3,73) = 0.6630
.5774
F(3,56) = 1.2774
.2910
Ethnicity
F(3,73) = 0.0362
.9907
F(3,56) = 0.1088
.9546
Repeated Sounds
Age
F(3,53) = 0.8868
.4540
F(3,36) = 2.3327
.0904
Ethnicity
F(3,53) = 1.5241
.2189
F(3,36) = 3.8748
.0169*
Mental Imagery
Age
F(3,76) = 0.5111
.6758
F(3,57) = 0.2054
.8923
Ethnicity
F(3,76) = 0.6934
.5589
F(3,57) = 0.4704
.7041
Music Therapy
Age
F(3,72) = 0.1459
.9320
F(3,54) = 0.3323
.8020
Ethnicity
F(3,72) = 0.2548
.8577
F(3,54) = 0.9119
.4414
Autogenic Training
Age
F(3,75) = 1.6812
.1782
F(3,57) = 1.1790
.3258
Ethnicity
F(3,75) = 0.7965
.4997
F(3,57) = 2.1314
.1063
Exam 1
Age
F(3,105) = 2.5655
.0585
F(3,83) = 0.5956
.6197
Ethnicity
F(3,105) = 2.4452
.0680
F(3,83) = 1.1662
.3277
Exam 2
Age
F(3,106) = 4.5747
.0047*
F(3,84) = 2.0178
.1176
Ethnicity
F(3,106) = 5.6326
.0013*
F(3,84) = 1.0390
.3797
Exam 3
Age
F(3,106) = 1.7652
.1583
F(3,84) = 0.7356
.5337
Ethnicity
F(3,106) = 2.2973
.0818
F(3,84) = 0.1963
.8986
Awareness of Stress Age
F(3,106) = 0.4079
χ
2
= 0.4563
.7476
.4994
F(3,84) = 0.0451 .9872
Ethnicity
F(3,106) = 2.1893
χ
2
= 1.6221
.0936
.2028
F(3,84) = 1.1657 .3278
Ability to Handle Stress Age
F(3,106) = 1.8142
χ
2
= 0.6842
.1490
.4081
F(3,84) = 1.3255 .2716
Ethnicity
F(3,106) = 1.1591
χ
2
= 0.9929
.3290
.3190
F(3,84) = 0.8089 .4925
*p < .05.
It was surprising that online students' heart rates dropped more than F2F students with one technique
(autogenic training). The relaxation techniques were taught using the same narrative for the live
classroom sessions as the recorded online sessions. The stronger outcome with autogenic training in
online students may be due in part to a comfort factor. Students attending F2F may have been somewhat
uncomfortable closing their eyes and sitting in a darkened room full of other students while performing
this technique. Any sense of awkwardness or embarrassment would have impeded ability to decrease
heart rate and achieve relaxation compared to online students who practiced the techniques by
themselves in the comfort and privacy of their own home. However, if online students were more
comfortable than F2F students, there should have been a significant difference in heart rate drop with the
other relaxation techniques. Thus, the question remains as to why online students' heart rates dropped
significantly more than F2F students.
Exam Scores
Although knowledge acquisition may not be the most important learning objective for a stress
management class, it is an important precursor to healthy behavior modification efforts, such as
managing stress. Therefore, a secondary but nonetheless important learning outcome for this class is the
cognitive component of learning as measured by exam scores. The question of cheating is always a
concern in online classes that incorporate non-proctored online exams, even when limitations are
imposed. In the past, informal reviews of this stress management course indicate that exam scores are
not different for non-proctored online students and proctored F2F students. Therefore, it was not
surprising to the researchers that there were no significant differences of exam scores among online and
F2F students when scores for all three exams were analyzed together. It is assumed that the limitations
imposed on the online students (timed, random, unable to skip ahead or go back) prevented, to a large
extent, possible attempts to cheat. It is unclear, however, why online students outperformed F2F students
only on Exam 3. Just as the results relating to exam scores of the current study were somewhat mixed,
results of other studies pertaining to this factor are also mixed.
187
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
Greenberg et al. (2009) investigated the effects of in-class and online exams on undergraduate students'
performance on an in-class comprehensive final (n = 141) in a required educational psychology course in
a teacher education program. Students were randomly assigned by course section to take one proctored
exam in-class and two other unit exams online. At the end of the course, students in all sections took a
proctored comprehensive final, consisting of a series of multiple-choice questions closely aligned with
questions from the unit exams. No significant differences were found between content items initially
assessed utilizing the online and traditional, classroom-based formats.
On the other hand, results of the current study are in contrast to results obtained by Schultz, Schultz, and
Round (2008). The researchers compared final grades of online and F2F classroom-based courses.
Grades for the 2005 and 2006 calendar years were compared to determine if significant differences exists
between the two modes. In the case of all four classes, the researchers found a significant difference in
final grades. It was somewhat surprising that in each case, the mean grade for the online courses was
significantly less than those of the traditional, on-campus lecture classes. Thus, these results were
inconsistent with the results of the current study, specifically for Exam 3 where scores were significantly
higher among online students compared to F2F students.
Awareness of Stress
Both online and F2F students were required to complete a stress journal, an assignment that required
students to record their stressors and their thoughts, feelings, and physical reactions to those stressors,
and pre- and post-treatment ratings that portrayed the intensity of the stressor after incorporating a coping
or relaxation technique. The primary objective of this assignment was to increase awareness of when
students are experiencing stress. Because this assignment was required of both online and F2F students,
the researchers assumed there would be no significant difference among online and F2F students.
Therefore, the researchers were surprised that F2F students reported that they were more aware of when
they were stressed compared to online students. Perhaps the ability to perform relaxation techniques in a
more comfortable environment may have reduced the online students' overall stress level thus making
them less aware of stress.
Ability to Handle Stress
Similar to awareness of stress, perceived ability to handle stress is an important learning outcome in a
stress management course. Self-efficacy, or the belief that one has the ability to succeed with a particular
task to affect change, is a critical determining factor with any behavior modification effort. Students must
develop the belief that the tools about which they learn in this class will help them effectively deal with
stress. All of the course material and requirements (lectures, assignments, and exams) were exactly the
same for both the online and F2F sections. Because online students were able to effectively decrease
their heart rate similar to F2F students in four out of five techniques assessed, it is not surprising that
there were no significant differences of perceived ability to handle stress among online and F2F students.
Age and Ethnicity
It was somewhat surprising that age and ethnicity had no significant effect on any of the four learning
outcomes because of inherent differences of values and beliefs among various age groups and cultures.
However, results from other studies indicate these variables are mixed. Anstine and Skidmore (2005)
conducted an exploratory factor analysis on a large-scale (n = 1,056) study that determined student
barriers to online learning. Independent variables that significantly affected student ratings of these
barriers included gender, age, and ethnicity. Although perceived barriers to learning outcomes are
somewhat different than learning outcomes themselves, as was examined in the current study, there are
obvious similarities. Thus, results of this study are in contrast to results of the current study.
Jost, Rude-Parkins, and Githens (2012) investigated the effects age, gender, and ethnicity and their
interactions had on academic performance in online courses delivered by public 2-year colleges.
Although differences in final grades were present among age and ethnicity, these differences
disappeared when controlling for cumulative grade point average. Furthermore, Dutton, Dutton, and Perry
(2001) used two separate measures of academic performance in distance learning classes and found
that neither age nor gender were significant predictors of academic performance. Kotey and Anderson
(2006) and Lu, Yu, and Liu (2003) found similar results.
188
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
Limitations and Recommendations
Similar to the views of Sitzmann et al. (2006), the effectiveness of online learning may depend on both
the learning objectives and the learning conditions. Because F2F students felt more aware of stress than
online students, replicating the current study but controlling for learning conditions or environmental
factors may shed light on the influence of environmental factors on learning in both online and F2F
settings.
The online learning model used for this course was a "bare-bones" approach without multimedia,
discussion boards, or videos. Audio/visual recordings were the medium for disseminating the lectures and
audio-only recordings for the relaxation techniques. Additional research is necessary in order to
determine if a multimedia approach, in both online and F2F students, would influence students' heart rate
drops, perceptions of their ability to handle stress, and their awareness of stress.
One limitation of this study is the relatively small sample size. Future research with larger sample sizes is
needed to corroborate and reinforce the results of the present study. Another limitation is the inherent
problem with using self-reported data, especially with a physiological parameter. Utilizing heart rate
monitors that would more objectively measure heart rates is recommended for future research. Similarly,
further research is needed in order to compare the effectiveness of additional relaxation techniques as
measured by additional physiological parameters (e.g., galvanic skin response, blood pressure) with
online learning relative to F2F delivery in a physical classroom.
Because results are mixed both in the literature and the current study, further research regarding exam
scores for online learning relative to F2F classroom delivery is needed. Specifically, additional research is
recommended that examines test scores particularly in relation to varied learning outcomes within online
learning relative to F2F delivery. Also, with the recent influx of online proctoring companies, it may be
helpful to provide additional research that documents the effectiveness of online proctoring services.
Finally, with the disparate testing environment, it is questionable as to why the online students improved
significantly more on Exam 3, unless the students engaged in behaviors that put them at an advantage
over their F2F counterparts. Additional research may shed light on this.
Regarding age and ethnicity, additional research may contribute to understanding possible differences in
the effectiveness of online learning relative to F2F delivery based on learning objectives among varied
target populations. In other words, do some target populations learn better with online learning than
through F2F delivery with a particular type of learning objective?
Finally, further research is needed in order to determine factors influencing the affective domain of
learning (i.e., attitudes, beliefs, values, and perceptions) within the field of stress management as well as
other disciplines with non-traditional learning objectives (i.e., objectives that extend beyond the cognitive
domain of learning) specifically in courses using online learning compared to F2F delivery.
Conclusion
This paper has reported on a study that examined learning outcomes in a stress management course
using online learning relative to F2F instruction delivered in a traditional classroom. Learning outcomes
assessed were participants' exam scores, perceptions concerning awareness of and ability to handle
stress, and self-reported decreases in heart rates following five relaxation exercises. Impact of age and
ethnicity on learning outcomes were examined as well, and no significant differences were found among
heart rate drops following relaxation techniques with the exception of autogenic training (heart rate drops
were greater in online students). No significant differences were found in scores of two out of three
exams. Students undertaking the course in the F2F format felt more aware of stress compared to those
undertaking the course online, but there were no significant differences in perceived ability to manage
stress. No significant differences were found among age and ethnicity and any of the learning outcomes.
The researchers assumed that tapping into creativity and imagination and developing mental skills in
order to affect physiology would be better learned in a F2F format, rather than in an online environment.
Based on results of this study, the assumption that students learn relaxation techniques better via F2F
classroom interaction seems questionable. In fact, learning relaxation techniques, specifically autogenic
training, in an online environment may actually be more effective than learning in the traditional classroom
based on these results. Results of this study point to the need to further examine the effectiveness of
online learning relative to F2F delivery in courses with varied learning objectives, specifically objectives
that don't fit neatly into the cognitive domain of learning.
189
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
Several sections of this online stress management course are offered each year at this institution,
resulting in approximately 2,000 students enrolled annually in the online sections compared to only 350
enrolled annually in the F2F sections. The ultimate goal of this project was to determine if online students
are learning, particularly in the form of ability to drop heart rate, just as much as F2F students. More
specifically, the researchers wanted to know if receiving instruction in a F2F format is more beneficial
compared to receiving instruction in an online format in a stress management course. Results of this
study validate and justify continuing to offer this stress management course using online learning.
Anecdotally, online stress management courses do not seem to be a particularly common curricular
offering among institutions of higher education. The primary implication of this research is that perhaps
they can be. Based on results of the current study, it may be time to incorporate online stress
management into our standard curricula at the postsecondary level. Online students appear to learn
possibly the most important aspect of a stress management course, and that is the ability to relax.
References
Allen, I. E., & Seaman, J. (2007). Making the grade: Online education in the United States, 2006:
Midwestern edition. Needham, MA: The Sloan Consortium. Retrieved from
http://www.sloanconsortium.org/sites/default/files/Making_the_Grade_Midwest.pdf
Anstine J., & Skidmore, M. (2005). A small sample study of traditional and online courses with sample
selection adjustment. The Journal of Economic Education, 36(2), 107-127. Available from JSTOR
database. (30042641)
Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., ... Huang, B. (2004).
How does distance education compare with classroom instruction? A meta-analysis of the empirical
literature. Review of Educational Research, 74(3), 379-439. doi:10.3102/00346543074003379
Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals.
Handbook 1: Cognitive domain. New York, NY: David McKay.
Caywood, K., & Duckett, J. (2003). Online vs. on-campus learning in teacher education. Teacher
Education and Special Education: The Journal of the Teacher Education Division of the Council for
Exceptional Children, 26(2), 98-105. doi:10.1177/088840640302600203
Carpenter, K. M., Stoner, S. A., Schmitz, K., McGregor, B. A., Doorenbos, A. Z. (2014). An online stress
management workbook for breast cancer. Journal of Behavioral Medicine, 37(3), 458-468.
doi:10.1007/s10865-012-9481-6
Chiauzzi, E., Brevard, J., Thurn, C., Decembrele, S., & Lord, S. (2008). MyStudentBody-Stress: An
online stress management intervention for college students. Journal of Health Communication,
13(6), 555-572. doi:10.1080/10810730802281668
D'Orsie, S. M., & Day, K. (2006, February). Ten tips for teaching a web course. Tech Directions,
65(7),18-20.
Dutton, J., Dutton, M., & Perry, J. (2001). Do online students perform as well as lecture students?
Journal of Engineering Education, 90(1), 131-136. doi:10.1002/j.2168-9830.2001.tb00580.x
Fridrici, M., & Lohaus, A. (2009). Stress-prevention in secondary schools: Online- versus face-to-face
training. Health Education, 109(4), 299-313. doi:10.1108/09654280910970884
Frost, P. J., & Fukami, C. V. (1997). Teaching effectiveness in the organizational sciences: Recognizing
and enhancing the scholarship of teaching. Academy of Management Journal, 40(6), 1271-1281.
doi:10.2307/257033
Furst-Bowe, J. A., & Dittmann, W. (2001). Identifying the needs of adult women in distance learning
programs. International Journal of Instructional Media, 28(4), 405-413.
Greenberg, K., Lester, J. N., Evans, K., Williams, M., Hacker, C., & Halic, O. (2009). Student learning
with performance-based, in-class and learner-centered, online exams. International Journal of
Teaching and Learning in Higher Education, 20(3), 383-393. Retrieved from http://www.isetl.org/
ijtlhe/pdf/IJTLHE477.pdf
190
MERLOT Journal of Online Learning and Teaching Vol. 10, No. 2, June 2014
Johnson, S. D., Aragon, S. R., Shaik, N., & Palma-Rivas, N. (2000). Comparative analysis of learner
satisfaction and learning outcomes in online and face-to-face learning environments. Journal of
Interactive Learning Research, 11(1), 29-49. Available from EdITLib Digital Library. (8371)
Jost, B., Rude-Parkins, C., & Githen, R. P. (2012). Academic performance, age, gender, and ethnicity in
online courses delivered by two-year colleges. Community College Journal of Research and
Practice, 36(9), 656-669. doi:10.1080/10668921003744876
Kotey, B., & Anderson, P. (2006). Performance of distance learning students in a small business
management course. Education + Training, 48(8-9), 642-653. doi:10.1108/00400910610710065
Kramarae, C. (2001). The third shift: Women learning online. Washington, DC: American Association of
University Women Educational Foundation. Retrieved from http://www.aauw.org/files/2013/02/The-
Third-Shift-Women-Learning-Online-2001.pdf
Lu, J., Yu, C.-S., & Liu, C. (2003). Learning style, learning patterns, and learning performance in a
WebCT-based MIS course. Information & Management, 40(6), 497-507. doi:10.1016/S0378-
7206(02)00064-2
Parker, K., Lenhart, A., & Moore, K. (2011). The digital revolution and higher education: College
presidents, public differ on value of online learning. Washington, DC: Pew Research Center.
Retrieved from http://www.pewsocialtrends.org/2011/08/28/the-digital-revolution-and-higher-
education/
Ricks, J., Naquin, M., Vest, A., Hurtt, D., & Cole, D. (2011). Examining the results of podcast relaxation
techniques in higher education. ICHPER-SD Journal of Research, 6(1), 68-72. Available from ERIC
database. (EJ936023)
Salcedo, C. S. (2010). Comparative analysis of learning outcomes in face-to-face foreign language
classes vs. language lab and online. Journal of College Teaching & Learning, 7(2), 43-54. Retrieved
from http://www.cluteinstitute.com/ojs/index.php/TLC/article/download/88/85
Schuman, A. H., & Sims, R. L. (1999, June). Learning in an online format versus an in-class format: An
experimental study. T.H.E. Journal, 26(11), 54-56. Retrieved from http://thejournal.com/articles/
1999/06/01/learning-in-an-online-format-versus-an-inclass-format-an-experimental-study.aspx
Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). The comparative effectiveness of web-based
and classroom instruction: A meta-analysis. Personnel Psychology, 59(3), 623-664.
doi:10.1111/j.1744-6570.2006.00049.x
Somenarain, L., Akkaraju, S., & Gharbaran, R. (2010). Student perceptions and learning outcomes in
asynchronous and synchronous online learning environments in a biology course. MERLOT Journal
of Online Learning and Teaching, 6(2), 353-356. http://jolt.merlot.org/vol6no2/somenarain_0610.htm
Schultz, M. C., Schultz, J. T., & Round, G. (2008). Management of academic quality: A comparison of
online versus lecture course outcomes. Journal of College Teaching & Learning, 5(10), 23-28.
Retrieved from http://www.cluteinstitute.com/ojs/index.php/TLC/article/download/1225/1209
Zhao, Y., Lei, J. , Yan, B., Lai, C., & Tan, S. (2005). What makes the difference? A practical analysis of
research on the effectiveness of distance education. Teachers College Record, 107(8), 1836-1884.
doi:10.1111/j.1467-9620.2005.00544.x
This work is published under a Creative Commons Attribution-Non-Commercial-Share-Alike License
For details please go to: http://creativecommons.org/licenses/by-nc-sa/3.0/us/
191
