Content uploaded by Fallon Richie
Author content
All content in this area was uploaded by Fallon Richie on Oct 02, 2019
Content may be subject to copyright.
Topical Article
The Benefits, Drawbacks, and Challenges
of Using the Flipped Classroom in an
Introduction to Psychology Course
Patricia V. Roehling
1
, Lindsey M. Root Luna
1
, Fallon J. Richie
1
,
and John J. Shaughnessy
1
Abstract
Flipped pedagogy has become a popular approach in education. While preliminary research suggests that the flipped classroom
has a positive effect on learning in Science, Technology, Engineering, and Mathematics and quantitative courses, the research on
the flipped classroom in a content heavy social science course is minimal and contradictory. We flipped four class topics in an
introduction to psychology course, evaluated resulting student attitudes, and compared students’ performance on the flipped
units to their performance on traditionally delivered content. We found mixed results for the effectiveness of the flipped
classroom that were moderated by student characteristics and experiences with previous online or flipped courses. Students
reported an overall preference for traditional classroom delivery but suggested retaining the flipped approach for some class
periods.
Keywords
pedagogy, flipped classroom, inverted classroom
The flipped, or inverted, classroom is a relatively new peda-
gogy that has been widely touted as an innovative and effective
teaching method, which some propose should replace the tra-
ditional lecture format (Fitzpatrick, 2012; Mazur, 2009; Rosen-
berg, 2013). In a flipped classroom, the material that has been
customarily presented in lecture is moved outside class time,
generally in the form of digital videos (vodcasts), which stu-
dents view before the class period (Bowen, 2012). Flipped
pedagogy differs from a fully online format, however, as in-
seat class time is spent actively applying concepts, working
through problem sets (activities previously assigned as home-
work) as well as answering student questions and engaging
students in discussions.
Despite the strong endorsement of the flipped classroom,
research investigating its effectiveness has been sparse and has
primarily examined the effectiveness of the flipped classroom
in Science, Technology, Engineering, and Mathematics and
quantitative courses (STEM-Q). Few studies have examined
whether the flipped classroom is an effective pedagogy in
introductory-level psychology classes, which do not lend them-
selves as easily to in-class applications and assignments. The
goal of the current study is to examine the impact of the flipped
classroom in an introductory psychology course on both stu-
dent attitudes and learning.
There are many potential advantages associated with the
flipped classroom, which are commonly extolled, along with
less widely advertised potential pitfalls. Possible benefits
include the following: Students may view the vodcasts at their
own pace or review the material multiple times (Goodwin &
Miller, 2013); lectures can be broken down into smaller units,
and students can view them when most convenient and optimal
(Forsey, Low, & Glance, 2013; Jensen, 2011); active learning,
an effective teaching technique, can be employed during class
time (Daniel & Braasch, 2013; Freeman et al., 2014; Karpiak,
2011); and instructors can work individually and make deeper
connections with students who are having difficulties (Good-
win & Miller, 2013). The following drawbacks are also possi-
ble: Students may find the taped lectures less engaging than the
typical classroom lecture and encounter more distractions
when viewing the vodcasts (Jensen, 2011; Foertsch, Moses,
Strikwerda, & Listzkow, 2002); students prefer to have a
teacher available during lecture to answer questions (Chandra
& Fisher, 2009; Foertsch et al., 2002); some students are less
able to anticipate, schedule, and complete the out-of-class
learning (Dunning, Johnson, Ehrlinger, & Kruger, 2003); when
students watch taped lectures, there may be decreased compli-
ance with the reading assignments (McLaughlin et al., 2014);
1
Psychology Department, Hope College, Holland, MI, USA
Corresponding Author:
Patricia Roehling, Psychology Department, Hope College, Holland, MI 49423,
USA.
Email: roehling@hope.edu
Teaching of Psychology
2017, Vol. 44(3) 183-192
ªThe Author(s) 2017
Reprints and permission:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/0098628317711282
journals.sagepub.com/home/top
and finally, students may have technical problems download-
ing or viewing the vodcasts, particularly those with fewer
financial or technical resources (O’Bannon, Lubke, Beard, &
Britt, 2011).
Student Attitudes Toward the Flipped Classroom
Several studies have assessed attitudes toward the flipped
classroom in STEM-Q courses, with generally encouraging
results. Students have reported strongly positive reactions to
the flipped paradigm in multiple courses, including biology,
economics, nursing, and physics, (e.g., Critz & Knight, 2013;
Deslauriers, Schelew, & Weiman, 2011; Lage, Platt, & Treglia,
2000; Moravec, Williams, Aguilar, & O’Dowd, 2010). Not
only have students reported positive attitudes toward flipped
classes, but they have also endorsed a preference for flipped
pedagogy over traditional content delivery in courses in
organic chemistry, nutrition, accounting, and calculus (Chris-
tiansen, 2014; Gilboy, Heinerichs, & Pazzaglia, 2015;
McGivney-Burelle & Xue, 2013; Phillips & Trainor, 2014).
On college-administered, anonymous course evaluations,
flipped courses have received higher ratings than traditionally
taught lecture courses in engineering computing, statistics, and
physics (Foertsch et al., 2002; Peterson, 2016; Stelzer,
Brookes, Gladding, & Mestre, 2010; Wilson, 2013). However,
flipped pedagogy did not affect teacher or course evaluations in
an introductory spreadsheet class (Davies, Dean, & Ball, 2013).
To our knowledge, only two studies have examined student
attitudes toward the flipped paradigm in an introductory-level
social science class. Jensen (2011) flipped four class periods of
an introduction to psychology course and, in contrast to the
broadly positive responses to flipped STEM-Q courses, found
that students preferred the traditional lecture method and per-
ceived it as more helpful to their learning. Forsey, Low, and
Glance (2013) flipped a sociology course taught in Australia
and found that, after 2 weeks, students had mixed reactions.
They appreciated the flexibility and the short length of the
vodcasts but had concerns about technological compatibility
and aptitude and the loss of the shared lecture experience.
In summary, there has been a generally positive response to
the flipped classroom in STEM-Q courses, which may be the
ideal venue for the flipped classroom. STEM-Q topic areas
lend themselves easily to spending class time solving math,
engineering, and computing problems or engaging in labora-
tory and practical applications. In contrast, there is preliminary
evidence that students in an introduction to psychology course
do not respond as positively to the flipped classroom. Subject
matter in this course often involve ideas or concepts that are
less amenable to hands-on applications in the classroom than
those in STEM-Q courses.
Effectiveness of the Flipped Classroom
The literature evaluating the effectiveness of the flipped peda-
gogy is even more scant than the work investigating student
attitudes. The research that does exist is mixed, although
generally positive. Again, the majority of the research exam-
ines STEM-Q courses that involve the learning of practical
skills or the application of formulas or algorithms.
Within mathematics-related disciplines, three studies
included measures of effectiveness. Wilson (2013) and Peter-
son (2016) compared student performance in flipped statistics
classes with performance in traditionally taught classes from a
previous academic year and found that students’ overall course
grades in the flipped classes were significantly higher than
those in the previous semesters. In a more targeted application
of flipped pedagogy, McGivney-Burelle and Xue (2013)
flipped a difficult topic in a calculus class in one section while
using the lecture format in an alternate section. Students in the
flipped condition performed better on the relevant exam and
assignment than those in the traditional condition.
In an introductory biology course, Moravec, Williams,
Aguilar, and O’Dowd (2010) flipped three units in one section
and compared students’ performance with students from pre-
vious years who used the lecture-only format. Students who
participated in the flipped units performed better than those in
the lecture-only classes but only on the questions directly
related to the flipped activities.
Within physics courses, Stelzer, Brookes, Gladding, and
Mestre (2010) moved one third of the lecture material to an
online format and used active pedagogy during class time. This
change was related to modest increase in test scores when
compared to the previous semester. In contrast, Deslauriers,
Schelew, and Weiman (2011) used a slightly different peda-
gogy in their physics class, which they called the deliberate
practice method. Rather than viewing videos, students in the
experimental group engaged in separate readings and quizzes
outside the classroom. Deslauriers et al. found that students in
the deliberate practice condition performed almost twice as
well as students in the traditional lecture condition on a test
of the material.
Final exam scores and final course grades were used as
measures of effectiveness in two studies. In a graduate phar-
maceutics course, McLaughlin et al. (2014) flipped the
entire course; students in the flipped course attended more
regularly and earned higher final exam scores than students
in the previous nonflipped semester. Talley and Scherer
(2013) flipped one unit on synaptic transmission in an upper
level physiological psychology course (which they labeled a
STEM course). Students who experienced the flipped unit
earned higher final grades than those from the previous
semester who did not. It is not clear, however, whether this
difference was due to the flipped exercise or to confounding
factors (e.g., variation in class performance across years), as
information about performance on synaptic transmission test
questions was not provided.
Also looking at global measures, Lavelle, Stimpson, and
Brill (2013) examined course grade distributions. They com-
pared student performance in a flipped one-credit engineering
economics class with the performance of those in a tradition-
ally taught course from three previous semesters. The overall
distribution of grades was similar for the flipped and the
184 Teaching of Psychology 44(3)
traditionally taught courses. However, students in the flipped
class were more likely to fail the course than students in the
traditionally taught courses.
Two studies have examined the effectiveness of the flipped
class in introductory social science courses. Jensen (2011) flipped
four class periods of an introduction to psychology course and
found no difference between the online quiz scores of the two
groups. Lewis and Harrison (2012) utilized a quasi-experimental
design for an “introductory social science course” comparing test
scores of a traditionally taught course with scores of the same
class with flipped pedagogy. They found the students in the
flipped classroom outperformed the students in the traditional
courses on two of the four exams and on the final exam.
In conclusion, evidence suggests the flipped classroom
tends to have a positive impact on student performance and
learning in STEM-Q courses. The utility of the flipped class-
room in a content and theory heavy social science course, such
as introduction to psychology, is only beginning to be explored
and findings are inconclusive.
Moderators
Regardless of the context of the pedagogy, little is known about
which students may prefer or benefit from the flipped class-
room. For online classes, the research suggests the effective-
ness of the pedagogy is not moderated by gender, age, or grade
point average (GPA; Wan, Fang, & Neufeld, 2007). Although
the flipped class partially relies on online delivery, the class-
room component may limit the applicability of this finding.
Lage, Platt, and Treglia (2000) reported no differences between
men and women on preference for the flipped versus traditional
classroom format in an economics course. However, women
believed they learned more from the flipped format, and they
rated the course activities more positively than the men.
Researchers need to replicate and extend these findings with
other types of flipped courses.
Experience with technology may also moderate the relation-
ship between performance and teaching technique. Arbaugh
et al. (2009), in a review article, reported that performance is
enhanced for online business courses when the student has
prior experience with technology. Studies have shown that stu-
dents become more favorably inclined toward the flipped class-
room as they become more familiar with the experience (Butt,
2014; McLaughlin et al., 2014). Little is known, however, if
students’ previous experiences with flipped or online courses
influence their success in flipped classrooms.
Personality may also impact preference for and performance
in the flipped classroom. Schniederjans and Kim (2005) exam-
ined the relationship between the Big Five personality factors
and performance in an online business information systems
class and found that students who were high in conscientious-
ness, openness, stability, and agreeableness (but not extrover-
sion) performed better in the online class. To date, researchers
have not explored the role of personality in students’ responses
to flipped classrooms, which rely heavily on in-class discus-
sions and interactions. It may be that, unlike the online
environment, extroverts have an advantage or a preference for
these classes.
Finally, cognitive abilities may affect attitudes and perfor-
mance in the flipped classroom. Harrington (1999) found that
students with lower GPAs performed more poorly in an online
statistics class than those in a traditional class, whereas scores
of students with a higher GPA were not impacted by the online
course format. The relationship between cognitive factors and
performance in flipped versus traditional courses may be dif-
ferent from online courses because of the extra scaffolding
flipped classes offer. This scaffolding may be particularly ben-
eficial for less accomplished or prepared students.
Research Questions
The goal of the present study was to examine the impact of the
flipped classroom in introduction to psychology, a content-
heavy, introductory-level course. Secondarily, we intended to
extend the current research by evaluating whether cognitive,
personality, or demographic variables impact the effect of
teaching style on outcome and attitudes. Our four major
research questions were: (1) What are students’ attitudes
toward a flipped introduction to psychology course? (2) What
effect do cognitive, personality, and demographic variables
have on these attitudes? (3) How does flipping the classroom
affect test performance among students in an introductory psy-
chology course? and (4) How do personality characteristics,
cognitive characteristics, or demographic features moderate the
effectiveness of flipped pedagogy?
Method
Participants
The students who participated in the flipped classroom were
131 undergraduate students enrolled in an Introduction to Psy-
chology course at a small liberal arts college in the Midwest
during the spring semester. Participants were primarily female
(68.9%) and Caucasian (82%). Most students were in their first
year of college (71.3%), followed by 19.7%sophomores, 4.1%
juniors, and 4.9%seniors. Two professors each taught two
sections of the course. Students enrolled without knowing that
the course would include flipped class periods. Additionally,
students were provided with informed consent in order for the
researchers to obtain prior academic records and standardized
test scores. Students who participated in the study received
credit toward the research participation requirement for the
course. Ninety-three percent (121) of the introductory psychol-
ogy students agreed to complete the survey about the flipped
classroom and allowed us to access their college records.
As a comparison group, we used aggregated test data (per-
cent answering a question correctly for each test item) from 126
students from the previous spring semester. These students
received traditional instruction in the Introduction to Psychol-
ogy course with the same two faculty members (two sections
each). Women comprised 56%of the 2013 sample, which was
89%Caucasian.
Roehling et al. 185
Procedure
The instructors designated four topics to be flipped: Research
methods, sensation and perception, learning, and personality.
The other class periods were taught using traditional pedagogy.
These topics were chosen because (a) students often struggle
with the concepts and (b) the content lent itself to active learn-
ing activities. Each topic was covered in a single, 80-min class
period, with the exception of sensation and perception, which
was covered in two class periods.
The traditional class periods involved lectures, with occa-
sional activities and discussions, and students were expected to
read the textbook prior to coming to class. When class periods
were designated as flipped, students were instructed to prepare
for class by watching the assigned vodcast(s) along with read-
ing the text. Vodcasts were prerecorded videos in which the
professor presented the lecture material, using PowerPoint,
similar to the way a lecture would be presented in a traditional
class period. Each vodcast lasted 10–15 min, and students were
assigned to watch two to four vodcasts per flipped class period.
As part of the flipped portion of the class, students engaged in
interactive activities including worksheets individually or in
pairs, small group or full class discussions, enacting and role-
playing particular concepts, identifying ideas or definitions
from the vodcasts, in-class videos or text examples that encour-
aged application of concepts, and interactive demonstrations.
Measures
Demographics and individual differences. Participants provided
self-report data regarding their prior experience with flipped
and online courses, year in college, gender, and ethnicity. Stu-
dents also completed the Big Five Personality Inventory (John,
Naumann, & Soto, 2008), which yielded scores in extroversion,
neuroticism, agreeableness, conscientiousness, and openness to
experience. Students’ college GPA as well as American college
testing (ACT) and scholastic assessment test (SAT) scores were
obtained from the Registrar’s Office.
Attitudes toward flipped classes. Usefulness and interest of each
flipped class period were measured on a 3-point scale: 1 (very
useful or interesting), 2 (slightly useful or interesting), and 3
(not useful or interesting), whereas overall preference was
based on a 5-point scale from 1 (strong preference for lecture)
to 5 (strong preference for flipped). The students were invited
to provide comments after each of the questions and contribute
suggestions.
At the end of the course, students were queried about their
preferred teaching method (flipped vs. traditional lecture) on a
1(strongly prefer lecture style)to5(strongly prefer flipped
style) response range. Interest level and overall effectiveness
were also measure on a 1 (low interest and effectiveness)to5
(high) response range. Additionally, students were asked if
they would recommend the continued use of the flipped class-
room, which was measured on a 1 (yes, flip all class periods)to
5(no, use the lecture format only) response range.
Course evaluations. We compared student responses from the
flipped and nonflipped semesters to college-wide course eva-
luation that is administered at the conclusion of each semester.
There were three types of questions on this assessment: Course
evaluation questions, which began with “This course helped me
enhance the following skills and habits of learning ...”
included items on curiosity and technology use. These ques-
tions had five options, ranging from 1 (a great deal)to5(not at
all). Teacher Assessment questions, which began with “Please
select the term that you feel best describes your instructor ...”
included items that addressed the degree to which the instructor
structured course activities effectively, stimulated subject
interest, provided challenge, and provided support of student
learning. These items also had five options ranging from 1
(strongly agree)to5(strongly disagree). One question
assessed the overall value of the course, with five possible
responses ranged from 1 (very valuable)to5(not at all valu-
able). We collapsed categories if there were less than three
responses in a cell for each year.
Course performance. Student course performance was assessed
using multiple-choice exam questions. For each flipped unit,
some exam questions reflected both the course content that had
been delivered prior to class and reviewed using an in-class
exercise, while other exam questions covered material pre-
sented only in the preclass preparation (the textbook or vod-
cast). Course performance was assessed in three key ways.
First, the performance of students from the flipped condition
was compared with the performance of students in the non-
flipped condition, using the relevant exam questions (between
subjects). Second, we examined only the students in the flipped
condition, comparing their performance on the flipped course
components to their performance on the nonflipped elements
(within subjects). Third, we isolated performance on the
flipped units only; here, we compared students’ performance
on test questions covered in the flipped class activities to the
questions from the same unit but covered only in preparatory
materials (within subjects).
Results
Experience With Flipped and Online Classes
A minority of students had taken at least one online class
(20.2%in high school and 7.7%in college). The majority of
students had experience with flipped classes (26.9%in high
school and 74.0%in college).
Preferences and Attitudes Regarding Flipped Versus
Traditional Classes
A majority (56.2%) of students either slightly or strongly pre-
ferred lecture to the flipped classroom, whereas only 38%pre-
ferred the flipped method (see Table 1).
Despite a slight preference for the lecture format, most stu-
dents found the flipped classroom more interesting than lecture
186 Teaching of Psychology 44(3)
(57.0%). However, they also believed that the traditional lec-
ture style classroom was more effective in helping them to
learn the material (see Tables 2 and 3).
Overall, the vast majority (93.0%) of students recommended
using the flipped classroom teaching method to some degree
(see Table 4). Therefore, although most students preferred the
traditional lecture format to the flipped, almost every student
wanted some combination of flipped and traditional classroom
experiences.
Predictors of Attitudes Toward Flipped Versus
Traditional Classroom
w
2
Tests of independence revealed that preferences for teaching
method did not differ reliably across gender, year in college,
White and non-White, and students with different instructors
(ps >.11). Similarly, the Big Five personality traits were not
correlated with student preferences (ps > .11).
To understand how cognitive factors may have impacted
student perceptions of the flipped classroom, we correlated our
cognitive measures with student preferences. GPA, r(97) ¼
.20, p¼.049; mean test performance, r(119) ¼.36, p<
.001; and final exam score, r(117) ¼.38, p< .001, were all
significantly related to preference toward the flipped class-
room. The correlation between standardized test composite
percentile and preference trended toward significance r(97)
¼.18, p¼.081. Students with lower GPAs and lower exam
and standardized test scores tended to prefer the flipped class-
room format.
Effect of Flipped Classroom on Course Evaluations
To assess whether the flipped classroom was related to the
standard college-administered course evaluation, w
2
analyses
were conducted comparing course evaluations from the flipped
(2014) versus the non-flipped conditions (2013). Compared to
students taught using traditional methods, students in the
flipped condition were more likely to agree that the workload
was heavier (w
2
¼6.2, n¼173, p¼.04), they felt more
challenged (w
2
¼9.58, n¼205, p¼.05), they used technology
and the library (w
2
¼18.33, n¼192, p< .01), and the class was
effective (w
2
¼5.86, n¼205, p¼.05). A statistical trend
indicated students in the flipped condition more strongly
agreed that the student–faculty relationship supported learning
(w
2
¼6.3, n¼204, p< .08).
Relationship Between the Flipped Classroom
and Test Performance
To understand whether the flipped classroom pedagogy was
related to student test performance, we conducted a 2 (peda-
gogy: flipped, traditional) 2 (faculty member) 2 (year:
2013, 2014) repeated-measuresanalysisofvariance
(ANOVA). Each line of data represented a test item that was
used in both 2013 and 2014. The dependent variable was the
percentage of students who answered the item correctly, result-
ing in a within-test item comparison between 2013 (traditional
pedagogy) and 2014 (flipped pedagogy). There was a main
effect for year, F(1, 331) ¼14.56, p< .001, Z
2
¼.037. Overall,
students performed better in 2013 (M¼78.6%,SD ¼.17, n¼
355) than in 2014 (M¼76.9%,SD ¼.18, n¼335), with a
mean difference of 2.3%,95%confidence interval [CI]: [1.1,
3.5]. The interaction effect of Year Pedagogy was not sta-
tistically significant, F(1, 331) ¼0.28, p¼.60, nor was the
main effect of pedagogy F(1, 331) ¼.30, p¼.25. Thus, stu-
dents’ performance was not related to whether the class was
flipped. The main effect for faculty was not significant F(1,
331) ¼.06, p¼.81. The Faculty Year interaction effect was
significant, F(1, 331) ¼39.62, p< .001, Z
2
¼.102; however,
the Year Faculty Pedagogy interaction effect was not
significant, F(1, 331) ¼2.11, p¼.15.
Table 2. Which Classroom Experience Was Most Interesting?
Response Percentage (n)
Lecture is much more interesting 9.9 (12)
Lecture is slightly more interesting 15.7 (19)
Same 17.4 (21)
Flipped classroom is slightly more interesting 37.2 (45)
Flipped classroom is much more interesting 19.8 (24)
Table 3. Which Classroom Experience Was Most Effective?
Response Percentage (n)
Strong preference for lecture 20.7 (25)
Slight preference for lecture 28.9 (35)
No preference 11.6 (14)
Slight preference for flipped classroom 28.9 (35)
Strong preference for flipped classroom 9.9 (12)
Table 4. Preferred Mix of Lecture and Flipped Classes.
Response Percentage (n)
Flip all 3.4 (4)
Flip more 20.3 (24)
Keep current mix 54.2 (64)
Fewer flipped 15.2 (18)
All lecture 6.8 (8)
Note.N¼121.
Table 1. Preference for Flipped Versus Traditional Classroom.
Response Percentage (n)
Strongly prefer lecture 21.5 (26)
Slightly prefer lecture 34.7 (42)
No preference 5.8 (7)
Slightly prefer flipped 30.6 (37)
Strongly prefer flipped 7.4 (9)
Roehling et al. 187
Next, we examined student performance on only the units
that were flipped (research methods, learning, sensation and
perception, and personality). This isolated the impact of the
flipped exercises on test performance, comparing performance
on test items that were covered by an in-class exercise/discus-
sion and test items covered only by the textbook and vodcast.
We conducted a 2 (exercise: exercise, no exercise) 2 (faculty
member) 2 (year: 2013, 2014) ANOVA for only those test
items from the flipped units (above). The dependent variable
was the percentage correct, a within test-item comparison
between 2013 and 2014. There was a main effect of year,
F(1, 75) ¼4.13, p¼.046, Z
2
¼.040. Again, students per-
formed better in 2013 (M¼80.8%,SD ¼.16) than 2014 (M¼
78.6%,SD ¼.18), a mean difference of 2.2%,95%CI [0.04,
4.1]. There was also a marginal effect for the Year Exercise
interaction, F(1, 75) ¼3.81, p¼.054, Z
2
¼.037. Figure 1
illustrates several points about this relationship. First, the
flipped exercises covered concepts that students had most dif-
ficulty with in 2013 (3.0%lower than items related to concepts
that were not flipped in 2014). Second, on average, students
performed more poorly in 2014 than in 2013. Third, on the
questions that were related to a class exercise, we did not see
the expected decrease in 2014 test scores that occurred with
nonflipped units (1.2%decrease) and with test items on flipped
units for which we did not create an exercise (3.6%decrease).
This suggests that the flipped exercises did help students learn
the material. Finally, the 3.6%decrease in the scores for the
flipped unit questions that were not related to an exercise was
larger than expected when compared to the decrease in scores
on the nonflipped units (1.2%). It appears the exercises neu-
tralized the overall poorer performance of students in 2014;
however, the flipped pedagogy may have also had a slightly
negative effect on the retention of information not covered in
class exercises.
As with the analysis of the flipped unit versus nonflipped
unit, the Year Faculty interaction effect was again signifi-
cant, F(1, 74) ¼20.29, p< .001, Z
2
¼.378. However, the three-
way interaction effect (Exercise Faculty Year) was not
statistically significant, F(1, 74) ¼0.07, p¼.79.
Moderators of the Relationship Between the Flipped
Classroom and Student Performance
First, to test categorical moderators (i.e., gender, ethnicity, and
year in college), we used mixed ANOVAs, comparing stu-
dents’ average scores on test items related to flipped units
versus nonflipped units. The majority (i.e., gender, ethnicity,
and year in college) did not have a statistically significant
impact (ps > .46). However, there was a significant interaction
effect of Experience Unit, F(1, 120) ¼4.33, p¼.04, Z
2
¼
.035. As Figure 2 displays, students with previous online class
experience performed better (M¼77.5%,SD ¼.09) on the
flipped units than on the traditionally taught units (M¼74.5%,
SD ¼.10), a mean difference of 3.0%,95%CI [0.6, 5.4],
whereas method of instruction did not affect scores of students
without a history of online courses.
We next examined the potential continuous moderators:
College GPA, standardized test scores (percentile), personality
traits, and preference for the flipped classroom. To represent
the impact of flipped pedagogy, we created a difference vari-
able, subtracting the average score on traditional content test
items from the average score on flipped content test items. We
correlated this variable with our potential moderators. These
correlations were not statistically significant (ps > .08) with the
exception of neuroticism, r(102) ¼.20, p¼.045. Students
higher in neuroticism scored higher on the flipped content test
items, relative to the traditional content items.
To identify which students, if any, benefited most from the
questions related to flipped exercises relative to the nonflipped
exercises, we used similar procedures, isolating flipped units,
comparing questions that were covered by an exercise with
those that were not covered by an exercise. Among categorical
variables, only year in college had a statistically significant
Year Exercise effect, F(1, 113) ¼6.46, p¼.012, Z
2
¼
.054. As seen in Figure 3, first-year students performed better
on test items with exercises (M¼75.8%,SD ¼.15) compared
to test items without exercises (M¼72.2%,SD ¼.15), a mean
difference of 3.3%,95%CI [0.7, 5.9].
To evaluate the continuous moderators, we correlated the
difference variable with the demographic, cognitive, and
75
76
77
78
79
80
81
82
83
41023102
Percent of students who answered correctly
flipped - exercise
flipped - no exercise
non-flipped topic
Figure 1. Test performance on flipped topics that were covered by an
in-class exercise versus test items that were not covered in class.
73
74
75
76
77
78
Experience No Experience
Percent Correct
Tradional
Flipped
Figure 2. Relationship between experience with online courses and
performance on test questions from flipped versus traditional (non-
flipped) class units.
188 Teaching of Psychology 44(3)
personality variables. Only the correlation with GPA
approached statistical significance, r(99) ¼.19, p¼.053. Stu-
dents with lower GPAs tended to perform more poorly on
flipped test items that were not directly covered by a class
exercise.
Discussion
This study extends our understanding of the attitudes toward
and the effectiveness of the flipped classroom in an introduc-
tory social science course, specifically introduction to
psychology.
Attitudes Toward Flipped Pedagogy
Our results regarding preference replicate and extend past
research on an introductory psychology course. Similar to Jen-
sen’s (2011) findings, students showed a slight preference for
the traditional teaching method over the flipped method, and
they reported that the traditional method was more effective for
learning. Students in our study, however, also report that the
flipped class was more interesting than the traditional lecture,
and the majority supported including some flipped classes dur-
ing the semester. These results diverge from the research on
flipped STEM-Q courses in which students believed the flipped
paradigm to be highly effective and tended to prefer it to tra-
ditional lecture (e.g., Christiansen, 2014; Deslauriers et al.,
2011; McGivney-Burelle & Xue, 2013; Wilson, 2013). We
believe this difference is due to the material in an introduction
to psychology course, which is less amenable to “flipping” than
material in the STEM-Q courses. As one student stated, “I liked
the activities for the flipped class, but the teaching style of
lecture would have been better for the material.”
The relationship between personality variables and the
majority of the demographic variables with pedagogical pre-
ference was nonsignificant. Only academic performance
affected preference. Students with lower college GPAs and
students with lower class test scores preferred the flipped class-
room more than higher performing students. Lower performing
students may need repeated exposure and more time interacting
with the material for long-term retention. Although flipped
pedagogy is not the only way to provide this extra time on task
(Chickering & Gamson, 1991), the flipped classroom does pro-
vide a better fit for the learning needs of these students than the
traditional classroom. In contrast, high-performing students
may comprehend and retain the information after one exposure,
rendering the videos and exercises redundant. As one student
stated:
I like a lecture style class best because often I only need to hear
things once for them to stick in my mind, so having to read the
modules, watch the vodcasts, and do the activities in class were a
bit of a waste of time for me.
The material in an introduction to psychology class is vast;
students often master the gist, but the challenge is to master the
breadth and depth of the material. In contrast, STEM-Q courses
often require the application of specific equations or algo-
rithms. In-class exercises focus on a targeted number of topics,
prioritizing depth over breadth. The flipped classroom provides
extra opportunities to learn and practice in a supervised setting,
which may explain student preferences for flipped pedagogy in
the STEM-Q courses and lecture in the introductory psychol-
ogy class.
Effectiveness of Flipped Pedagogy
The flipped classroom appears to have had a very modest and
nuanced effect on learning in an introductory psychology class.
The flipped class exercises resulted in increased performance
on test items directly related to those exercises (cf. Moravec
et al., 2010). However, this increase may come at a slight cost
to the test items not covered by a class exercise. This may
clarify the contradictory findings of Lewis and Harrison
(2012) and Jensen (2011). As the convergence of concepts
included in the class period and the concepts covered on the
test increases, we would expect the positive impact of flipped
methodology on test scores to increase. Lewis and Harrison
may have tested more heavily on concepts covered in class
activities.
Expanding the existing literature, we have identified vari-
ables that may impact the effectiveness of the flipped peda-
gogy. Specifically, prior experience can influence the
effectiveness of the flipped methodology. First-year students
and novice students may have interpreted the lack of class
coverage as a signal that those concepts were not important
and therefore neglected them. Alternatively, the increased
demands associated with the flipped classroom (e.g., additional
vodcasts) may have led students to take shortcuts (such as
skimming readings or multitasking during videos), resulting
in less careful course preparation. Indeed, in course evalua-
tions, flipped course students rated it as more challenging and
time intensive than the students who experienced the traditional
approach. One student summed up this sentiment, “Having to
watch the vodcasts and read the modules was quite time
70
71
72
73
74
75
76
77
78
Exercise No Exercise
Percent correct
1st Year
2nd-4th Year
Figure 3. Relationship between class year and performance on test
questions from flipped classroom units that were covered versus were
not covered by a class exercise.
Roehling et al. 189
consuming. If all my classes were like that, it would be pretty
difficult to get everything done.” Some students may not
devote the time necessary to learn the material not covered in
class exercises.
In contrast to the flipped introductory psychology courses,
flipped STEM-Q classes appear to have a more consistently
positive effect on student performance (e.g., Deslauriers
et al., 2011; McGivney-Burelle & Xue, 2013; Stelzer et al.,
2010; Wilson, 2013).
Moderators of the effectiveness of flipped pedagogy were
both similar and dissimilar to online environments. Neither
gender nor age was significant moderators (cf., Wan et al.,
2007). Of the Big Five personality characteristics, only neuro-
ticism moderated the relationship between pedagogy and test
performance; students higher in neuroticism performed better
on the flipped units than the nonflipped units. This diverges
from the research related to online classes, which found that
students who were high on conscientiousness, openness, stabi-
lity, and agreeableness performed better in online compared to
traditional courses (Schniederjans & Kim, 2005). The varied
environments of flipped and online classrooms seem to impact
the relevance of personality traits regarding performance. The
multiple opportunities to learn in the flipped classroom may
provide an added level of reassurance to the highly anxious
(neurotic) students, positively impacting their achievement.
Additionally, students with previous experience online class
experience benefited from the flipped classroom, whereas
those without prior experience did not. Learning how to
approach the flipped classroom may be a skill that students
need to develop to receive the benefits of the flipped paradigm
in an introductory psychology course.
Our findings suggest that students with lower GPAs and
first-year students did not benefit as much from the flipped
exercises as those with higher GPAs and upper level students.
Specifically, they tended to do poorly on test questions based
on content not covered in class. These results may be due to the
approach lower performing and inexperienced students take
toward the flipped material. They may rely more heavily on
classroom lecture than independent reading when learning the
material. If lecture is removed, students may not get the under-
lying scaffolding they need to benefit from flipped class exer-
cises. Additionally, underperforming students may substitute
the vodcasts for the reading, which may be a less effective form
of learning (Daniel & Woody, 2010). Ironically, students with
lower GPAs were more likely to prefer the flipped environ-
ment. Although flipped pedagogy has been marketed as a way
to effectively reach more students (e.g., Bergmann & Sams,
2012), our findings suggest that this approach may have privi-
leged already successful and more experienced students.
Summary and Recommendations
The flipped classroom has the potential to be an effective peda-
gogy in an introduction to psychology course. Given students’
preferences, we recommend faculty use a combination of lec-
ture and flipping in the introductory course. However, faculty
must choose class activities carefully. The concepts considered
central to psychology will be best emphasized in in-class exer-
cises, as in-class exercises appear to increase retention. Addi-
tionally, we suggest faculty judiciously select topics amenable
for class exercises. For example, hands-on experiences and
discussions can easily reinforce the concepts presented in vod-
casts and readings for research methods and sensation and
perception. Finally, rather than flipping an entire class period,
portions of a class could be flipped, retaining lecture for con-
cepts that don’t lend themselves to in-class exercises.
For optimal impact, faculty will want to orient students to
the most effective ways of consuming online resources. This
could be particularly helpful for novice, first-year, and lower
performing students who do not appear to benefit from the
pedagogy as much as other students. Learning how to approach
the flipped classroom (e.g., avoiding distractions and taking
notes while watching videos) may be a skill that students need
to develop to get the full benefit of the paradigm. Faculty must
also signal to students that information not covered during class
time is still important to learn. This can be achieved through
quizzes or worksheets that hold students accountable for learn-
ing prior to coming to class, which other researchers have
included in their study design (e.g., McLaughlin et al., 2014;
Wilson, 2013). With guidance and experience, students may
grow to rely less on the instructor, taking more responsibility
for their learning via flipped pedagogy.
Limitations and Future Research
Flipped classes are only as good as the class exercises and
vodcasts that accompany them. Additional research is needed
to explore the benefits of the flipped classroom in introduction
to psychology and other social science courses, specifically
which techniques and exercises should be paired with each type
of course material to maximize learning. Research on the effec-
tiveness of the flipped classroom for a more diverse group of
students, particularly international students, is also recom-
mended. Students with limited or developing English skills
could control the pace of the vodcasts and replay the recordings
to understand concepts that are unclear to them, benefiting their
learning. These unanswered questions underscore the need for
more investigation of this still new pedagogy. While the flipped
classroom has a great deal of potential for psychology students,
educators need to approach it with caution to ensure that it is
used appropriately and benefits all students.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to
the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the
research, authorship, and/or publication of this article: This research
was supported in part by an award to Hope College from the Howard
Hughes Medical Institute through the Undergraduate Science Educa-
tion Program.
190 Teaching of Psychology 44(3)
References
Arbaugh, J. B., Godfrey, M. R., Johnson, M., Pollack, B. L., Niendorf,
W. W., & Wresch, W. (2009). Research in online and blended
learning in the business disciplines: Key findings and possible
future. Internet and Higher Education,12, 71–87. Retrieved from
http://dx.doi.org/10.1016/j.iheduc.2009.06.006
Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every
student in every class every day. Alexandria, VA: International
Society for Technology in Education.
Bowen, J. A. (2012). Teaching naked: How moving technology out of
your classroom will improve student learning. San Francisco, CA:
John Wiley.
Butt, A. (2014). Student views on the use of a flipped classroom
approach: Evidence from Australia. Business Education & Accred-
itation.6, 33–43.
Chandra, V., & Fisher, D. L. (2009). Students’ perceptions of a
blended web-based learning environment. Learning Environment
Research,12, 31–41. Retrieved from http://dx.doi.org/10.1007/
s10984-008-9051-6
Chickering, A. W., & Gamson, Z. F. (1991). Applying the seven prin-
ciples for good practice in undergraduate education. New direc-
tions for teaching and learning. San Francisco, CA: Jossey-Bass.
Christiansen, M. A. (2014). Inverted teaching: Applying a new peda-
gogy to a university organic chemistry class. Journal of Chemical
Education,91, 1845–1850. Retrieved from http://dx.doi.org/10.
1021/ed400530z
Critz, C. M., & Knight, D. (2013). Using the flipped classroom in
graduate nursing education. Nurse Educator,38, 210–213.
Retrieved from http://dx.doi.org/10.1097/NNE.
0b013e3182a0e56a
Daniel, D. B., & Woody, W. D. (2010). They hear, but they do not
listen: Retention for podcasted material in a classroom context.
Teaching of Psychology,37, 199–203. Retrieved from http://dx.
doi.org/10.1080/00986283.2010.488542
Daniel, F., & Braasch, J. L. G. (2013). Application exercises improve
transfer of statistical knowledge in real-world situations. Teaching
of Psychology,40, 200–207. Retrieved from http://dx.doi.org/10.
1177/0098628313487462
Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom
and instructional technology integration in a college-level infor-
mation systems spreadsheet course. Educational Technology
Research and Development,61, 563–580. Retrieved from http://
dx.doi.org/10.1007/s11423-013-9305-6
Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning
in a large-enrollment physics class. Science,332, 862–864.
Retrieved from http://dx.doi.org/10.1126/science.1201783
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why
people fail to recognize their own incompetence. Current Direc-
tions in Psychological Science,12, 83–87. Retrieved from http://
dx.doi.org/10.1111/1467-8721.01235
Fitzpatrick, M. (2012, June 24). Classroom lectures go digital. The
New York Times. Retrieved from http://www.nytimes.com/2012/
06/25/us/25iht-educside25.html?_r¼0
Foertsch, J., Moses, G., Strikwerda, J., & Litzkow, M. (2002). Rever-
sing the lecture/homework paradigm using eTEACH web-based
streaming video software. Journal of Engineering Education,91,
261–274.
Forsey, M., Low, M., & Glance, D. (2013). Flipping the sociology
classroom: Towards a practice of online pedagogy. Journal of
Sociology,49, 471–485. Retrieved from http://dx.doi.org/10.
1177/1440783313504059
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okorafor, N.,
Jordt, H., & Wenderoth, M. P. (2014). Active learning increases
student performance in science, engineering, and mathematics.
Proceedings of the National Academy of Sciences,111,
8410–8415.
Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing
student engagement using the flipped classroom. Journal of Nutri-
tional Education and Behavior,47, 109–114. Retrieved from
http://dx.doi.org/10.1016/j.jneb.2014.08.008
Goodwin, B., & Miller, K. (2013). Evidence on flipped classrooms is
still coming in. Educational Leadership,70, 78–80.
Harrington, D. (1999). Teaching statistics: A comparison of traditional
classroom and programmed instruction/distance learning
approaches. Journal of Social Work Education,35, 343–352.
Jensen, S. S. (2011). In-class versus online video lectures: Similar
learning outcomes, but a preference for in-class. Teaching of Psy-
chology,38, 298–302. Retrieved from http://dx.doi.org/10.1177/
0098628311421336
John, O. P., Naumann, L. P., & Soto, C. J. (2008). Paradigm shift to the
integrative big-five trait taxonomy: History, measurement, and
conceptual issues. In O. P. John, R. W. Robins, & L. A. Pervin
(Eds.), Handbook of personality: Theory and research (pp.
114–158). New York, NY: Guilford Press.
Karpiak, C. P. (2011). Assessment of problem-based learning in the
undergraduate statistics course. Teaching of Psychology,39,
251–254.
Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom:
A gateway to creating an inclusive learning environment. The
Journal of Economic Education,31, 30–43. Retrieved from
http://dx.doi.org/10.2307/1183338
Lavelle, J. P., Stimpson, M. T., & Brill, E. D. (2013). Flipped out
engineering economy: Converting a traditional class to an inverted
model. In A. Krishnamurthy & W. K. V Chan (Eds.), Proceedings
of the 2013 Industrial and Systems Engineering Research Confer-
ence (pp. 397–406). Institute of Industrial and Systems Engineers
(IISE).
Lewis, J. S., & Harrison, M. A. (2012). Online delivery as a course
adjunct promotes active learning and student success. Teaching of
Psychology,39, 72–76. Retrieved from http://dx.doi.org/10.1177/
0098628311430641
Mazur, E. (2009). Farewell, lecture? Science,323, 50–51. Retrieved
from http://dx.doi.org/10.1126/science.1168927
McGivney-Burelle, J., & Xue, F. (2013). Flipping calculus. Primus,
23, 477–486.
McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N.,
Davidson, C. A., Griffin, L. M., ...Mumper,R.J.(2014).The
flipped classroom: A course redesign to foster learning and engage-
ment in a health professions school. Academic Medicine,89,1–8.
Moravec, M., Williams, A., Aguilar-Roca, N., & O’Dowd, D. K.
(2010). Learn before lecture: A strategy that improves learning
Roehling et al. 191
outcomes in a large introductory biology class. CBE-Life Sciences
Education,9, 473–481.
O’Bannon, B. W., Lubke, J. K., Beard, J. L., & Britt, V. G. (2011).
Using podcasts to replace lecture: Effects of student achievement.
Computers & Education,57, 1885–1892. Retrieved from http://dx.
doi.org/10.1016/j.compedu.2011.04.001
Peterson, D. J. (2016). The flipped classroom improves student
achievement and course satisfaction in a statistics course: A
quasi-experimental study. Teaching of Psychology,43, 10–15.
Retrieved from http://dx.doi.org/10.1177/0098628315620063
Phillips, C. R., & Trainor, J. E. (2014). Millennial students and the
flipped classroom. Journal of Business and Educational Leader-
ship,5, 102–112.
Rosenberg, T. (2013, October 23). In ‘flipped’ classrooms, a method
for mastery. The New York Times. Retrieved from http://opiniona
tor.blogs.nytimes.com/2013/10/23/in-flipped-classrooms-a-
method-for-mastery/
Schniederjans, M. J., & Kim, E. B. (2005). Relationship of student
undergraduate achievement and personality characteristics in a
total web-based environment: An empirical study. Decision
Sciences Journal of Innovative Education,3, 205–221.
Stelzer, T., Brookes, D. T., Gladding, G., & Mestre, J. P. (2010).
Impact of multimedia learning modules on an introductory
course on electricity and magnetism. American Journal of Phy-
sics,78, 755–759. Retrieved from http://dx.doi.org/10.1119/1.
3369920
Talley, C. P., & Scherer, S. (2013). The enhanced flipped classroom:
Increasing academic performance with student-recorded lectures
and practice testing in a flipped STEM course. The Journal of
Negro Education,82, 339–357.
Wan, Z., Fang, Y., & Neufeld, D. J. (2007). The role of information
technology in technology-mediated learning: A review of the past
for the future. Journal of Information Systems Education,18,
183–191.
Wilson, S. G. (2013). The flipped class: A method to address the
challenges of an Undergraduate Statistics course. Teaching of Psy-
chology,40, 193–199. Retrieved from http://dx.doi.org/10.1177/
0098628313487461
192 Teaching of Psychology 44(3)
