The Experience of Three Flipped Classrooms in an Urban University: An Exploration of Design Principles

Abstract

As a response to the call for technology enhanced, student-centered learning environments, the flipped classroom approach has drawn much attention from both the research and practice communities. Despite over fifteen years of flipped classroom implementation, design principles have been minimally elaborated upon in relation to diverse disciplinary contexts. Focusing on this gap, we engaged in a mixed methods study that examined three instances of the flipped classroom across unique disciplines and to extract specific design principles. Three instructors and 115 students enrolled in three separate classes in fall 2012 participated in the study. Building upon the Revised Community of Inquiry Framework, we developed a flipped classroom design framework and identified nine design principles.

Full text

The Experience of Three Flipped Classrooms in an Urban University: An
Exploration of Design Principles
Min Kyu Kim, So Mi Kim, Otto Khera, Joan Getman
PII: S1096-7516(14)00021-9
DOI: doi: 10.1016/j.iheduc.2014.04.003
Reference: INTHIG 536
To appear in: The Internet and Higher Education
Received date: 7 January 2014
Revised date: 1 April 2014
Accepted date: 24 April 2014
Please cite this article as: Kim, M.K., Kim, S.M., Khera, O. & Getman, J., The Expe-
rience of Three Flipped Classrooms in an Urban University: An Exploration of Design
Principles, The Internet and Higher Education (2014), doi: 10.1016/j.iheduc.2014.04.003
This is a PDF file of an unedited manuscript that has been accepted for publication.
As a service to our customers we are providing this early version of the manuscript.
The manuscript will undergo copyediting, typesetting, and review of the resulting proof
before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that
apply to the journal pertain.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
The Experience of Three Flipped Classrooms in an Urban University:
An Exploration of Design Principles
Author Names and Affiliations:
Min Kyu Kim, Ph.D. (Corresponding Author)
Center for Scholarly Technology
University of Southern California, Los Angeles, CA 90089-2571 USA
Email: kimmk@usc.edu / Phone: (706) 614-6920
So Mi Kim, ABD
Learning, Design, and Technology
University of Georgia, Athens, GA, USA
221 Rivers Cross, Athens, GA 30602
E-mail: cotton93@uga.edu
Otto Khera, Senior Manager
Center for Scholarly Technology
University of Southern California, Los Angeles, CA 90089-2571 USA
Email: khera@usc.edu
Joan Getman, Director
Center for Scholarly Technology
University of Southern California, Los Angeles, CA 90089-2571 USA
Email: jgetman@usc.edu
Abstract
As a response to the call for technology enhanced, student-centered learning environments, the
flipped classroom
a
pproach has drawn much attention from both the research and practice
communities. Despite over fifteen years of flipped classroom implementation, design principles
have been minimally elaborated upon in relation to diverse disciplinary contexts. Focusing on
this gap, we engaged in a mixed methods study that examined three instances of the flipped
classroom across unique disciplines and to extract specific design principles. Three instructors
and 115 students enrolled in three separate classes in fall 2012 participated in the study. Building

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
upon the Revised Community of Inquiry Framework, we developed a flipped classroom design
framework and identified nine design principles.
Keywords: flipped classroom, inverted classroom, technology-enhanced learning, blended
learning, student-centered learning, mixed methods research

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
The Experience of Three Flipped Classrooms in an Urban University:
An Exploration of Design Principles
1. Introduction
Over the past decades higher education standards have emphasized the potential value of
student-centered learning environments in which students are actively engaged in higher-order
tasks and taking charge of their own learning (Hannafin, Hill, & Land, 1997; Means, 1994; Shea
et al., 2012). Student-centered learning environments necessitate applying more active learning
strategies to classroom teaching that, for example, involve student presentations, small group
problem solving, self and peer evaluation, and group discussions (Zappa, et al., 2009). Yet
creating such environments remains a challenge. Teachers are not necessarily prepared to apply
new pedagogies or to support the expanded roles and responsibilities associated with student-
centered learning. This is evidenced by challenges encountered in designing and supporting
student-centered learning (Brush & Saye, 2000; Hannafin et al., 1997). For example, teachers
often have difficulties managing their finite classroom time and limited number of face-to-face
classroom meetings to achieve an effective balance between lectures and active learning
strategies (Stayer, 2012). Instructors who are implementing student-centered learning would
benefit from a set of teaching strategies and tools to ease the tension among these activities.
Flipped classroom models have attempted to address these challenges by allocating more
class time for active learning approaches and by leveraging accessibility to advanced
technologies to support a blended learning approach. A typical flipped classroom approach
provides students with access to online video lectures prior to in-class sessions so that students
are prepared to participate in more interactive and higher-order activities such as problem
solving, discussions, and debates. (Baker, 2000; Bergmann, Overmyer, & Wilie 2012; Davies et

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
al., 2013; Foertsch et al., 2002; Fulton, 2012; Hughes, 2012, Lage et al., 2000; Talbert, 2012;
Zappe et al., 2009). Students benefit from the outside classroom events because they can allocate
their time and pace their online learning to meet their individual levels of comprehension. In
face-to-face classroom sessions, students have the opportunity to become more active and
interactive through group activities rather than passively listening to lectures. Teachers in turn
are able to commit more in-class time to monitoring student performance and providing adaptive
and instant feedback to an individual or group of students (Fulton, 2012; Herreid & Schiller,
2013; Hughes, 2012).
Strayer (2012) posits that “the regular and systematic use of interactive technology”
(p. 172) makes flipped classroom models unique, countering a critique that flipped classroom
models are not new because teachers have always relied upon readings, and computer-assisted
instructions to prepare students for in-class activities. We argue here that the ‘systematic use’ of
technologies is influenced by the design of the flipped classroom instance. The design limitations
of previous flipped classroom studies are listed below.
1.1 Limitations Found in Previous Studies of the Flipped Classroom Approach
The design of flipped classrooms has often been limited to the concept of replacing in-
class instruction with videos and using class time for homework. In contrast, we define the
‘flipped classroom’ as an open approach that facilitates interaction between students and teachers,
and differentiated learning (Bergmann et al., 2012; Keefe, 2007; Lage et al., 2000; Tomilnson,
2003) by means of flipping conventional events both inside and outside of the classroom and
supporting them with digital technologies (Hughes, 2012). A notable pioneer of the flipped
approach, Lage et al. (2000), did not limit “ flipping” to lectures and homework, stating:

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Inverting the classroom means that events that have traditionally taken place
inside the classroom now take place outside the classroom and vice versa. The use of
learning technologies, particularly multimedia, provide new opportunities for students to
learn, opportunities that are not possible with other media. (p. 32).
Research is needed on what aspects of flipped classroom implementations
explicitly benefit teaching and learning. Zappe et al. (2009) experimented with a flipped
undergraduate engineering class, concluding that students perceived the course as having a
positive impact on their learning. Herreid and Schiller (2013) reported the benefits of flipped
classrooms based on the results of a large-scale survey administered to STEM case study
teachers who used flipping methods. However, these reports fall short of an explicit accounting
of what features of the flipped classroom yielded benefits for learners and instructors. Another
recently conducted experiment (Davies et al., 2013; Strayer, 2012) indicated that there was no
significant difference in student performance between flipped classrooms and traditional
classrooms. Strayer (2012) reported that students perceived a significantly lower level of
structural support to facilitate student conduct during flipped events, warning that this perceived
lack of support might lead to lower engagement. We posit here that this possibility (a perceived
dearth of support prior to and during a flipped event) does not indicate that the flipped classroom
approach is of low value to teaching and learning. Rather, we make the argument that it is
necessary to explicitly define the values connected with flipped classroom models.
Few studies detail the design principles of the flipped classroom, and we found no
scientific articles that detailed the flipped classroom design principles in our literature review.
Many studies discussed what benefits can be expected from flipping the class (Davies et al.,
2013; Foertsch et al., 2002; Fulton, 2012; Gannod et al., 2008), but fell short of defining and

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
building design principles for the flipped classroom. Bergmann and Sams (2012) suggested a list
of design considerations such as ‘time to learn new software’ and ‘support from administration.’
However, their guidance was limited to technological elements. Later, Bergmann et al. (2012)
listed what characterized the flipped classroom (e.g., “a means to increase interaction and
personalized contact time between students and teachers”) contrasting these elements to
misleading manifestations of what are ‘Not Flipped Classrooms.’ Their proposal of what defines
the flipped classroom suggests many potential discussions of what can be added to a list of
design principles for the flipped classroom. This study aims to define design principles for the
flipped classroom and those principles posited here build directly upon the first four design
principles suggested by Brame (n.d.) at the Vanderbilt University’s Center for Teaching:
Provide an opportunity for students to gain first exposure prior to class; provide an incentive for
students to prepare for class; provide a mechanism to assess student understanding; and provide
in-class activities that focus on higher-level cognitive activities.
1.2 Analytic Framework: the Revised Community of Practice (RCOI)
This study deploys the theory-driven analytic framework−Revised Community of Inquiry
(RCOI) (Garrison, Anderson, & Archer, 2000; Shea & Bidjerano, 2010; Shea et al., 2012; Swan
et al., 2012) – by first investigating the impact of the flipped classroom approach on three
participating classrooms as a means of eliciting a model that is able to guide the elaboration of
design principles. This framework posits that knowledge building results from the collaborative
interaction between active students and teachers particularly in online/blended learning
environments (Shea & Bidjerano, 2010; Shea et al., 2012). The RCOI framework theorizes four
elements that contribute to a successful learning environment: Cognitive Presence, Social

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Presence, Teaching Presence, and Learner Presence (see Table 1). Figure 1 illustrates that the
four RCOI components are featured in the student-centered learning environment.
Table 1
Revised Community of Inquiry Framework
Type of
Presence
Definition Example
Cognitive
Presence
Knowledge
building involving
critical and creative
thinking
Challenging tasks, the cyclical process of
practical inquiry, and a multivariate measure of
critical and creative thinking
Social
Presence
Encouraging
collegial settings
Discourses among students and instructor
that promote positive affect, interaction, and
cohesion
Teaching
Presence
Instructional
orchestration
appropriate to the
learning environments
Task sets such as organization, design,
discourse facilitation, and direct instruction
Learner
Presence
Self- and co-
regulation of learning
Self- and co-regulatory strategies that
marshal thoughts, emotions, motivations,
behaviors and strategies

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Figure 1. The RCOI framework applied in this study.
1.3 Goals and Research Questions of the Current Study
This study is based on a pilot project conducted at the University of Southern California
(USC) located in urban Los Angeles, with three undergraduate flipped classroom instances. Each
instance was explored in terms of the unique interpretation of “flipping a class” made by each
instructor, their respective flipping strategies, and how the instructors used technologies to
facilitate flipped classroom events according to their unique interpretations. Building on the
RCOI framework, this study aimed to investigate participants’ perceived values of the flipped
classrooms with respect to the RCOI components and to elaborate a design framework from
which design principles for the flipped classrooms could be specified. The following research
questions guided the study:
• How do the instructors interpret and apply ‘flipping’ to their classrooms?
• What are the students’ perceptions of the value of the flipped classroom?

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
• What are suggestions for the design of the flipped classrooms?
2. Research Context: Three Flipped Classrooms
USC is a large research institution with an enrollment of over 40,000 students. Since
2010 the university has transitioned from a commuter campus to a residential campus with
extensive housing and corresponding facilities. Borne out of a desire to reconcile the needs of
undergraduate residential learners who seek value in face-to-face classroom learning experiences
with the convenience and efficiency of online instruction, the flipped classroom project was
initiated in pursuit of providing better learning environments in which students can be more
engaged, active, and responsible for their learning. Over the period of the fall 2012 semester,
three classes, one each in engineering (ENG), social studies (SOC), and humanities (HUM)
participated in the project. The project rendered useful data on discipline-specific flipped
classroom applications including course events, feasible instructional technologies, and the
internal support resource allocations required, which could give rise to design principles for the
flipped classroom.
Table 2.
Three Flipped Classrooms
Engineering (ENG) Social Studies (SOC) Humanities (HUM)
Pedagogy In-class Problem
Solving Project-Based
Learning Self-/Co-regulated
Discussion
Flipping Approach Flipping in-class
lectures and quiz Flipping in-class
lectures and extended
collaboration
Flipping the role of
Instructor and
Students
In-Class Activities Problem solving in
small groups Discussion of group
projects for assigned
time in classroom
The small group
discussion without the
presence of the
Instructor; recording a
discussion
Out-of-Class
Activities Online video lecture;
Quiz; comments on
the videos
Small group project
via technologies Instructor ‘s review of
group discussions

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Technology Used YouTube, LMS YouTube, LMS,
Google Docs Google Hangout,
Video Cam, Dropbox
Each participating class was carefully selected and a mentor-protégé relationship
established amongst the participating instructors whereby seasoned ‘flipped classroom’
instructors were paired with the instructors who were newcomers to ‘flipping a class.’ In addition
to mentorship, a learning technology service unit provided technology and instructional design
support. Participating instructors had brief consultations with their respective support person(s),
followed by nine project meetings as well as frequent email communications and some telephone
support through the semester.
The three participating instructors designed their flipped classroom events in light of their
individual contexts and purposes for flipping classrooms, which resulted in widely different
forms of flipping across three disciplines. Table 2 describes the dimensions of the flipped
classroom approach with accompanying details for each classroom implementation.
2.1 Engineering Class (ENG): Maintaining the Essence of a Flipped Classroom
ENG was an undergraduate biomedical-engineering course with lectures and a weekly
two-hour lab session for the application of concepts and principles that focused learning on the
theme of ‘conservation of energy’ through problem-solving methodologies and activities.
Students viewed videos outside of class on YouTube, discussing the course focus on
‘conservation of charge.’ Student learning of content in the video lectures was monitored through
a combination of short quizzes (3-5 multiple choice questions) created on the Blackboard
learning management system (LMS), and low-stakes points students received for participation on
respective YouTube discussion boards. During class, the instructor presented a problem relating
to, but not exclusive to, the video(s) viewed outside of class. Groups self-assembled among the

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
52 class members to solve problems during class while the instructor and two TA’s provided
facilitation. The instructor applied his prior experience with flipping a class whereby both the
out-of-class and in-class experiences were improved by creating a clearly defined contextual link
between the two. For example, the instructor used select student comments as a springboard for
the next lecture and had students answer each other’s questions in groups, based on the questions
they posted on YouTube.
2.2 Sociology Class (SOC): Building a Project-Based Learning Environment
For part of this course, the SOC instructor assigned students to group research projects as
an out-of-class assignment and devoted some class time to making progress on those projects
during a segment of the course. The flipped classroom events aimed to help students learn how
to construct and refine a testable research question over the course of several weeks of discussion;
how to collaborate with peers; and how to research across an array of data sources that address
the testable research question. The instructor assigned the students to view a short video that
described a specific set of activities designed to help students formulate a valid and measurable
research question. Primary technologies used were the Blackboard LMS for discussion, with
each group being required to post their key findings and remaining questions, and with at least
one group member required to respond to another group; and YouTube for video postings.
Students worked in groups to create an 8-10 minute presentation that they shared with the entire
class, and that served as the content for a 5-7 page group paper. Students also used Google Docs
for collaboration. All group members shared a project grade. Within each group, students
individually evaluated their peers on the levels of their unique contributions to the effort.
2.3 Humanities Class (HUM): Experimenting Radical Student-Directed Learning

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
The HUM instructor taught 13 students in a seminar twice weekly. The instructor sought
to create course continuity during three weeks when he was lecturing at other universities. For a
period of three weeks, the students were given three different assignments designed to prepare
three respective student groups for a set of student-directed in-class activities conducted without
the instructor present. The instructor required students to record their discussions or a summary
of their group discussion using some kind of video technology, and later viewed the groups’ self-
reported in-class collaboration. The recording was expected to work as a tool that helped the
instructor monitor student discussions remotely, while promoting student engagement in a
meaningful, albeit non-instructor facilitated dialogue.
Discussion prompts were used to help facilitate discussion within each small
group. Flipped events permitted students to (a) figure out how to lead the process in the
instructor’s absence and who should lead it and (b) edit spontaneous live discussion into an
interstitial artifact: more edited than classroom conversation, less edited than a finished essay.
Students created simple video-captures that they rendered in YouTube. Students were not
expected to edit using software such as iMovie, motivated by the idea that developing video
production skills was not a chief learning goal of this process. The YouTube platform enabled
the conversation to be shared with the instructor.
The students conducted small group conversations off camera. The flipped element was
to redact conversation for the camera in such a way that it would yield an edited artifact. The
mentor and a supporting staff member, who was an instructional designer, supplied students with
a variety of role-playing scenarios that would aid them in thinking about how to move the free-
form conversation into an edited thought-piece.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
However, in practice, students almost uniformly found the role play and redaction to be
inauthentic and laborious. Students were leery of ‘performing’ to the physically invasive camera,
which due to its presence and the need to manage it, forced disruptions and interruptions during
conversations. The goal of the HUM class was to have students develop an ‘interpretive build’
on a topic via their active in-class conversation. For students who already perceive the
humanities classroom as student-centered, this flipped experience might be perceived as an
authentic self-regulated conversation accompanied by the extra work of creating a video
recording of inauthentic discussions.
3. Research Methods
We conducted a mixed methods study to capitalize on the strengths of both quantitative
and qualitative methods (Greene, 2007). The Revised Community of Inquiry (COI) framework
was used as an underlying factor model to evaluate the levels of the implementation of three
flipped classrooms for both quantitative and qualitative analysis. We first analyzed quantitative
data to portray overall features of the flipped classrooms and then extracted design principles
from multiple qualitative data sources.
3.1 Participants
Three instructors from different disciplines−engineering (ENG), social studies (SOC),
and humanities (HUM) were selected on the basis of their commitment to the flipped classroom
concept, their disciplinary focus in light of the other disciplines, and past experience using
technology to support their online instruction. 115 students were enrolled in the three classes
during the fall 2012 semester. A total of 41 students responded, with a response rate of 36%.
Twice as many female students as male students participated in the study (see Table 3). The
majority of the students in the ENG course were in their first year, while many students in SOC

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
were in their third year or more. For the HUM, the response rate was 26%, but the number of
responses was too small to analyze (only 4 students). With respect to prior courses taken, most
students in the ENG had no previous experience with related courses, while over half of SOC
students had taken course(s) similar to the SOC course.
Table 3
The Demographics of the Participants
Total ENG SOC HUM
Total Enrolled 115 52 48 15
Responded 41 (36%) 13 (25%) 24 (50%) 4 (26%)
Gender Female 29 9 18 2
Male 11 4 6 1
Class Year Freshman 13 12 1 0
Sophomore 8 0 7 1
Junior 11 1 10 0
Senior 9 0 6 3
Graduate 0 0 0 0
Course Taken Yes 13 2 11 0
No 28 11 13 4
3.2 Data Collection
We collected data from diverse sources including surveys, interviews, instructor
reflections, and documents (e.g., meeting minutes, course syllabi and student outcomes).
3.2.1 Student Survey
The student survey consisted of four sections: (a) Teaching Orientation; (b) Revised
Community of Inquiry (RCOI); (c) Technology Use; and (d) open-ended questions. The
Teaching Orientation measure asked students whether the classroom culture is more teacher-
oriented or student-oriented. The Revised Community of Inquiry (RCOI) measure consisted of
four sub-scales: (a) The Teaching Presence, (b) Social Presence, (c) Cognitive Presence, and

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
(d) Learner Presence. These measures were deemed to account for the specific features of
the Teaching Orientation for the flipped classrooms. The Technology Use measure was also
included to indicate the extent to which students felt easy and comfortable when using
technologies, a key factor considering that the flipped classrooms employed a variety of online
learning technologies. The total number of survey items was 50: seven items for orientation
toward student-centered learning (Teaching Orientation); eight items for each of four RCOI
subdomains; five items for the use of technology; and six open-ended questions. For 46 items, a
four-level Likert scale was used, ranging from ‘Strongly Disagree,’ ‘Disagree,’ ‘Agree,’ to
‘Strongly Agree.’
Seven items for Teaching Orientation were created on the basis of the Section IV of the
Classroom Lesson Observation (CLO) survey of CITERA (Comprehensive Information
Technology Education in Rural Appalachia, funded by the National Science Foundation;
http://www.theedventuregroup.org/citerawv/). Section IV of the CLO questioned teachers about
the impact of technology integration on classroom teaching in middle and high school contexts
(Darrah & Blake, 2009). For this study two higher education professionals helped design the
items for students within a higher education context.
We used a bipolar scale to rate the design and implementation of flipped classroom
activities that ranged from teacher-centered (highly structured, directed learning) to student-
centered (mostly unstructured, open-ended learning) (see Figure 2). For example, given two
polarized descriptions of the classroom (On the one end “Classroom activities focused on
knowledge abstraction and acquisition,” and on the other end “Classroom activities focused on
making real-world connections”), students selected the button nearest the classroom environment

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
that most closely resembled their experiences. For seven items, the reliability score was 0.72
overall (see Table 5).
Figure 2. Items for the teaching orientation.
The second part of the survey was composed of the Revised Community of Inquiry
(RCOI) measure. To measure the three sub-domains (Teaching, Social, and Cognitive Presence),
we used items that were selected from the Community of Inquiry instrument (Shea & Bidjerano,
2010; Swan et al., 2012) according to the study context (i.e., flipped classrooms as a type of
blended learning). Reliability scores were 0.87, 0.87, and 0.88 respectively (see Table 5). For the
Learner Presence, previous studies introduced the concept of learner presence and developed a
coding scheme for the use of content analysis, but did not develop questions for the construct
(Shea et al., 2012). For measures of the Learner Presence, eight items were adopted from two
instruments: four items from the Self-Regulation section of the Motivated Strategies for Learning
Questionnaire (Pintrich & Groot, 1990) and the other four items from the Metacognitive Self-

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Regulation (Artino, 2005), resulting in an internal reliability of 0.83 (for the items, see Appendix
A; for internal reliability, see Table 5).
Five items about the Use of Technology composed the third part of the survey.
For example, a question was “In general, I felt confident using the technologies associated with
the out-of-class activities.” However, student responses to the items were inconsistent, resulting
in a low reliability of 0.49 (see Table 5). Due to this insufficient reliability, we interpreted the
results not as a construct value, but rather as individual item values.
3.2.2 Student Interview
In addition to students’ responses to the open-ended questions in the survey, student
interviews were implemented to gain deeper understanding, using a semi-structured interview
protocol in a consistent manner within a 30 minute interview for each individual (see Appendix
C). In order to obtain a wider range of understanding, two groups of students whose levels of
participation and performance were positioned at opposing ends of the spectrum were selected.
As a result, two students in the ENG class were interviewed. We planned to interview paired
students from each of three classes, but there were no student volunteers from the other two
classes due to their final examination schedules. Considering the small number of interviewees,
we decided to limit the use of the interview data when interpreting the flipped events in the ENG
class.
3.2.3 Instructor Reflection
In order to understand the evolving status of the flipped classrooms, at the beginning of
the semester we collected each of the instructor’s syllabi. Notes were created for nine meetings
with the professors. Stored data including the syllabus was gathered via the repositories. After
conducting their final exams, the instructors submitted their reflections on the semester-long

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
flipped classroom experiment according to a guiding protocol. The protocol focused on three
areas: (a) how flipped classroom events were defined by specific instances; (b) how the
instructors implemented the designed events; and (c) how their students engaged in the events
(see Appendix D).
3.3 Data Analysis
The use of several types of data from multiple sources serves as validation and reliability
of the research findings, a method that is often referred to as triangulation and saturation. First,
we investigated the student perceptions of the value of the flipped classrooms and the relations of
six factors (Teaching Orientation, Four RCOI factors, and Technology Use), drawing on
quantitative data. Descriptive statistics and correlation-coefficient analysis methods were applied
for analysis using SPSS 21. The results provided us with overall trends in the three flipped
classrooms.
Subsequently a qualitative data review was conducted to identify cases that describe
participant experiences from which it was possible to draw meaningful guidance for the design
of the flipped classrooms. As described in Table 4, according to the ROIC framework, we
investigated qualitative data collected from multiple sources (i.e., documents, interviews, and
reflections) and sorted instances of positive or negative comments into each sub-category. Two
reviewers discussed the coding scheme and had two coding practices to be in closer alignment
with the sorting. One reviewer investigated the data and organized the identified instances, and
then the other reviewer reviewed the instances sorted by the first reviewer. For conflicting views
about a given instance, the two reviewers discussed it until they reached an agreement about how
it should be categorized. Nvivo 10 was used for the qualitative analysis.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Table 4
Coding Examples
Type
of Presence
Cod
e
Example
Cogni
tive Presence
Posit
ive
I would look up information if I didn't originally
understand what he was lecturing on YouTube.
Neg
ative
The lectures provided no substance and lacked
enthusiasm. They were very hard to follow.
Social
Presence
Posit
ive
They were valuable because they allowed me to network
with people in the class. I am a new student at USC so this
allowed me to meet people and network. It was also an interesting
methodology and approach to class activities.
Neg
ative
The initial peer interactions could have been facilitated so
that there were icebreakers.
Teach
ing Presence
Posit
ive
The instructor put up a PowerPoint during class and then
facilitated classroom discussions.
Neg
ative
The project would be more clearly outline at the beginning
of the class, so we understood the steps we should be taken / the
paper should have more clear, different sections
Learn
er Presence
Posit
ive
We coordinated our meetings using text messaging and
used Google PowerPoint to collaboratively contribute information
to everyone.
Neg I didn't really enjoy this project because I felt like my

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
ative teammates and I weren't exactly on the same page, but it was very
difficult to all get us on the same level.
4. Student Perception: The Value of the Flipped Classroom
Overall students were satisfied with the flipped classroom activities, with many agreeing
that the class time interaction was helpful to their understanding of course concepts. Students
perceived that the flipped classroom activities were more student-oriented than traditional class
activities (mean=3.7 out of 4) (see Table 5). The four elements of the RCOI illustrated the
specific values of student-centered experience in the flipped classrooms (over 2.9 out of 4):
Teaching Presence (3.18); Social Presence (3.08); Cognitive Presence (2.94); and Learner
Presence (2.90).
Table 5
Descriptive Statistics
N Mean Std. Alpha
T E S T E S T E S T E S
Teaching
Orientation
41 13 22 3.67 3.23 3.81 5.2 5.2 4.9 0.72 0.67 0.73
Teaching presence
40 13 22
3.18
3.21
3.20
4.0 3.7 4.6 0.88 0.78 0.93
Social Presence
40 13 22
3.08
3.03
3.11
4.0 4.1 3.9 0.87 0.88 0.88
Cognitive
Presence
40 13 22 2.94 2.85 2.99 4.1 4.1 4.3 0.87 0.81 0.92
Learner Presence
40 13 22
2.90
2.84
2.98
3.9 3.9 3.8 0.83 0.83 0.82
Technology Use
40 13 22
3.04
2.90
3.16
2.2 2.2 1.6 0.49 0.41 0.36
Note: T (Total), E (Engineering class), S (Social Studies class)
First, Teaching Presence had the highest level of satisfaction (3.18 out of 4). Students
responded that their instructors appropriately structured the assignments and provided
information about learning goals, due dates, and time frames for activities. Students universally

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
sought clearer instructions on the details of how to participate in the learner-centered activities,
and sought immediate feedback to verify their participation and performance (see Appendix B).
Second, Social Presence posed a value of 3.08. The flipped classroom assignments (e.g.,
in-class group problem solving and hybrid group projects through on/off-line collaboration)
promoted students’ positive affect, interaction, and cohesion. In particular, students felt that the
class environments were very open. Students perceived that their contributions were
acknowledged by other participants (93% positive response to the items S7; see Appendix B). In
contrast, students felt the need to improve upon online discussion strategies toward helping them
engage more in collaboration with peers (56% positive response to the items S8).
Third, Cognitive Presence denotes the extent to which the assignments challenged
students to utilize their higher-order thinking skills (e.g., evaluation, problem solving, and
critical thinking) and to apply what they learned into the broader areas (2.94 out of 4). Students
perceived that the assignments challenged them to construct solutions (85% positive response to
the item C6). However, relatively, they wanted to be more motivated to explore problems,
questioning themselves about diverse issues associated with the contents (see items C2 and C3).
Lastly, the perceived level of the Learner Presence was 2.9. Overall the flipped classroom
assignments helped students to regulate learning by self or by peers in terms of goal setting,
monitoring their progression, and evaluating their own achievements. Students were confident
about confirming whether they understood a concept (90% positive response to item L7), while
they had a need for advancing their ability to evaluate and control their knowledge and skills in
complex problem solving (see items L2 and L3). In addition, they thought appropriate
technologies were adopted to support the flipped classroom assignments (3.0 out of 4).

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
This study investigated the impacts of the flipped classroom experience with the merged
data from three classes. In order to use the merged data to produce general discussions, it was
necessary to support the argument that the flipped classroom experience across three classes was
similar in terms of RCOI factors. As Table 6 describes, we compared ENG and SOC class for the
key factors. The HUM class was not included because there were not enough student responses
(only four responses out of fifteen). Amongst six factors, only the Teaching Orientation had a
significant difference between the ENG and SOC class. The student perception of the levels of
student-centeredness was higher in the SOC class than ENG class (t (33)=-2.362, p< .05). The
result seemed partly affected by the types of the assignments. The SOC class assigned research-
oriented projects in groups that expand student collaboration into online environments, while
student interaction in the ENG was limited to in-class problem-solving activities. Considering
other factors made no difference, we decided that it is safe to merge data for further review.
Table 6
Independent t-test for Two Classes (ENG and SOC)
EN
G
SO
C
t-
value df p
Teaching Orientation 3.23
3.81
-
2.362 33
0.02
4*
Teaching presence 3.21
3.20
0.
068 33
0.94
7
Social Presence 3.03
3.11
-
0.452 33
0.65
4
Cognitive Presence 2.85
2.99
- 33
0.44

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
0.778 2
Learner Presence 2.84
2.98
-
0.808 33
0.42
5
Technology Use 2.90
3.16
-
1.940 33
0.06
1
* P < 0.05
Table 7
Correlations
1
2
3
4
5
6
1.Teaching Orientation 1
0.3
70*
0.
278
0.5
12**
0.3
85*
0.2
19
2.Teaching Presence 1
0.
609**
0.6
48**
0.4
03*
0.2
49
3.Social Presence 1
0.5
35**
0.4
94**
0.3
28*
4.Cognitive Presence
1
0.6
01**
0.3
28*
5.Learner Presence
1
0.4
12**
6.Technology Use
1
Note. *. P < 0.05. **. P < 0.01.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
To examine the associations among the factors, correlation-coefficient analyses were
conducted (see Table 7). Admittedly, the correlation values of the four factors (i.e., Teaching,
Social, Cognitive, and Learner Presence) of the Revised Community of Inquiry Model were
significantly high, ranging from 0.403 to 0.601. The expanded associations with the Teaching
Orientation also showed significant relations except for the Social Presence, ranging from r =
0.370 to r = 0.512. Interestingly, in contrast to what common sense might dictate, the student-
oriented learning culture had no significant positive relation with students’ affect and cohesion
(association between Teaching Orientation and Social Presence, r = 0.278).
Table 8
Number of Elements Identified for the Five Factors
ENG
n=13 SOC
n=24 HUM
n=4 Sum
n=41
Total All 78 (3.0) 66 (2.8) 7 (1.8) 112 (2.7)
Negative 21 (1.6) 41 (1.7) 4 (1.0) 66 (1.6)
Positive 18 (1.4) 25 (1.0) 3 (0.8) 46 (1.1)
Cognitive Presence All 4 1 0 5
Negative 1 0 0 1
Positive 3 1 0 4
Learner Presence All 9 13 0 22
Negative 3 8 0 11
Positive 6 5 0 11
Social Presence All 1 9 1 11
Negative 0 4 1 5
Positive 1 5 0 6
Teaching Presence All 14 27 4 45
Negative 13 26 2 41
Positive 1 1 2 4
Note. The value in parenthesis is mean. ‘n’ is the number of students who responded.
For content analysis, elements indicating positive or negative views were identified and
counted based on the coding protocol (see Table 4). For the Cognitive Presence, positive
comments were more than negative (see Table 8). For the Learner Presence and Social Presence,
overall there was no big difference between the numbers of positive and negative comments.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
However, in the SOC class negative cases outnumbered positive remarks. For all classes,
students remarked they needed more support and facilitation from the instructors (Teaching
Presence). In addition, there were very few negative comments about technology, and no specific
technologies were mentioned in a negative manner. Only positive comments pointed to specific
supporting technologies (e.g., Google Docs were mentioned by many students as useful to their
collaboration work). Students were savvy about using technology for learning and as discerning
digital consumers, they required better networking, richer multimedia, and more collaboration
through social media.
5. An Emerging Design Framework
Figure 3 describes the selected relations among key variables that were higher
correlations over 0.5 except for the relation between the Teaching Orientation and Social
Presence that was the lowest correlation (r = 0.278). The figure informs us of how the flipped
classroom events worked and where we need to improve in the design of a flipped classroom.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Figure 3. An emerging design framework. Asterisk (*) denotes a significant coefficient-
correlation value.
First, Cognitive Presence is at the center of the relationship with the other
domains. This means that students tend to be better engaged in higher-order thinking such as
complex problem solving when other domains are developed and implemented in an appropriate
manner in support of students’ active learning. The level of the instructors’ facilitation and
support (Teaching Presence) had the highest association (r = 0.648, p < 0.5) followed by the
Learner Presence (the level of self-regulation, r = 0.601, p < 0.5). Admittedly, students seems
better motivated to do problem solving in a condition that teachers provide adaptive guidance
and they have ownership in their learning tightly connected with other students.
Second, Teaching Presence was a critical factor in making the flipped classroom
activities successful. The instructors’ role was changed to acting more like a facilitator and
helper. The significance of the instructor’s active role was never diminished, but became greater
in the flipped, student-centered learning environments. Students' open-ended comments were
largely consistent with the correlation-coefficient scores. Students often had trouble in regulating
themselves and working with other group members during problem-solving activities. Students
asked for more structured guidance and timely support from the instructors. Students believed
that the better the instructors facilitate online discussions, the more they can be engaged in
problem-solving assignments (association with Social Presence). Students wanted to be better
motivated to explore issues related to the course content and given problems (association with
Cognitive Presence).
Third, interestingly, a low correlation between Teaching Orientation and Social
Presence was identified (r = 0.278). Just providing student-centered instructional events was not

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
directly connected with building close relationships among students even though students
reported high levels of perceived value in both Teaching Presence and Social Presence, m = 3.67
and 3.08, respectively (see Table 5). These results imply that instructors need to design
elaborated strategies and do more to facilitate student interaction so that students can better
collaborate and bond with one another.
6. Design Principles of Flipped Classrooms
This study proposes nine design principles for the flipped classroom on the basis of the
design framework that emerged from the data. The first three design principles of this study were
adopted and validated from the design suggestions of Brame (n.d.): Provide an opportunity for
students to gain first exposure prior to class; provide an incentive for students to prepare for class;
provide a mechanism to assess student understanding. The other six principles were developed as
new suggestions for creating flipped events to better foster student-centered learning (see Figure
4).
Figure 4. Nine design principles of the flipped classrooms.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
6.1 Provide an Opportunity for Students to Gain First Exposure Prior to Class
One of the benefits of the flipped classroom is that students are able to prepare for in-
class activities by watching and exploring on-line learning materials (e.g., online video lectures)
outside the classroom according to their own time schedules and levels of understanding (Davies
et al., 2013; Foertsch et al., 2002). This differentiated learning (Keefe, 2007) has been
materialized by the provision of technologies such as webcasting (He, Swenson, & Lents, 2012;
Hill & Nelson, 2011; Holbrook & Dupont, 2011; Kay & Kletskin, 2012; Traphagan, Kucsera, &
Kishi, 2010; Vajoczki et al., 2010). Indeed, students tried to accommodate their understanding of
the given online content and explored different types and levels of resources.
I watched online videos and looked up things on the Internet if I didn't understand;
I would look up information if I didn't originally understand what he was lecturing on
YouTube; In watching the videos, I can pause them, or rewind when I missed something
to need time to take notes (an ENG student).
I used the online library resource and PowerPoint along with the Internet to
complete the inverted classroom project (a SOC student).
6.2 Provide an Incentive for Students to Prepare for Class
Successful face-to-face interaction, in a typical flipped classroom, depends on the extent
to which students have prepared before engaging with the in-class activities. However, in reality,
one of the participating instructors observed that about 25% of the students had not watched
online lectures in his previous experiment. This observation was reinforced by an interview with
a student in his class. The student described:

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Not all students watched videos. Learning depends on self. Without preparation
meant low participation in a group work. Instructor became aware of that and added
activities such as weekly quiz and annotating video lectures. These really worked.
Online discussions (e.g., YouTube comments) with low-stakes grading appear to have
motivated many students to engage the learning activities/assignments and problem solving. The
ENG instructor stated: “To foster more interaction and connection between out-of-class and in-
class activities, I required students to submit questions/comments on YouTube by giving them
points on homework assignments. This drastically boosted the number of questions posted on
YouTube and I used those as a springboard for the next lecture.”
6.3 Provide a Mechanism to Assess Student Understanding
Low-stakes quizzes and other forms of formative assessment appeared to be effective in
both ensuring out-of-classroom activities and helping to prepare students for in-class activities.
Throughout the semester, students generally appreciate having the videos and
having more time to work on problems in class. Most students seem to watch the videos
once as I asked them to, but would usually not watch them again for review. Short
quizzes (3-5 multiple choice questions) were created on Blackboard for them to answer.
Questions were intentionally made simple. Most (~85%) students were able to correctly
answer all questions (ENG instructor).
Despite these positive findings, a student’s confession implied that there is still
room for improvement: “When I didn't have enough time to watch the videos I just clicked on
the link and then made up a question/comment so I got points on my homework.”
6.4 Provide Clear Connections Between In-Class and Out-of-Class Activities

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Online content and activities should directly support or connect with the associated in-
class activities. Many studies have warned that the lack of cohesive alignment of face-to-face
and online portions of blended learning classes can distract students from engaging in given
activities (Buerck et al. 2003; Elen & Clarebout 2001). Strayer (2012) observed even very active
and hard-working students often found it difficult to map online assignments onto in-class
activities. Considering flipped classrooms are a form of blended learning, it is crucial to tie
online and in-class activities clearly and cohesively so that students can successfully achieve
learning goals (Ginns & Ellis 2007; Strayer, 2012). The ENG instructor demonstrated this good
practice:
To foster even more interaction during the last lecture period in this module, I still
required students to post questions on YouTube, but in class I had students self-assemble
into groups and had them answer each other’s questions that were posed on YouTube.
Groups were then asked to post their answer back onto YouTube in a reply. I met with
each group for about 5 minutes to help answer any questions that the group could not.
6.5 Provide Clearly Defined and Well-Structured Guidance
Students require clearly defined and well-structured guidance and scaffolding for flipped
classroom activities. At the beginning, students may be resistant to a new method that requires
them to be more engaged in the out-of-class assignments to prepare for in-class problem solving.
Strayer (2012) stated that it is still beneficial to employ the task-oriented environment supported
by a well-designed structure that clearly guides students to solve given problems. Evidently, a
clear course structure with supporting tools such as guiding prompts and instructions must be
designed to help students prepare for participation and then success in achieving learning goals.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
With a clear set of instructions and a structure, instructors and TAs can also facilitate in-
class activities to help reinforce the connections between the in-class activities and the out-of-
class activities.
For example, an ENG student claimed “There should have been more specific goals for
each day of in-class work to focus the groups' discussions.” A more serious issue was found in a
SOC student’s comments:
Even though the concept of the "inverted classroom" project and the deadlines for
the project and the fact that we were to use class time to work on specific things were
clear, what the actual project should have looked like when we were done was not (a
SOC student).
6.6 Provide Enough Time for Students to Carry Out the Assignments
In-class activities should be designed with adequate time to apply the knowledge,
information, and skills class students acquire online. As Gannod and colleagues (2008) described,
given a finite number of minutes in a classroom meeting, instructors mixed traditional lecture
with some active learning, which often created tension between two portions of the use of
classroom time. Students in one course perceived the need for more in-class time to apply
information and skills they acquired online in preparation for the class and to conduct group
work. The timing of out-of-class activities was considered important by many students, with the
ability to review content online as a major benefit to preparing for in-class activities. The
positive responses relating to the ability to review content in this context indicates a positive
sense of learning self-regulation amongst those students.
As related to the assigned group presentation, giving us at least 4 full class
meetings to meet with our groups and work on our parts would be necessary. Otherwise,

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
there is no point in giving us any time at all in class as the time provided does not allow
for any progress to be made (a SOC student).
Time Constraints - more time outside of lecture is necessary in order for this to be
beneficial. The timing of when the videos were posted was occasionally not at an ideal
time (a ENG student).
6.7 Provide Facilitation for Building a Learning Community
A learning community can provide students with space to take in new ideas by learning
from one another. It is critical for instructors to create learning communities that connect
students and help them collaborate well (Garrison & Kanuka 2004; So & Brush 2008).
Especially since group work continues to be a universal challenge, there should be well-prepared
facilitation and guidance for student collaboration. In-class group work appears to be difficult for
many students (i.e. group dynamics, roles and levels of participation, and satisfaction with
grading schema). Students valued the effect of peer collaboration in groups or as a whole group
while they often encountered the lack of guidance to equip better group work.
The flipped classroom activities were valuable because they allowed me to
network with people in the class. I am a new student at this university so this allowed me
to meet people and network. It was also an interesting methodology and approach to class
activities (a SOC student).
Indeed, the instructors’ role as an initiator and facilitator for building a good community
and collaborative learning culture could not be emphasized more.
A HUM student told: The initial peer interactions could have been facilitated so
that there were icebreakers. Students should be encouraged the share their opinions freely.
This proved to be difficult in the first few interactions regarding the first assignment.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
I didn't really enjoy this project because I felt like my teammates and I weren't
exactly on the same page, but it was very difficult to all get us on the same level. There
was a pretty unequal distribution of work, and the project is set up so that it's really hard
to have equal participation (a SOC student).
6.8 Provide Prompt/Adaptive Feedback on Individual or Group Works
Some students did not consider in-class activities as increasing engagement (i.e., ability
to answer questions raised in course activities, constructing explanations/solutions, motivated to
explore content related to the questions raised). Many students mentioned the need for greater
and more prompt feedback for various reasons including improved group work and/or to connect
the in-class problem-solving activities with the out-of-class preparation. Instructors need to
provide adaptive feedback and instructional supports suited to students’ different needs
(Foreman, 2003). For example, students who tended to be more engaged reported difficulty in
maintaining their engagement for the duration of class time. These students, appear not to need
the same level of highly-structured in-class set of activities and may have been bored at the end
of a problem-solving activity.
During the flipped classroom activities, my role was similar since the other parts
of the course were already partially inverted. However, I felt that my lecturing and my
interaction with the students catered more to their questions/confusion about the material.
I met with each group for about 5 minutes to help answer any questions that the group
could not. Based again on my perception alone, this approach was much more interactive
(ENG instructor).
6.9 Provide Technologies Familiar and Easy to Access

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
For most students there was no technological barrier. They were actually very familiar
with current learning technologies. However, it appeared worthwhile to use familiar and easy to
access technologies and establish acceptable standards for the development and delivery of
online content. For example, a student in the ENG class said “The videos should be shorter and
on a specific topic with more enthusiasm.” In addition, technology must be selected and aligned
with flipped events for the purpose of learning goals. Admittedly, how to integrate technology
into pedagogy is much more important than mere technology use.
The video artifacts would have been better if students had simply captured their
original conversation. I suspected that the presence of the camera triggered a “reality TV”
paradigm that potentially disoriented students: “we’re not all ready for our on-camera
moment.” Particularly for undergraduate learners, who are struggling to master tough
theoretical concepts, the added self-consciousness that the camera seemed to have
imparted was unwelcome (HUM instructor).
7. Conclusion
7.1 Recommendations for Future Research
This study piloted flipped classroom activities across three classes selected from different
disciplines so as to find design guidance within the broader definition of the flipped classroom.
As a result, a design framework and nine design principles of the flipped classroom were
proposed. All of the implementation and design principles offer actionable ways by which
instructors can meet learner needs in the context of the flipped classroom, irrespective of the
specific flipping strategy invoked. In spite of all benefits from this study, there are a few
limitations that require further research.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
First, it is necessary to further investigate and define design specifications that
integrate technology into flipped classrooms. This study used a short survey and open-ended
questions to see how students and instructors used technologies in the flipped classrooms.
However, low internal reliability of the survey items limited the interpretation of the results.
Agreeing with Strayer’s (2012) claim that the systematic use of interactive technology is a key
feature of flipped classrooms, we suggest future studies that more deeply investigate the use of
technology and that employ superior assessment instruments. For example, we can elaborate
ways to integrate technologies with the applications according to the respective nine design
principles.
Second, there is still the need to explore the implementation feasibility of the
flipped classroom approach in large-size classes. This study included undergraduate classes that
are mid-sized or less (i.e., the number of students in the classrooms was less than 50). The heart
of flipped classroom is to engage students in their own learning in the context of collaboration
and frequent interactions amongst individuals, which necessarily demands significant facilitation
and administration on the instructor’s side. Accordingly, the nine principles might not always
work in every classroom instance or may require specific design details applied in different
course contexts that address the varying sizes of classes.
Third, this study primarily relied upon the participants’ perception of their own
experiences in the flipped classrooms to evaluate the quality of the teaching model. However,
this study did not include the participants’ performance (i.e., achievement scores) and changes in
motivation and emotion. Future research should consider contradictory reports regarding the
impact of flipped classrooms on performance, and more research including other dependent
indicators (i.e., non-cognitive domains) is required.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Lastly, evaluation accountability needs to be shared with instructors. We found it very
difficult to obtain sufficient evaluation data from participants. Two classes of the three
participating classes resulted in only a 25% response rate for the student survey, which reduced
average rate down by 36%. Students’ voluntary participation in the interview was much lower
than expected for this study. Shared evaluation ownership could be established by means of
getting instructors involved in evaluation planning where they set their expected benefits from
evaluation results. Professors’ active role in evaluation is critical to produce valuable directions
for implementation and following improvements.
7.2 Closing Thoughts
This study is unique in terms of: (a) applying the broader definition of the flipped
classroom whereby traditional activities and events are exchanged between in-class and online
environments for an improved learning experience; and (b) examining multi-disciplinary
applications of the flipped classroom.
Given the broader definition of the flipped classroom focused on the hybrid or blended
learning model, it becomes obvious that these same principles can be broadly applied to a typical
undergraduate course. Today’s typical undergraduate student uses an online learning
management system (LMS) for most of their courses. Predictably, most instructors today use an
online LMS for teaching courses. Upon examination and not surprisingly, most of the nine
emergent design principles appear also to apply to a typical undergraduate face-to-face course.
While this study is unique as a multi-disciplinary examination of the flipped classroom
together with the broader definition of ‘flipping’ applied, the ability to test discipline-specific
flipping strategies across a larger set of instances and a diverse set of disciplines would help

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
emerge a more robust set of discipline-specific flipping strategies, especially in light of the
relevance to typical undergraduate courses. Having more specific and tested strategies that
support course learning goals while offering the student a higher value within their own learning
experience across online and in-class environments, also guides the institution in identifying
effective and sustainable instructional technologies that support the respective flipping strategies.
Sustainable support of flipping strategies is arguably tantamount to a sustainable support
strategy for the majority of undergraduate courses toward providing a higher value for students
without adding unnecessary complications and workloads for the instructor. More efforts in this
regard would appear to render a significant benefit for the institution, its students, and its
instructors.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
References
Artino, A. R. (2005). Review of the Motivated Strategies for Learning Questionnaire. Online
Submission. Retrieved from
http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED499083
Baker, J. W. (2000, April). The ‘‘classroom flip’’: Using web course management tools to
become a guide by the side. Paper presented at the 11th international conference on
college teaching and learning, Jacksonville, FL.
Bergmann, J., & Sams, A. (2012). Before you flip, consider this. Phi Delta Kappan, 94 (2), 25.
Bergmann, J., Overmyer, J., & Wilie, B. (2012). The flipped class: Myths versus reality. The
Daily Riff. Retrieved 4 June 2013 from http://www.thedailyriff.com/articles/the-flipped-
class-conversation-689.php.
Brame, C. J. (n.d.). Flipping the classroom. Retrieved from The Vanderbilt University Center for
Teaching website: http://cft.vanderbilt.edu/teaching-guides/teaching-activities/flipping-
the-classroom/
Brown, J., & Duguid, P. (1991). Organizational learning and communities of practice: Toward a
unified view of working, learning, and innovation. Organizational Science, 2 (1), 40-57.
Brush, T., & Saye, J. (2000). Implementation and evaluation of a student-centered learning unit:
A case study. Educational Technology Research and Development, 48(3), 79–100.
doi:10.1007/BF02319859
Buerck, J. P., Malmstrom, T., & Peppers, E. (2003). Learning environments and learning styles:
Nontraditional student enrollment and success in an internet-based versus a lecture-based
computer science course. Learning Environments Research, 6, 137–155.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Choi, I., & Lee, K. (2009). Designing and Implementing a Case-Based Learning Environment
for Enhancing Ill-Structured Problem Solving: Classroom Management Problems for
Prospective Teachers. Educational Technology Research and Development, 57(1), 99–
129. doi:10.2307/25619959
Darrah, M., & Blake, A. (2009). Providing real world experience for high school teachers and
students. In I. Gibson, et al. (Eds.), Proceedings of society for information technology &
teacher education international conference 2009 (pp. 3390-3394). Chesapeake, VA:
AACE.
Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom and instructional
technology integration in a college-level information systems spreadsheet course.
Educational Technology Research and Development, 61(4), 563–580.
doi:10.1007/s11423-013-9305-6
Elen, J., & Clarebout, G. (2001). An invasion in the classroom: Influence of an ill-structured
innovation on instructional and epistemological beliefs. Learning Environments Research,
4, 87–105.
Ericsson, K. A. (2005). Recent advances in expertise research: A commentary on the
contributions to the special issue. Applied Cognitive Psychology, 19, 233-241.
Foertsch, J., Moses, G., Strikwerda, J., & Litzkow, M. (2002). Reversing the Lecture/Homework
Paradigm Using eTEACH® Web-based Streaming Video Software. Journal of
Engineering Education, 91(3), 267–274. doi:10.1002/j.2168-9830.2002.tb00703.x
Forman, J. (2003). Next-generation educational technology versus the lecture. Educause Review,
35(5), 12-22.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Fulton, K. (2012). Upside Down and Inside Out: Flip Your Classroom to Improve Student
Learning. Learning & Leading with Technology, 39(8), 12–17.
Gannod, G. C., Burge, J. E., & Helmick, M. T. (2008). Using the inverted classroom to teach
software engineering. In ACM/IEEE 30th International Conference on Software
Engineering, 2008. ICSE ’08 (pp. 777–786). doi:10.1145/1368088.1368198
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical Inquiry in a Text-Based
Environment: Computer Conferencing in Higher Education. The Internet and Higher
Education, 2(2–3), 87–105. doi:10.1016/S1096-7516(00)00016-6
Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential
in higher education. Internet and Higher Education, 7(2), 95–105.
Gerstein, J. (2012, May). Flipped classroom: The full picture for higher education. Retrieved
from http://usergeneratededucation.wordpress.com/2012/05/15/flipped-classroom-the-
full-picture-for-higher-education
Ginns, P., & Ellis, R. (2007). Quality in blended learning: Exploring the relationships between
on-line and face-to-face teaching and learning. Internet and Higher Education, 10(1), 53–
64.
Greene, J. C. (2007). Mixed methods in social inquiry (Vol. 9). John Wiley & Sons.
Hannafin, M., Hill, J., & Land, S. (1997). Student-centered learning and interactive multimedia:
Status, issues, and implication. Contemporary Education, 68(2), 94-99.
He, Y., Swenson, S., & Lents, N. (2012). Online Video Tutorials Increase Learning of Difficult
Concepts in an Undergraduate Analytical Chemistry Course. Journal of Chemical
Education, 89(9), 1128–1132. doi:10.1021/ed200685p

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Herreid, C., & Schiller, N. (2013). Case studies and the flipped classroom. Journal of College
Science Teaching, 42(5), 62.
Hill, J. L., & Nelson, A. (2011). New technology, new pedagogy? Employing video podcasts in
learning and teaching about exotic ecosystems. Environmental Education Research,
17(3), 393–408. doi:10.1080/13504622.2010.545873
Holbrook, J., & Dupont, C. (2011). Making the Decision to Provide Enhanced Podcasts to Post-
Secondary Science Students. Journal of Science Education and Technology, 20(3), 233–
245. doi:10.1007/s10956-010-9248-1
Hughes, H. (2012). Introduction to flipping the college classroom. In T. Amiel & B. Wilson
(Eds.), Proceedings from world conference on educational multimedia, hypermedia and
telecommunications 2012 (pp. 2434–2438). Chesapeake: AACE.
Kay, R., & Kletskin, I. (2012). Evaluating the use of problem-based video podcasts to teach
mathematics in higher education. Computers & Education, 59(2), 619–627.
doi:10.1016/j.compedu.2012.03.007
Keefe, J. (2007). What is personalization? Phi Delta Kappan, 89(3), 217–223.
Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the Classroom: A Gateway to Creating
an Inclusive Learning Environment. The Journal of Economic Education, 31(1), 30–43.
doi:10.1080/00220480009596759
Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.
Cambridge: Cambridge University Press.
Means, B. (1994). Introduction: Using technology to advance educational goals. In B. Means
(Ed.), Technology and education reform: The reality behind the promise (pp. 1-21). San
Francisco: Jossey-Bass.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components
of classroom academic performance. Journal of Educational Psychology, 82, 33–40.
Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in social context. Oxford
University Press.
Shea, P., & Bidjerano, T. (2010). Learning presence: Towards a theory of self-efficacy, self-
regulation, and the development of a communitie of inquiry in online and blended
learning environments. Computers in Education, 55, 1721–1731.
doi:http://dx.doi.org/10.1016/j.compedu.2010.07.017
Shea, P., Hayes, S., Smith, S. U., Vickers, J., Bidjerano, T., Pickett, A., … Jian, S. (2012).
Learning presence: Additional research on a new conceptual element within the
Community of Inquiry (CoI) framework. The Internet and Higher Education, 15(2), 89–
95. doi:10.1016/j.iheduc.2011.08.002
So, H.-J., & Brush, T. A. (2008). Student perceptions of collaborative learning, social presence
and satisfaction in a blended learning environment: Relationships and critical factors.
Computers & Education, 51, 318–336.
Spiro, R. J., Feltovich, P. J., & Coulson, R. L. (1996). Two epistemic world-views: Prefigurative
schemas and learning in complex domains. Applied Cognitive Psychology, 10, s51-s61.
Strayer, J. F. (2012). How learning in an inverted classroom influences cooperation, innovation
and task orientation. Learning Environments Research, 15(2), 171–193.
doi:10.1007/s10984-012-9108-4
Swan, K., Matthews, D., Bogle, L., Boles, E., & Day, S. (2012). Linking online course design
and implementation to learning outcomes: A design experiment. The Internet and Higher
Education, 15(2), 81–88. doi:10.1016/j.iheduc.2011.07.002

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Talbert, R. (2012). Inverted classroom. Colleagues, 9(1), Article 7.
Tomlinson, C. (2003). Fulfilling the promise of the differentiated classroom: Strategies and tools
for responsive teaching. Alexandria, VA: Association for Supervision and Curriculum
Development.
Traphagan, T., Kucsera, J. V., & Kishi, K. (2010). Impact of class lecture webcasting on
attendance and learning. Educational Technology Research and Development, 58(1), 19–
37. doi:10.1007/s11423-009-9128-7
Vajoczki, S., Watt, S., Marquis, N., & Holshausen, K. (20100800). Podcasts: Are they an
effective tool to enhance student learning? A Case Study. Journal of Educational
Multimedia and Hypermedia, 19(3), 349–362.
Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge:
Cambridge University Press.
Zappe, S., Leicht, R., Messner, J., Litzinger, T., & Lee, H. (2009). “Flipping” the classroom to
explore active learning in a large undergraduate course. Proceedings of the 2009
American Society for Engineering Education Annual Conference and Exhibition.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Appendix A
STUDENT SURVEY
Select the radio buttons nearest the descriptions that most closely characterize your in-class activities and
your instructor's role. Review the above description of your specific class to recall the "inverted classroom"
activities to which these items refer.
You look for a correct answer.
☐
☐
☐
☐
☐
You accept or revise your hypotheses
based on evidence.
You did not reflect on the comments and
ideas of peers and the instructor.
☐
☐
☐
☐
☐
You reflected on the comments and
ideas of peers and the instructor.
You seek information to complete the
assigned work.
☐
☐
☐
☐
☐
You seek clarification for conceptual
understanding.
Teacher’s role was to provide knowledge.
☐
☐
☐
☐
☐
Teacher’s role was to facilitate
activities.
Teacher asked you to answer questions,
make comments, use memory and facts.
☐
☐
☐
☐
☐
Teacher asked you to answer questions,
make comments, demonstrate
comprehension and give opinions.
Classroom activities were structured and
sequential.
☐
☐
☐
☐
☐
Classroom activities were experience
and discovery-based.
Classroom activities focused on
knowledge abstraction and acquisition.
☐
☐
☐
☐
☐
Classroom activities focused on real
world connections.
The following statements relate to your perceptions of the inverted classroom activities. Please indicate
your agreement or disagreement with each statement
[Teaching Presence]
1.
The instructor clearly communicated important activity goals.
2.
The instructor provided clear instructions on how to participate in the learning activities.
3.
The instructor clearly communicated important due dates/time frames for the learning activities.
4.
The instructor helped to keep course participants engaged and participating in productive dialogue.
5.
The instructor encouraged course participants to explore new concepts in the activities.
6.
Instructor actions reinforced the development of a sense of community among course participants.
7.
My instructor provided useful illustrations that helped me make the course content more understandable
to me.
8.
My instructor provided clarifying explanations or other feedback that allowed me to better carry out the
activities.
[Social Presence]
9.
Getting to know other course participants gave me a sense of belonging in the course.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
10.
I was able to form distinct impressions of some course participants.
11.
I felt comfortable conversing through the online medium.
12.
I felt comfortable participating in the course discussions.
13.
I felt comfortable interacting with other course participants.
14.
I felt comfortable disagreeing with other course participants while still maintaining a sense of trust.
15.
I felt that my point of view was acknowledged by other participants.
16.
(Online) discussions helped me to develop a sense of collaboration.
[Cognitive Presence]
17.
Problems posed increased my interest in course issues.
18.
I felt motivated to explore content related questions.
19.
I utilized a variety of information sources to explore problems posed in this course.
20.
Brainstorming and finding relevant information helped me resolve content related questions.
21.
Combining new information helped me answer questions raised in course activities.
22.
Learning activities helped me construct explanations/solutions.
23.
Reflection on course content and discussions helped me understand fundamental concepts in this class.
24.
I can apply the knowledge created in this course to my work or other non-class related activities.
[Learner Presence]
25.
When I studied for the activities, I set goals for myself in order to direct my activities in each study period.
26.
I asked myself questions to make sure I know the assigned activities I have been worked on.
27.
I tried to change the way I studied in order to fit the activity requirements and the instructor’s teaching
style.
28.
I worked hard to get a good grade even when I was not interested in some topics.
29.
I tried to think through a topic and decide what I am supposed to learn from it rather than just reading
materials or following directions.
30.
Before I began studying I thought about the things I will need to do to learn.
31.
When studying for the activities I tried to determine which concepts I didn’t understand well.
32.
When I was working on learning activities I stopped once in a while and went over what I have done.
33.
In general, I felt confident using the technologies associated with the out-of-class activities.
[Technology Use]
34.
It was easy for me to find and access the out-of-class materials associated with inverted classroom activities
in Blackboard.
35.
In general, technologies associated with the out-of-class activities were easy to use.
36.
The technologies used for the out-of-class activities interfered with my ability to learn.
37.
The technologies used for the out-of-class activities enabled me to collaborate with other students
Please surmise your semester-long experience of your Inverted Classroom experience and answer the
following questions:
33. Describe how you would change the "Inverted Classroom" activities to be more valuable to you.
34. Describe how you used technology to support your work on the "Inverted Classroom" activities.
35. Describe if you were nervous about your ability to use the technology. If you were nervous, did your
anxiety lessen as you worked with the tools, stay the same, or increase?
36. Describe any technologies that might improve the "Inverted Classroom" activities.
37. Provide any additional comments about your experience with “Inverted Classroom” course activities.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Appendix B
Student Survey Responses
Students’ Perception of the Characteristics of In-Class Activities and the Instructor’s
Role
Traditional/Structured Problem-Solving/Inquiry
You look for a correct answer. 3 6 9 12 11 You accept or revise your
hypotheses based on evidence.
7% 15% 22% 29% 27%
You did not reflect on the
comments and ideas of peers
and the instructor.
0 2 6 10 23 You reflected on the comments and
ideas of peers and the instructor. 0% 5% 15% 24% 56%
You seek information to
complete the assigned work. 3 6 7 11 14 You seek clarification for
conceptual understanding. 7% 15% 17% 27% 34%
The instructor’s role was to
provide knowledge. 5 5 10 10 11 The instructor’s role was to
facilitate activities. 12% 12% 24% 24% 27%
The instructor asked you to
answer questions, make
comments, use memory and
facts.
2 8 9 8 14 The instructor asked you to answer
questions, make comments,
demonstrate comprehension and
give opinions.
5% 20% 22% 11% 34%
Classroom activities were
structured and sequential. 3 6 14 11 7 Classroom activities were
experience and discovery based. 7% 15% 34% 27% 17%
Classroom activities focused on
knowledge abstraction and
acquisition.
3 1 10 12 15 Classroom activities focused on
making real world connections.
7% 2% 24% 29% 37%
Students’ Perception of the Inverted Classroom Activities
Items Strongly
Disagree Somewhat
Disagree Somewhat
Agree Strongly
Agree
The following statements related to your perception of “Teaching Presence” – the design of and facilitation of the
activities.
T1. My instructor clearly communicated the
goals of course activities. # 0 3 30 10
% 0% 7% 70% 23%
T2. My instructor provided clear instructions
on how to participate in course activities. 1 8 20 14
2% 19% 47% 33%
T3. My instructor clearly communicated
important due dates/time frames for course
activities.
0 1 23 19
0% 2% 53% 44%
T4. My instructor kept course participants
engaged in productive dialogue. 0 7 25 11
0% 16% 58% 26%
T5. The instructor encouraged course
participants to explore new concepts through
course activities.
0 6 20 17
0% 14% 47% 40%
T6. My instructor provided illustrations that 0 7 21 15

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Items Strongly
Disagree Somewhat
Disagree Somewhat
Agree Strongly
Agree
made the course content more understandable
to me. 0% 16% 49% 35%
T7. My instructor provided clarifying
explanations and feedback that helped me to
better carry out the course activities.
1 9 19 14
2% 21% 44% 33%
T8. My instructor reinforced the development
of a sense of community among course
participants.
0 8 22 13
0% 19% 51% 30%
The following statements related to your perception of “Social Presence” – the degree to which you feel socially
and emotionally connected with others in the activities.
S1. Getting to know some of the course
participants gave me a sense of belonging in
the course.
2 7 22 12
5% 16% 51% 28%
S2. I was able to form distinct impressions of
some course participants. 0 6 24 13
0% 14% 56% 30%
S3. I felt comfortable conversing through the
online medium. 0 9 24 10
0% 21% 56% 23%
S4. I felt comfortable participating in the
course discussions. 0 5 29 9
0% 12% 67% 21%
S5. I felt comfortable interacting with other
course participants. 0 2 26 15
0% 5% 60% 35%
S6. I felt comfortable disagreeing with other
course participants while still maintaining a
sense of trust.
0 4 31 8
0% 9% 72% 19%
S7. I felt that my point of view was
acknowledged by other course participants. 0 3 27 13
0% 7% 63% 30%
S8. (Online) discussions helped me to
develop a sense of collaboration. 6 13 17 7
14% 30% 40% 16%
The following statements related to your perception of “Cognitive Presence” – the extent to which you were able to
develop a good understanding of course topics.
C1.Problems posed increased my interest in
course issues. 1 9 26 6
2% 21% 62% 14%
C2.I felt motivated to explore content related
questions. 1 13 19 9
2% 31% 45% 21%
C3.I utilized a variety of information sources
to explore problems posed in this course. 0 13 20 9
0% 31% 48% 21%
C4.Brainstorming and finding relevant
information helped me resolve content
related questions.
2 8 24 8
5% 19% 57% 19%
C5.Combining new information helped me
answer questions raised in course activities. 0 7 28 7
0% 17% 67% 17%
C6.Learning activities helped me construct
explanations/solutions. 0 7 29 6
0% 17% 69% 14%
C7.Reflection on course content and
discussions helped me understand
fundamental concepts in this class.
0 9 23 10
0% 21% 55% 24%
C8.I can apply the knowledge created in this
course to my work or other non-class related
activities.
1 7 25 9
2% 17% 60% 21%
The following statements related to your perception of “Learner Presence” – the extent to which you were able to
regulate or co-regulate your behaviors in order to achieve learning goals.
L1.When I studied for the course, I set goals 1 12 24 5

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Items Strongly
Disagree Somewhat
Disagree Somewhat
Agree Strongly
Agree
for myself in order to direct my activities in
each study period. 2% 29% 57% 12%
L2.I asked myself questions to make sure I
understood the assigned activities I had
worked on.
2 14 19 7
5% 33% 45% 17%
L3.I tried to change the way I studied in
order to meet the activity requirements and
the instructor’s teaching style.
2 14 21 5
5% 33% 50% 12%
L4.I worked hard to get a good grade even
when I was not interested in some course
topics.
0 5 22 15
0% 12% 52% 36%
L5.I tried to think about a topic and decide
what I was supposed to learn from it rather
than just reading materials or following
directions.
0 7 27 8
0% 17% 64% 19%
L6.Before I began studying I thought about
the things I would need to do in order to
learn.
1 13 21 7
2% 31% 50% 17%
L7.When studying for course activities I tried
to determine which concepts I didn’t
understand well.
1 3 27 11
2% 7% 64% 26%
L8.When I was working on course activities I
stopped once in a while and went over what I
had done.
3 6 28 5
7% 14% 67% 12%

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Appendix C
STUDENT INTERVIEW PROTOCOL
Thank you for taking the time to speak with me today. This interview will take 60
minutes to complete. This interview intends to solicit your thoughts and reactions during your
participation in the Inverted Classroom activities. This interview will be used only for this
purpose and will be kept confidential. The interview will also be digitally recorded and
transcribed. Note that all recordings will be completely discarded once this study finishes.
Please take your time in answering. Feel free to “pass” on any question you don’t care to
answer. Do you have any questions for me before we start?
1.
Would you mind briefly introducing yourself?
a.
Name, grade, and major
2.
Please, tell me about the activity in your class.
a.
What was the activity?
b.
Were there technologies to support the activity?
c.
How did you access and use online lectures if there are?
d.
How was technology-supported communication and collaboration involved in the
activity?
3.
What was your role in doing the activity?
a.
Was this something that you decided to do by yourself? If yes, what did you
determine and how?
b.
How did you look for resources (information) and how did you determine what to
use?
c.
How did you evaluate what you did? What questions did you ask to yourself?
d.
How did you organize your work? How did you think doing that would help you?
4.
What was the role of the teacher when you were working on that activity?
5.
What was the role of your group mates in completing the activity? What were your group
members doing?

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
6.
What specific difficulties do you remember?
a.
What were you trying to do at that moment?
b.
What was unhelpful or redundant/unnecessary?
c.
What suggestions do you have to make the activity better?
These are all the questions I have. I appreciate your cooperation. You’ve shared some
very interesting ideas. Anything else would you like to add about your experience or thoughts of
this activity? Thank you so much for answering all my questions and participating in my study.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Appendix D
INSTRUCTORS REFLECTION PROTOCOL
Describe your Inverted Classroom activities in the following context (questions below
should be incorporated into your answer in some form/order):
a. What were the issues and what changes did you expect to happen (i.e., how were your
learning objectives impacted or changed)?
b. What were the instructional strategies you introduced to your class?
c. What technologies were employed to support implementing the instructional strategies?
How were the technologies integrated with your inverted classroom activities?
Describe how you implemented the designed learning activities(questions below should
be incorporated into your answer in some form/order):
a. How well were the learning objectives achieved or not achieved?
b. In what ways did the learning activities contribute to the learning objectives achieved?
c. What issues did you encounter in the course of the learning activities?
d. What changes did you make to deal with the issues?
e. What was your role in general through the class? What was your specific role during
the Inverted Classroom activities? How did your role shift?
f. How did the specific technologies support or work against the implementation of your
learning activities?
Describe how students engaged in the learning activities:

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
a. How would you characterize students and their attitude toward the learning activities
required in the inverted classroom?
b. If any, what were there notable nonverbal behaviors?
c. What were the special instances related to student participation that surprised/
concerned you?
d. How did students use the technologies?
Are there any areas in which you might modify in the future implementation so as to
strengthen the course?
Please provide additional comments or thoughts about the Inverted Classroom project.
Please describe any future efforts that you might wish to engage relating to this grant that
might involve a publication.
Please describe any future efforts that you might wish to engage relating to this grant that
might involve external funding.

ACCEPTED MANUSCRIPT
ACCEPTED MANUSCRIPT
Highlights
• This study engaged in a mixed research methods that rendered useful data on discipline-
specific flipped classroom applications.
• Building on the Revised Community of Inquiry framework, a design framework for the
flipped classroom emerged.
• This study proposes nine design principles for the flipped classroom on the basis of the
design framework that emerged from the data.