ArticlePDF Available

Active Learning Classroom Observation Tool: A Practical Tool for Classroom Observation and Instructor Reflection in Active Learning Classrooms

Abstract and Figures

Centers for Teaching and Learning (CTLs) have long offered the service of classroom observations to instructors who seek to improve in-class teaching effectiveness. Classroom observations, however, have not explicitly addressed the recent emergence of "active learning classrooms," classrooms that are designed to support active and collaborative learning approaches. Given the absence of an observation protocol explicitly designed to address instructional approaches within active learning classrooms, in spring and fall of 2015, CTL faculty developers and researchers at Indiana University-Bloomington created the Active Learning Classroom Observation Tool (ALCOT). The ALCOT allows a holis-tic consideration of the learning experience, providing a view to the instructor’s attempt to combine the spatial and technological affordances of a classroom with active and collaborative learning pedagogies. Faculty developers and researchers developed and piloted the Active Learning Classroom Observation Tool to elicit thoughtful reflection and meaningful feedback on teaching and learning undertaken within these new learning spaces.
Content may be subject to copyright.
Birdwell, T., Roman, T. A., Hammersmith, L., & Jerolimov, D. (2016). Active learning
classroom observation tool: A practical tool for classroom observation and instructor
reflection in active learning classrooms. Journal on Centers for Teaching and Learning,
8, 28-50.
28
Active Learning Classroom Observation Tool:
A Practical Tool for Classroom Observation and
Instructor Reflection in Active Learning
Classrooms
Tracey Birdwell
Tiffany A. Roman
Leslie Hammersmith
Douglas Jerolimov
Centers for Teaching and Learning (CTLs) have long offered
the service of classroom observations to instructors who seek to
improve in-class teaching effectiveness. Classroom observations,
however, have not explicitly addressed the recent emergence of
"active learning classrooms," classrooms that are designed to
support active and collaborative learning approaches. Given the
absence of an observation protocol explicitly designed to address
instructional approaches within active learning classrooms, in
spring and fall of 2015, CTL faculty developers and researchers at
Indiana University-Bloomington created the Active Learning
Classroom Observation Tool (ALCOT). The ALCOT allows a holis-
tic consideration of the learning experience, providing a view to
the instructor’s attempt to combine the spatial and technological
affordances of a classroom with active and collaborative learning
pedagogies. Faculty developers and researchers developed and
piloted the Active Learning Classroom Observation Tool to elicit
thoughtful reflection and meaningful feedback on teaching and
learning undertaken within these new learning spaces.
Journal on Centers for Teaching and Learning
29
Introduction
Centers for Teaching and Learning (CTLs) have long offered the service
of classroom observations to instructors who seek to improve in-class teach-
ing effectiveness. Classroom observations, however, have not explicitly ad-
dressed the recent emergence of "active learning classrooms," classrooms
that are designed to support active and collaborative learning approaches.
As such, there is a need for a reflective pedagogical observation tool specific
to the context of active learning classrooms, given the rising prevalence of
these classrooms. For example, in the past four years, Indiana University
Bloomington has designed many active learning classrooms. Instructors
who taught in these active learning classrooms increasingly sought out the
university's teaching and learning center for assistance in rethinking their
teaching practices and strategies in these new settings. Instructors requested
teaching observations from faculty developers at IU-Bloomington’s Center
for Innovative Teaching and Learning (CITL) to help them create effective
instruction that would coherently bring together the design features and
affordances of an instructional space with new active learning pedagogies.
The classroom observation is commonly employed at university centers
for teaching and learning to help instructors improve their instruction and
course designs. When a formative assessment is the goal of an observation,
rather than a summative assessment (Chism, 2007), the classroom observa-
tion itself is usually situated between a pre-observation meeting and a post-
observation meeting. The process is characterized by instructor reflection,
feedback from the observer, and an ongoing dialogue between the two about
the observed class and the instructor's teaching goals (Fullerton, 1999).
Implementations of classroom observations are sometimes designed to gen-
erate quantitative data, such as in the well-established Reformed Teaching
Observation Protocol (RTOP; Sawada et al., 2002) and the Classroom Obser-
vation Protocol for Undergraduate STEM (COPUS; Smith, Jones, Gilbert, &
Wieman, 2013). However, the majority of observation approaches tend to
feature structured forms that the observer uses to gather qualitative data,
which are, in turn, used to facilitate further discussion and reflection (Gos-
ling, 2000; Millis, 1992).
Numerous peer observation approaches feature instructor reflection and
feedback from teaching experts, what Gosling has labeled the "development
model" of peer evaluation (2002). When observation protocols call for
instructor reflections, the instructor benefits from engaging in a process of
self-directed professional development for their role as an instructor (Schön,
Journal on Centers for Teaching and Learning
30
1983). The instructor also benefits from a dialogue with, and feedback from,
a teaching expert who can introduce the instructor to different teaching strat-
egies that may better help the instructor facilitate student learning (Hattie &
Timperley, 2007). Other benefits are institutional, such as improvements in
instructional quality, especially when the instructor controls the process of
observation (McMahon, Barrett, & O'Neill, 2007). Although faculty develop-
ers at teaching centers generally offer the service of classroom observations
to improve faculty instruction, few of their classroom observation protocols
explicitly address the relationship of instructional approaches to the spatial
and technological features of a classroom.
ALC Classroom Tools in the Literature
A review of literature, and search of observation tools to use within IU
Bloomington's designated active learning classrooms, revealed that no pub-
lished tools featured classroom space and affordances as an important con-
sideration in the context of teaching. For example, Millis (1992) provides sev-
eral possible observation tools, highlighting different approaches to class-
room observation, but only one mentions a category linked to classroom
space, asking the observer to note classroom inadequacies, such as size and
temperature. Dezure (1999) also considers the classroom environment, but
not as an active support or element of pedagogy, and the classroom techno-
logical affordances are not considered. The ISTE Classroom observation tool
(ICOT) accounts for the use of classroom technologies as an affordance of
ALCs (Bieledfeldt, 2012), but the ICOT does not consider the context of class-
room environment and pedagogy. The Classroom Observation Protocol for
Undergraduate STEM (COPUS) is an instrument employing twenty-five dif-
ferent codes to document classroom behaviors for both students and instruc-
tors in two-minute intervals (Smith et al., 2013) but, while providing quanti-
tative evidence of behaviors, it does not measure the efficacy of the practices
witnessed nor offer a means to provide guidance to the instructor to improve
one's teaching practices. While it should be noted that Millis (1992) and
Dezure (1999) acknowledge that the spatial characteristics and affordances
of classrooms may diminish the effectiveness of certain teaching practices,
neither these existing protocols nor others emphasize the ways in which an
instructor leverages and integrates the design of a classroom and its
affordances in the creation of coherent and effective teaching strategies.
Given the absence of an observation protocol explicitly designed to
address instructional approaches within active learning classrooms, in
Journal on Centers for Teaching and Learning
31
spring and fall of 2015, CTL faculty developers and researchers at Indiana
University created the Active Learning Classroom Observation Tool (AL-
COT). The ALCOT allows a holistic consideration of the learning experience,
providing a view to the instructor’s attempt to combine the spatial and tech-
nological affordances of a classroom with active learning pedagogies. The
intent of the instrument was to support an emerging group of instructors at
the university who sought feedback on their teaching practices within the
new active learning classrooms on campus. Faculty developers and research-
ers developed and piloted the Active Learning Classroom Observation Tool
to elicit thoughtful reflection and meaningful feedback on teaching and
learning undertaken within these new learning spaces.
Designing the Active Learning Classroom Observation Tool
The design of active learning classrooms at Indiana University heavily in-
fluenced the development of the Active Learning Classroom Observation
Tool (ALCOT). Indiana University’s active learning classrooms, known as
Mosaic classrooms, represent a rich variety of spaces designed to meet
widely varying instructional needsmuch like the unique tiles that com-
prise a mosaic. Specifically, Mosaic classrooms are designed differently, each
with particular consideration given to the size of the class, the choice of
teaching approaches, and the variety of disciplines that will teach in them.
All Mosaic classrooms feature sharable screens or whiteboard surfaces
intended for student collaboration and presentation, flexible or fixed furni-
ture that allow natural student grouping, and square footage requirements
that provide students and instructors the space move and engage in a vari-
ety of ways. Thus, given that the instrument would be used in classrooms
with dissimilar design, the ALCOT had to be inclusive of variations of space,
furniture, and technologies.
The Active Learning Classroom Observation Tool was designed primarily
to be used during the classroom observation to guide reflection on the ways
that a given instructor employs the capabilities of the classroom—the room’s
physical arrangements and technologiesin support of teaching and learn-
ing. The faculty developers and researchers at this institution considered
several factors in the process of creating the reflective tool.
First, the ALCOT development team wanted instructors to consider the
intersections of space, technology, and pedagogy (Radcliffe, 2008) as they
reflected on their teaching. Applying pressure, so to speak, on any one of
these intersections changes the way students can be engaged, and in an
Journal on Centers for Teaching and Learning
32
active learning classroom, the results of these interactions are more apparent
than in the context of other learning classrooms employing traditional lec-
ture-style strategies. ALCOT was designed to raise the awareness of the in-
terdependent relationships between classroom design, technology-en-
hanced instruction, and pedagogical strategies supporting active learning.
Next, the ALCOT development team recognized that teaching in spaces
designed for active and collaborative learning is often uncomfortable for
instructors at first due to changes in the expected role of the instructors and
more flexible design in the layouts of many active learning classrooms. The
ALCOT development team wanted to take into account the unique class-
room management issues that active learning classrooms can present.
Depending on the learning space, some of these concerns can include a lack
of focal point and multiple distractions (Petersen & Gorman, 2014). In addi-
tion to the unique concerns of teaching in an active learning classroom, the
ALCOT development team also took into consideration issues of concern of-
ten addressed in general classroom observation, including presentation,
classroom management, learning activities, and instructor-student interac-
tion.
Additionally, the ALCOT development team took a prescriptive approach
to the categories observed, in that observation categories were based on what
should be happening in an active learning classroom. Recent literature sug-
gests that students fair worse academically in an active learning classroom
when instructors lecture instead of engage students in active learning and
collaborative approaches (Brooks, 2012). Based on the above research, the
ALCOT categories for observation encourage instructors who teach in active
learning classrooms to implement instructional approaches that best facili-
tate student learning outcomes through collaboration and active learning
strategies when possible.
The categories that compose the Active Learning Classroom Observation
Tool are prompts intended to elicit descriptive responses to questions about
the instructor's classroom practices (Dezure, 1999; Millis 1992). Descriptive
prompts tend to be more contextual (Evertson & Holley, 1981). The descrip-
tive nature of the responses are a critical characteristic of the ALCOT, given
the importance of the spatial context in this type of observation (Radcliffe,
2008).
Finally, the ALCOT development team sought to streamline the tool by
limiting it to just four categories: (1) support of active learning, (2) creation
and implementation of student collaborative learning activities, (3) forma-
tive assessment in the classroom, and (4) classroom management. By limiting
Journal on Centers for Teaching and Learning
33
the number of categories observed and considered, an observer's attention is
more focused during an observation. Fewer prompts also allow for a more
directed post-observation discussion between a given instructor and faculty
developer.
The Observation Protocol
Most observation protocol designs entail a pre-observation meeting, the
observation itself, and a post-observation meeting, with the emphasis on the
post-observation meeting exchange between instructor and observer (Fuller-
ton, 1999; Hammersley-Fletcher & Orsmond, 2004). The ALCOT observation
protocol conforms to this common structure. A pre-observation meeting is
held between the instructor and the observer to discuss the observation pro-
cess, the background and goals of the instructor, and the questions for re-
flection that will be posed to the instructor. A set of pre-observation ques-
tions (see Appendix A) shape the conversation between the instructor and
the observer.
During the actual observation, the observer constructs a chronological
representation of the class-meeting using the Chronological Note-Taking In-
strument (see Appendix B). The data gathered using this instrument is then
used to help the observer complete the ALCOT (see Appendix C). The in-
structor does not see the Chronological Note-Taking Instrument, but is per-
mitted to view the ALCOT.
Following the observation of the class-meeting, the observer completes
the ALCOT. The observer then meets with the instructor to discuss the ob-
servation while using the completed ALCOT as a prompt for conversation
and questions.
It is recommended that a blueprint or diagram of the room where the ob-
servation takes place be used during the pre-observation and the post obser-
vation meetings. The blueprint of the room can be a useful tool for the nota-
tion and discussion of issues regarding instructor and student movement in
the space, use of affordances in the room, and the arrangement or re-arrange-
ment of configurable furniture. See Figure 1 for an example of the blueprint
that was shared with instructors who taught in the Collaborative Learning
Studio, SB 015.
Journal on Centers for Teaching and Learning
34
Figure 1. Blueprint of the Collaborative Learning Studio, SB 015
Items Intentionally Omitted from Observation Protocol
It is important to emphasize that the Active Learning Classroom Obser-
vation Tool is intended for development rather than for evaluation. The tool is
intended to help instructors develop their approach to teaching in the room
and facilitate reflection on active learning instructional practices. As such,
check box categories are not included within the tool. Although checkboxes
offer greater standardization among the observations, and are quite common
(Brent & Felder, 2004; Jarzabkowski & Bone, 1998) checklists can be distract-
ing (DeZure, 1999) and cumbersome (Millis, 1992). Checkboxes often do not
give a sense of the duration of an activity, nor do they allow for descriptions
of quality or context in interactions.
Checkboxes were considered as a means to record technologies present in
active learning classrooms, as they could be used to make instructors aware
of the technologies available within the rooms while also being used as a use-
reporting aid for faculty developers. Checkboxes that focused on available
classroom technologies were not included in the development of the tool due
to the variation of technologies between classrooms. Additionally, because
there are so many technologies within the active learning classrooms, it was
essential that when instructors had an opportunity to review their observa-
tion, the ALCOT development team did not want them to feel like they were
Journal on Centers for Teaching and Learning
35
not using the rooms to their full potential if they only engaged with certain
affordances given their instructional needs.
One category for consideration that has since been removed from
ALCOT's current draft pertains to instructor presentation. Most observation
forms do include instructor presentation as a main category for observation.
At first, this category was included in the ALCOT as a way for instructors to
reflect on their presentations in the context of the space and the technology
(e.g., Did they move around the room? Could students make eye contact
with them? Did they make smooth transitions?). However, in order to focus
the ALCOT on student engagement rather than instructor presentations,
instructor presentations are addressed within a sub-category of classroom
management (see Appendix C, question 4b).
Piloting and Applications of the ALCOT Instrument
In the spring and fall of 2015, faculty developers and researchers at Indi-
ana University-Bloomington piloted the Active Learning Classroom Obser-
vation Tool with eight instructors from a variety of disciplines. The ALCOT
was piloted in three different Mosaic classrooms (see Figures 2-4). Below are
the three classrooms in which faculty developers and researchers piloted the
ALCOT:
Journal on Centers for Teaching and Learning
36
Figure 2. Image of Collaborative Learning Studio SB 015.
The Collaborative Learning Studio (SB 015) has sixteen six-student tables,
accommodating 96 individuals. Each table contains a computer, large moni-
tor, connections for three laptops, and two push-to-talk microphones. Hud-
dleboards (portable whiteboards) are available for each table group for col-
laborative work. A twenty-foot wide video wall allows instructors to display
multiple types of content, including computer, document camera, and the
screens of individual student tables. Made up of sixteen monitors, the wall
can project a view of all sixteen student table computers, a combination of
four sources (tables and/or instructor tools), or one large image.
Journal on Centers for Teaching and Learning
37
Figure 3. Image of active learning classroom GA 0009.
GA 0009 is located in Indiana University-Bloomington’s new Global and
International Studies building. GA 0009 has ten tables and twenty-two
wheeled chairs to allow for multiple configurations. Three of the four walls
are whiteboard walls for student collaboration. The instructor station has a
Crestron display with a document camera. On one wall, GA 0009 has two
80’’ flat panel displays. The room also has an HD video camera for lecture
capture and PC-based video conferencing and collaborative technologies.
Journal on Centers for Teaching and Learning
38
Figure 4. Image of active learning classroom Cedar Hall 002.
Cedar Hall 002 has seven, eight-person tables and sixty chairs for group
collaboration. Students have access to multiple whiteboards on the walls and
Huddleboards for collaborative work. Cedar Hall 002 has three projection
screens and a Copycam (whiteboard camera), as well as an interactive white-
board that enables the capture of instructional materials produced in the
classroom by both the professor and the students.
Lessons Learned from the Pilot
After eight observations, in three different classrooms of instructors from
different disciplines, faculty developers and researchers at IU-Bloomington
reflected on several initial lessons learned:
1. The Active Learning Classroom Observation Tool allowed for
focused observations of the instructor’s integration of the
room’s spatial arrangements, technologies, and pedagogies. It
also served as a guide to observers’ comments on the instruc-
tional choices made in the context of the space.
Journal on Centers for Teaching and Learning
39
The ALCOT guided observers' attention to the use and interac-
tions within the active learning classrooms. In one observation of a
class held in GA 0009 (see Figure 3), a French language instructor
moved seamlessly from an in-class activity with the document cam-
era, to an explanation of a student's question on a whiteboard, to
engaging students with an activity using the screens. The observer
was impressed with the instructor’s comprehensive use of the
room’s spatial arrangement and technologies.
Significantly, ALCOT’s categorizations inspired the observer to
focus on recording the instructor’s pedagogical approaches, rather
than to focus on simply the fact that many room affordances were
being used. The observer was able to write an observation report,
and hold a post observation conversation, that focused on the inte-
grations of space, technology, and pedagogy instead of merely the
use of the technology in the room. The post-observation conversa-
tion, then, focused on refining both pedagogy and use of the room’s
spatial arrangements and technologies in ways that more fully sup-
ported the instructors’ pedagogical goals.
Observers noted that it could be tempting to focus on how many
room affordances an instructor used during a classroom observation
in an active learning classroom. But, by focusing on the pedagogical
approaches (addressed in the categories for observation), observers
found that they could provide more effective feedback to instructors
regarding teaching in the space.
2. The Active Learning Classroom Observation Tool inspired
conversations about teaching in all types of classrooms.
Even though the focus of the post-observation reflection meeting
was to discuss and reflect on an instructor's approach to teaching in
an active learning classroom, faculty developers found that conver-
sation often included discussion of teaching in traditional class-
rooms.
For example, one post-observation conversation highlighted how
an instructor moved around the classroom space and interacted with
students during an in-class session in an active learning classroom.
In a later discussion, the instructor stated that she had rethought how
she positions herself in relation to students in all of her courses as the
result of the observation in an active learning classroom.
Journal on Centers for Teaching and Learning
40
Such an observation has inspired observers to ask informal ques-
tions during the post-observation conversation about how the ways
that instructors have taught in an active learning classroom could be
applied to any classroom. In the future, faculty developers at IU will
consider adding a prompt as part of a post-observation conversation
protocol to help instructors reflect on ways that the lessons that they
have learned from teaching in an active learning classroom could
transfer to other classrooms.
3. The Active Learning Classroom Observation Tool helped
faculty developers better identify how faculty were using the
active learning classroom. This perspective helped faculty
developers better understand how to support the particular
faculty being observed, but also rethink broader faculty
development efforts.
Through classroom observations and conversations with instruc-
tors, faculty developers gained a better sense of how the active learn-
ing classrooms were being used or under-used in the context of an
instructor’s intended pedagogical approach.
For example, in SB 015 (see Figure 2), several instructors observed
displayed their slides on the three large wall screens in the room but
were not displaying slides on the screens at the student tables, even
though the technology in the room easily allowed them to do so. In
the post-observation protocol, observers asked instructors why they
were not sharing content on the student screens, especially since stu-
dents would better be able see content on slides or see the instruc-
tions for a group activity if they did so. Instructors responded that
they did not realize that sharing their presentation to the student
table screens was an option. Observers then noted that by sharing
their slides on student screens, a very small change on their part,
they could better support student attention during lecture or small
group collaboration by providing students a closer view of course
materials.
With this knowledge, faculty developers are able inform faculty
who teach in any classroom with student table screens about options
in consultations and workshops for instructors teaching in these
spaces (Hendry & Oliver, 2012).
Journal on Centers for Teaching and Learning
41
Though intended as a classroom observation tool, the ALCOT could also
serve faculty developers in Centers of Teaching and Learning in other ways,
including:
Traditional classroom observations
The categories presented within the Active Learning Classroom
Observation Tool are useful prompts for reflection for traditional
classroom observations. The guide can serve as a way for instructors
to think more about how their pedagogy, classroom space, and tech-
nologies intersect in any learning classroom.
Peer observation of teaching
The ALCOT could be used as part of the Peer Observation of
Teaching process for instructors who teach in active learning class-
rooms. The guide can serve as a way for instructors to help other
instructors think more about how their pedagogy, classroom space,
and technologies intersect in any classroom.
Guide for self-reflection
Instructors who teach in active learning classrooms can use the
ALCOT as a guide for thinking about how they should or could
approach teaching in an active learning classroom. It can serve as a
self-checking guide for instructors who may wish to prepare for a
classroom session or think about teaching a course within an active
learning classroom.
Open classroom observations
The ALCOT could be a guide for a Master or Open Classroom, a
"model" classroom session where an instructor teaches a class and
hosts a discussion with peer observers following the class session.
Peer observers could use the ALCOT as a guide for their Open Class-
room observation in active learning classrooms. The ALCOT could
be used for note taking and as a prompt for post-class observation
discussion.
Journal on Centers for Teaching and Learning
42
Conclusion
Previous research has shown that active learning classrooms, when used
to facilitate the practices of active learning pedagogies, can positively influ-
ence student learning (Beichner et al., 2007; Brooks, 2012; Byers, Imms, &
Hartnell-Young, 2014; Dori & Belcher, 2005; Gaffney, Richards, Kustusch,
Ding, & Beichner, 2008). When active learning classrooms are coupled with
active learning pedagogies, there is potential to reduce student failure rates,
improve students’ conceptual understanding of a given topic, increase class
attendance, support student problem solving skills and improve student
attitudes toward learning (Beichner et al., 2007; Gaffney et al., 2008). Given
the benefits of active learning classrooms and pedagogies, it is not surprising
that more universities are building active learning classrooms. As more
active learning classrooms emerge, the opportunity for faculty developers to
support the specific needs of instructors who teach in those spaces will con-
tinue to grow.
In this article, faculty developers and researchers describe the creation
and application of the Active Learning Classroom Observation Tool (AL-
COT) as a reflective observation protocol specific to the context of active
learning classrooms. An implicit assumption behind the use of ALCOT is
that effective learning experiences require that the instructor use the spatial
and technological affordances of a classroom in ways that enhance, rather
than undermine, the goals and practices of active and collaborative learning
pedagogies. In other words, given the importance of the instructor’s choice
of spatial, technological, and social arrangements of learners to the success
of particular learning activities, it is imperative that these choices be exam-
ined and evaluated in classroom observations. As more instructors redesign
their pedagogical approaches to exploit a growing number of active learning
classrooms, it is imperative that faculty developers offer guidance and sup-
port to take advantage of the unique pedagogical possibilities that these rich
learning environments offer. Faculty developers and researchers at IU-
Bloomington believe that the ALCOT is a useful tool for classroom observa-
tions, one that allows faculty developers an opportunity to help instructors
design and realize effective active learning pedagogies for their students.
Ultimately, however, it is expected that the insights that faculty developers
and instructors gain through the use of the ALCOT instrument may be used
to leverage the spatial and technological affordances to improve active learn-
ing pedagogies in any and all classrooms, not just active learning classrooms.
Journal on Centers for Teaching and Learning
43
Future Development and Future Use of the ALCOT
The ALCOT development team intends to further refine the tool and the
observation process surrounding it. Ideally, after two years of continuous
use of the ALCOT, which requires an additional academic year at the time
of this writing, the development team will conduct a full analysis of the pre-
vious observations. This analysis of the efficacy of the categories used by the
ALCOT will include a deeper look into how the pre-observation questions
integrate with and support the ALCOT. Faculty developers and researchers
at IU speculate that the pre-observation questions and process could be more
prescriptive, which would align with the design of the ALCOT questions.
Further analysis may prove that the pre-observation process has the poten-
tial to be an effectively-timed faculty development opportunity, and it
would be interesting to examine whether these conversations alone have an
effect on instructor behaviors and attitude in the active learning classroom.
Another valuable mechanism to gather feedback on the efficacy of the AL-
COT is through instructor interviews to discover how the process helped
them teach in active learning classrooms. Refinement of the ALCOT could
be achieved through this three-pronged approach, by conducting in-depth
analysis of the ALCOT outcomes, considering the role and influence of the
pre-observation questions and process, and gauging instructor perception of
any change in his/her satisfaction teaching in active learning classrooms as a
result of the ALCOT. This approach provides a replicable process for other
teaching centers to deploy and refine the ALCOT to suit the needs of their
particular institution or teaching situation(s).
References
Beichner, R. J., Saul, J. M., Abbott, D. S., Morse, J. J., Deardorff, D., Allain, R.
J., & Risley, J. S. (2007). The student-centered activities for large en-
rollment undergraduate programs (SCALE-UP) project. Research-
based reform of university physics, 1(1), 2-39.
Bielefeldt, T. (2012). Guidance for technology decisions from classroom ob-
servations. Journal of Research in Technology Education, 44(3). 205-223.
Brent, R., & Felder, R. M. (2004). A protocol for peer review of teaching. Paper
presented at the American Society for Engineering Education An-
nual Conference and Exposition, Salt Lake City, Utah.
https://peer.asee.org/13897
Journal on Centers for Teaching and Learning
44
Brooks, D. C. (2012). Space and consequences: The impact of different formal
learning spaces on instructor and student behavior. Journal of Learn-
ing Spaces, 1(2), 1-10.
Byers, T., Imms, W., & Hartnell-Young, E. (2014). Making the case for space:
The effect of learning spaces on teaching and learning. Curriculum
and Teaching, 29(1), 5-19.
Chism, N. V. N. (2007). Peer review of teaching: A sourcebook (2nd ed.). San
Francisco: Jossey Bass.
Dori, Y. J., & Belcher, J. (2005). How does technology-enabled active learning
affect undergraduate students' understanding of electromagnetism
concepts?. The Journal of the Learning Sciences, 14(2), 243-279.
Evertson, C., & Holley, F. (1981). Classroom observation. In J. Millman (Ed.),
Handbook of teacher evaluation (pp. 90-109). Beverly Hills, CA: Sage
Publications.
Fullerton, H. (1999). Observation of teaching. In H. Fry, S. Ketteridge, & S.
Marshall (Eds.), A handbook for teaching and learning in higher educa-
tion: Enhancing academic practice (pp. 220-235). London: Kogan Page.
Gaffney, J. D., Richards, E., Kustusch, M. B., Ding, L., & Beichner, R. J. (2008).
Scaling up education reform. Journal of College Science Teaching, 37(5),
48.
Gosling, D. (2000). Guidelines for peer observation of learning and teaching. ES-
Calate Regional Networking Seminars. Retrieved from http://esca-
late.ac.uk/resources/peerobservation/gosling.zip
Gosling, D. (2002). Models of peer observation of teaching. York, United King-
dom: Generic Centre, Learning and Teaching Support Network.
Hammersley-Fletcher, L., & Orsmond, P. (2004). Evaluating our peers: Is
peer observation a meaningful process? Studies in Higher Education,
29(4), 489-503.
Jarzabkowski, P., & Bone, Z. (1998). A how-to guide and checklist for peer
appraisal of teaching. Innovations in Education and Training Interna-
tional, 35(2), 177-182.
McMahon, T., Barrett, T. & O'Neill, G. (2007). Using observation of teaching
to improve quality: Finding your way through the muddle of com-
peting conceptions, confusion of practice and mutually exclusive in-
tentions. Teaching in Higher Education, 12(4), 499-511.
Millis, B. J. (1992). Conducting effective peer classroom observations. To Im-
prove the Academy, 11, 189-206.
Radcliffe, D. (2009). A Pedagogy-Space-Technology (PST) framework for de-
signing and evaluating learning spaces. In D. Radcliffe, H. Wilson,
Journal on Centers for Teaching and Learning
45
D. Powell, & B. Tibbetts (Eds.), Learning spaces in higher education: Pos-
itive outcomes by design. Proceedings of the Next Generation Learning
Spaces 2008 Colloquium (pp. 11-16). Brisbane: University of Queens-
land.
Sawada, D., Piburn, M. D., Judson, E., Turley, J., Falconer, K., Benford, R., &
Bloom, I. (2002). Measuring reform practices in science and mathe-
matics classrooms: The reformed teaching observation proto-
col. School Science and Mathematics, 102(6), 245-253.
Schön, D. (1983). The reflective practitioner: How professionals think in action.
London: Temple Smith.
Smith, M. K., Jones, F. H. M., Gilbert, S. L., & Wieman, C. E. (2013). The Class-
room Observation Protocol for Undergraduate STEM (COPUS): A
new instrument to characterize university STEM classroom prac-
tices. Life Sciences Education, 12, 618-627.
Journal on Centers for Teaching and Learning
46
Tracey Birdwell is the Principle Instruction Technology consultant for the
Mosaic Active Learning Initiative at Indiana University. She has 13 years of
experience teaching in face-to-face and online environments. Before coming
to Indiana University, she worked in faculty development at the University
of Delaware and at Virginia Tech. Dr. Birdwell currently works with faculty
to encourage the adoption of active and collaborative learning strategies for
instructors who teach in Mosaic classrooms. Her research focuses on effec-
tive faulty support for instructors teaching in active learning classrooms.
Tiffany Roman is a doctoral candidate in the Instructional Systems Technol-
ogy department at Indiana University. As a graduate assistant research
within the Educational Research and Evaluation division of University
Information Technology Services (UITS), she studies instructor pedagogies
and student learning practices across active learning spaces. Her scholarship
includes research on teacher technology integration practices and secondary
design education. She has over 10 years of teaching experience at the K-16
level. Leslie Hammersmith is the Assistant Dean for Technology Enhanced
Instruction for the University of Illinois College of Medicine at Peoria, where
she focuses on incorporating technologies to advance medical education,
simulation, and innovative educational programs for medical students and
residents. Leslie has held leadership roles in educational technologies at Uni-
versity of Illinois at Urbana-Champaign, Indiana University, and University
of Illinois College of Medicine. Her projects have encompassed implement-
ing the enterprise LMS on a large R1 campus, creating and supporting com-
prehensive faculty development, and leading efforts on multiple campuses
for transforming student learning spaces. Douglas Jerolimov is an Instruc-
tional Design Consultant at the Center for Teaching and Learning at Indiana
UniversityPurdue University Indianapolis. Doug has 15 years of experi-
ence teaching face-to-face and online environments. Before coming to IUPUI,
he held appointments as lecturer at the University of Virginia’s Department
of Science, Technology, and Society, and as historian at the Historic Ameri-
can Engineering Record. Dr. Jerolimov advises faculty engaged in the design
of courses and training for online, blended, and face-to-face environments.
His research focuses on pedagogical strategies for interprofessional educa-
tional settings.
Journal on Centers for Teaching and Learning
47
Appendix A
Pre-Observation Checklist for
Active Learning Spaces Observation
1. What would you like me to focus on as I observe your course?
2. What is your learning objective for the class I am about to observe?
3. How have you designed your class session to achieve this goal?
4. How are you planning on using the affordances of the room to
support your goals? To support active learning? To support
collaborative learning?
5. Is there anything else you would like me to consider as I observe
this class?
When possible, at each stage of the observation, provide a diagram or
blueprint to act as a point of reference for discussion about activities and
interactions. A diagram or blueprint can be a particularly useful point of ref-
erence in spaces with configurable furniture.
Journal on Centers for Teaching and Learning
48
Appendix B
Chronological Note-Taking Instrument
Use this form for note-taking during the observation.
Under the “Time” category, note the time and duration of activities and
the various interactions that took place during the observation. Under the
“Description” category, note what happened during the class, offering
merely descriptions of events observed. Under the “Comments” category,
note thoughts, possible suggestions, or reactions to what you are observing.
After the observation, use the information and ideas gathered and organized
in the form to inform your responses to the ALCOT.
Time
Description
Journal on Centers for Teaching and Learning
49
Appendix C
Active Learning Classroom Observation Tool
Instructor:
Department:
Course:
Section:
Course Enrollment:
Classroom:
Observation Date:
Use the following criteria that apply to guide your classroom observation
descriptions, comments, and suggestions:
1. Instructor use of the Active Learning Classroom to support active
learning:
a) In what ways did the instructor engage students in active learning
during this class?
b) How did the instructor use instructional technologies in the room
(i.e., media, tables, huddle boards) to engage students in in-class
activities and instruction?
2. Collaborative Learning in the Active Learning Classroom:
a) How did the instructor engage students in collaborative learning?
b) How did the instructor provide directions for collaborative
activities?
c) How did the instructor ensure that all students participated in
collaborative activities?
3. Formative Assessment in an Active Learning Classroom:
a) What artifact(s) of learning did the instructor ask students to
produce during (or prior) to class?
b) How and with whom did students share their artifacts?
c) How did the instructor provide feedback to students during
learning activities or assessments?
d) How did the instructor facilitate peer feedback during learning
activities or assessments?
Journal on Centers for Teaching and Learning
50
4. Classroom Management in the Active Learning Classroom
a) How did the instructor indicate where students needed to focus for
various methods of instruction?
b) How did the instructor use the classroom space while engaging the
entire class in a presentation or a learning activity? Did they walk
around? Could students see, hear, or find the instructor?
c) How did the instructor make transitions between different
instructional events (e.g., move from lecture to group activity)?
5. General Observations:
a) What instructional choices worked exceptionally well?
b) What instructional choices do you think could be improved and
how would you improve them?
... However, it is difficult to assess how successful the design of spaces like this are for active learning [22]. Active learning practices are not being incorporated as widely as expected in some STEM fields [14] and other active learning classroom installations report an increased volume of support requests related to the transition from traditional to active learning classrooms [2]. ...
... In our review of existing protocols [27], two emerged as potentially well suited to this project -the Active Learning Classroom Observation Tool (ALCOT) [2] and the Teaching Dimensions Observation Protocol (TDOP) [16]. In 2016, Birdwell et al. [2] suggested that there are no existing observation protocols or tools that emphasize the active learning ecosystems per se -the intersection of space, technology, and pedagogy -as important considerations that factor into the context of teaching. ...
... In our review of existing protocols [27], two emerged as potentially well suited to this project -the Active Learning Classroom Observation Tool (ALCOT) [2] and the Teaching Dimensions Observation Protocol (TDOP) [16]. In 2016, Birdwell et al. [2] suggested that there are no existing observation protocols or tools that emphasize the active learning ecosystems per se -the intersection of space, technology, and pedagogy -as important considerations that factor into the context of teaching. As a result, they proposed their own: The ALCOT [2]. ...
Article
Full-text available
In our work, we are focusing on the use of classroom observation to provide feedback on instructional space design. An initiative to redesign teaching space began a decade ago at a large, research-intensive institution. In September 2018, a large-scale (477 seat) active learning classroom became operational. The affordances of this space are intended to enable active teaching and learning in large classes. However, it is difficult to assess how successful this space is for active learning. A multi-year study has been undertaken to observe teaching practice in situ, with the goalof developing design principles for instructional space and technology that support the development, design, and implementation of teaching activities. Existing teaching observation protocols do not fully capture the interaction between the instructor and the space because such protocols were generally intended for other purposes. The goal is to develop a protocol that captures activities that are both intrinsic and extrinsic to teaching. This paper describes the development and use of an observation protocol. The core of the protocol is the wellknownTeaching Dimensions Observation Tool (TDOP). The scope of the TDOP is extended to active learning activities drawing from the Active Learning Classroom Observation Tool (ALCOT). The resulting extended protocol, TDOP+, was used for coding both live and recorded classroomobservations in the Winter 2020 term. This extended protocol allows the researchers to capture information that characterizes the intersection of pedagogy, space, and technology through Activity Theory. In future work, the data gathered through observations will be analyzed using theDifferentiated Overt Learning Activities (DOLA) framework, to provide insight into what types of teaching activity happens in a large-scale active learning classroom across STEM education and how active learning in large classrooms compares to pedagogy in other spaces.
... The more general observation protocols used for engineering include the Classroom Observation Protocol for Undergraduate STEM (COPUS) and the Active Learning Spaces Observation Tool (ALCOT) (Birdwell, Roman, Hammersmith, & Jerolimov, 2016;Smith, Jones, Gilbert, & Wieman, 2013). The intent of COPUS is to characterize various instructor and student behaviors such as lecturing, use of clicker questions, working in groups, and listening in STEM classrooms . ...
... The intent of COPUS is to characterize various instructor and student behaviors such as lecturing, use of clicker questions, working in groups, and listening in STEM classrooms . ALCOT captures qualitative data related to the use of technology and pedagogies in active learning spaces (Birdwell et al., 2016). These more general protocols are not specific enough to capture specific characteristics of engineering or ED and both are validated for use at the postsecondary level. ...
Article
Recent education reforms including the Next Generation Science Standards emphasize integrating engineering into K‐12 science instruction. Although prior studies have characterized engineering design (ED) in K‐12 settings, no validated protocols attempt to characterize ED integration in secondary science classrooms. The present study used the ED and observation protocol literature to develop, validate, and test the reliability of the Classroom Observation Protocol for Engineering Design (COPED) instrument. Consistent with the ED literature, the COPED characterizes two components of ED: process and habits of mind. To use COPED, researchers code for ED process components, engineering habits of mind, and instructional grouping in 2‐min increments. Researchers also record qualitative field notes. Post‐observation, researchers include descriptions of disciplinary ties and ED structure. Support for content validity was established through two cycles of expert panel review and revision. Interrater reliability for the instrument was Cohen's κ = .81. The data collected from the reliability study were subsequently analyzed to demonstrate the utility of COPED in characterizing ED integration. The COPED provides researchers a tool to systematically characterize ED instruction within secondary science classrooms. Using COPED to characterize ED may support science educators in developing ED‐related teacher preparation and professional development.
... Over 200 U.S. institutions currently have ALCs (Beichner 2014), and formal campus-wide initiatives exist at multiple universities, including the University of Iowa (Florman 2014;Van Horne et al. 2012), the University of Minnesota (Baepler et al. 2016), and Indiana University (Birdwell et al. 2016). ...
... Eddy, Converse, and Wenderoth (2015) created an observational tool for capturing the presence, absence, and frequency of best practices for active learning, although it is specific to large ALCs (i.e., 50-700 students) in STEM domains. An additional classroom observation tool exists that is applicable to a range of ALC sizes, but its aim is to foster instructor reflection on active learning pedagogies within ALCs, rather than serve research purposes (Birdwell et al. 2016). ...
Article
Full-text available
To better support pedagogies of engagement, such as peer learning, team learning, and inquiry-based learning, Active Learning Classrooms (ALCs) are equipped with a range of physical and technological affordances, including movable furniture, ample whiteboard access, and immediate work-sharing capabilities using mobile technologies and displays. Although ALCs are created with pedagogical flexibility in mind, previous research on ALCs has favoured quasi-experimental research designs that relate qualities of space and qualities of learning within large introductory STEM courses. By comparison, smaller active learning classrooms that serve 24 students or fewer have different spatial, pedagogical, and technological affordances and therefore benefit from alternate research designs. We argue that qualitative approaches, particularly methods that examine the co-occurrences of teacher and learner interactions within ALCs, are suited to exploring dimensions of enacted pedagogies in context. In this article, we present how video platform technologies and qualitative data analysis software can be used to support research methods that combine observational and experiential data while minimizing classroom disruptions. For contextualization, we provide a case study example of world language instruction employing the proposed methods within a small ALC.
Conference Paper
Full-text available
The teaching and learning modality hybridity in the (post-)Covid era has triggered a number of evaluation and assessment-related challenges. Zeroing in on teacher evaluation, observation and good practices in HyFlex learning environments, the present paper focuses on the development of an evaluation tool for Education Administrators. In particular, it discusses and traces the pre-, while-and post-developmental stages along with the literature-informed micro-/macro-decisions needed to arrive at a comprehensive observation form that may offer actionable insights into effective quality monitoring. Summary The teaching and learning modality hybridity in the (post-)Covid era has triggered evaluation and
Thesis
Après une décennie marquée par l’essor et les promesses d’un enseignement dématérialisé, les années 2010 ont vu sur les campus l’émergence d’espaces physiques d’apprentissage innovants – communément appelés Learning Spaces – qui semblent avoir formalisé un retour vers le physique. Active Learning Classrooms, Collaborative Lecture Theaters, Learning Commons et autres Learning Centers ont ainsi progressivement intégré le langage des praticiens, mais aussi des gouvernances. Ces espaces portent la double ambition d’une modernisation des lieux d’apprentissage visant à s’adapter aux nouveaux modes de vie quotidiens des étudiants, et d’une innovation pédagogique. Pourtant, ils fascinent autant par leur matérialité qu’ils interrogent par rapport à la réalité de leurs usages. Dans ce mouvement global, le Japon a lui aussi vu les Learning Spaces s’imposer dans les perspectives stratégiques de ses universités. Notre démarche vise à mettre en lumière les dynamiques internes et externes qui ont guidé et guident encore leur mise en place sur ce territoire, dans un schéma d’interrelation complexe et multidimensionnelle avec le système d’Enseignement Supérieur dans lequel ils évoluent. Elle positionne ces espaces à l’intersection d’enjeux à la fois distincts, interconnectés, parfois contradictoires, et héritant d’une historique particulière, tout en s’articulant avec des aspects socio-culturels eux aussi spécifiques. Elle questionne par ailleurs leur finalité réelle, et une vocation qui outrepasserait les questions pédagogiques pour notamment répondre à des tensions institutionnelles. Elle liste enfin des enseignements tirés de ce cas, en les confrontant à une transposition potentielle.
Article
This research discusses the characteristics of a global set of Classroom Observation Protocols (COPr) to uncover their design goals and assess the value propositions for Engineering Education (EE) in active learning environments. To achieve the research purpose, a Systematic Literature Review (SLR) method was chosen, based on a process-oriented model that covered a set of 23 global databases, from January 2000 to December 2020. The search and screening process retained 109 literature sources for deep study, after quality assessment. The research results revealed the approaches of 111 classroom observation applications and 68 different protocols. The upward trend of new protocols and applications in the last 6 years identified by the study has resulted in considerable information overload for practitioners. The authors address this by analysing the 68 protocols, characterising them and listing the strengths of each one aligned with the EE context. They achieve this goal by applying a systematic 4-category and 5-dimension framework to compare the protocols. The bibliometric data also reveals how the observation strategies in active learning environments incorporate Engineering disciplines in several applications over time, with 82% of the total.
Article
Full-text available
This paper proposes three models of peer observation - an evaluation model, intended to make judgements about the quality of teaching; a developmental model, intended to help develop teaching especially new teachers in higher education; and a collaborative peer review model intended to foster dialogue about teaching.
Article
Full-text available
An explanatory, mixed method study examined the impact of learning spaces on teachers’ pedagogy, student engagement and student learning outcomes in a technology-rich school setting. Its quasi-experimental design allowed examination of differences in these variables between two settings ‐ ‘traditional’ classrooms, and ‘new generation learning spaces’ (NGLS). Results from quantitative analyses over a one-year period indicated that particular configurations of learning spaces did have a measurable effect on how students’ perceived their learning experiences and their engagement levels, with improvements often linked to NGLS. In addition, comparative analyses of experimental and control group standardised assessment data in subjects English and mathematics indicated a similar effect for the same participants. The study suggests that a single-subject, repeated measures design (SSRD) can be used to measure the outcomes effect of space on student learning outcomes. In this regard, this approach addresses a perceived lack of empirical data highlighted by recent reviews of research on this topic.
Article
Full-text available
Instructors and the teaching practices they employ play a critical role in improving student learning in college science, technology, engineering, and mathematics (STEM) courses. Consequently, there is increasing interest in collecting information on the range and frequency of teaching practices at department-wide and institution-wide scales. To help facilitate this process, we present a new classroom observation protocol known as the Classroom Observation Protocol for Undergraduate STEM or COPUS. This protocol allows STEM faculty, after a short 1.5-hour training period, to reliably characterize how faculty and students are spending their time in the classroom. We present the protocol, discuss how it differs from existing classroom observation protocols, and describe the process by which it was developed and validated. We also discuss how the observation data can be used to guide individual and institutional change.
Article
Full-text available
This paper begins by reviewing some of the different models of third-party observation of university teaching that can be found in the literature. Having analysed these, it argues that—if ‘peer’ is taken to indicate equality of status—only one is genuinely a model of peer-observation. It proposes an alternative categorisation of third-party observations of teaching dependant on who controls the information generated by the process. A preferred six-dimensional model based on control by the person being observed of the data-flow, and other procedural aspects, is presented and explored. Evaluative comments, by university teachers who have undertaken the process, are presented to illustrate the benefits of adopting this model.
Article
Correlational analysis of two years of classroom observation indicates relationships between technology use and various classroom characteristics, including teacher roles and instructional strategies. Three observers used the ISTE Classroom Observation Tool (ICOT) to record 144 observations of classrooms participating in a variety of educational technology grant programs in the United States. The findings suggest that decision makers consider the instructional implications and possible unintended consequences of implementing different information and communication technologies.
Article
Peer classroom observations—if conducted systematically, professionally, and collegially—can provide significant documentation of what occurs in a university classroom. Most important, the collegial dialogues they engender also serve as catalysts for teaching enhancement. This article discusses some of the issues and options associated with classroom observations and provides experience-based guidelines for conducting them.
Article
Peer observation of teaching can be seen as a means by which the quality of the teaching and learning process in higher education establishments is both accounted for and improved. The majority of the literature to date has focused on the mechanisms for implementing peer observation systems and its links to enhanced professional practice. However, little attention has been given to the complexities involved in delivering the peer observation process, and how it may be managed and integrated in order to maximize benefits for teaching and learning. This article reports on an evaluation of two systems of the peer observation within one ‘post 1992’ university. It utilizes data from semi‐structured interviews with lecturers, and identifies the need for a clear focus and goals. It also illustrates the necessity to see both the breadth and depth of the process.
Article
In tertiary teaching it can be quite rare for academics to talk to each other about their actual teaching: their practices, motivational techniques, approaches to students and methods of assessment. By this we mean focused discussion about personal teaching practices as distinct from discussion revolving around student performance or the type of ‘jockeying for position’ which is part of the political process. Given that teaching is considered to be an important element of the academic role and institutions are under increasing pressure to better serve their clients, it is necessary for academics to engage in discussion and evaluation of their teaching performance. Peer appraisal is one method of evaluation which can provide feedback on teaching for both personal development and provide information for institutional or personnel purposes. The following paper provides a step‐by‐step guide for peer appraisal that has been developed and used successfully by the authors. While this process has been designed principally for self‐development purposes, if each stage is documented and justified the outcomes may be useful, in conjunction with other material, for institutional purposes such as promotion and grant applications. The paper is written as a form of checklist for academics wishing to engage in peer appraisal and includes some notes based upon our personal experiences.
Article
This sourcebook is intended to help college administrators develop a strong peer review component for their faculty evaluation and development system. Part 1, an overview of peer review, provides arguments for peer review and briefly describes the process of establishing a peer review system. Chapter 1 reviews the tenets of effective evaluation of teaching in general, provides a rationale for the use of peer review, and discusses issues that arise when considering peer review. Chapter 2 offers suggestions for developing and implementing a peer review system. Chapter 3 considers the role of chairs, colleagues, and peer reviewers, and Chapter 4 lists the components of a peer review system. The chapters in Part 2 consider various aspects of peer review and the procedures and forms that can be employed in conducting each type of activity. Individual chapters offer resources on reviewing course material (Chapter 5); classroom performance (Chapter 6); scholarship of teaching and contributions to departmental teaching efforts (Chapter 7); and teaching portfolios (Chapter 8). The concluding chapter (Chapter 9) presents guiding principles for the peer review of teaching and reviews major points in the book. (Contains 55 references.) (DB)