Learn before Lecture: A Strategy That Improves Learning
Outcomes in a Large Introductory Biology Class
Marin Moravec, Adrienne Williams, Nancy Aguilar-Roca, and Diane K. O’Dowd
Department of Developmental and Cell Biology, University of California, Irvine, Irvine, CA 92697-1280
Submitted April 22, 2010; Revised September 4, 2010; Accepted September 5, 2010
Monitoring Editor: Barbara Wakimoto
Actively engaging students in lecture has been shown to increase learning gains. To create time
for active learning without displacing content we used two strategies for introducing material
before class in a large introductory biology course. Four to five slides from 2007/8 were removed
from each of three lectures in 2009 and the information introduced in preclass worksheets or
narrated PowerPoint videos. In class, time created by shifting lecture material to learn before
lecture (LBL) assignments was used to engage students in application of their new knowledge.
Learning was evaluated by comparing student performance in 2009 versus 2007/8 on LBL-
related question pairs, matched by level and format. The percentage of students who correctly
answered five of six LBL-related exam questions was significantly higher (p ? 0.001) in 2009
versus 2007/8. The mean increase in performance was 21% across the six LBL-related questions
compared with ?3% on all non-LBL exam questions. The worksheet and video LBL formats were
equally effective based on a cross-over experimental design. These results demonstrate that LBLs
combined with interactive exercises can be implemented incrementally and result in significant
increases in learning gains in large introductory biology classes.
The traditional teaching/learning pattern in a large lecture
course begins with faculty introducing new material each
class, students reviewing the information at a later time,
followed by a summative exam to assess student under-
standing. Higher education practices that promote learning
through active engagement in class have been shown to
improve student performance (Hake, 1998; Knight and
Wood, 2005; Michael, 2006; Freeman et al., 2007; Chaplin,
2009). We found that creating a more learner-centered envi-
ronment in our large introductory biology course by replac-
ing a small amount of new information in each class period
with active learning exercises increases student engagement,
encourages critical thinking, and has improved student at-
titudes (O’Dowd and Aguilar-Roca, 2009). However, typical
of introductory biology courses, we cover a large amount of
material and have conformed to the traditional pattern of
introducing most of this in lecture, thus limiting the time
available for active engagement exercises in class. It is clear
that if more knowledge-level information could be moved
out of lecture, there would be more time in class for active
learning. The question is, do students learn material as ef-
fectively if the first exposure is moved out of the classroom
and time in lecture is devoted to teaching higher-order
There is mounting evidence that a variety of preclass
activities that introduce new material can increase student
performance compared with traditional lectures. Preclass
online quizzes that encourage students to complete prepa-
ratory reading have resulted in improved exam performance
(Narloch et al., 2006; Dobson, 2008; Johnson and Kiviniemi,
2009). “Just-in-time teaching” (JiTT), a pedagogical strategy
first developed for use in physics, uses preclass assignments
to prompt thinking about the upcoming lecture topic
(Novak and Patterson, 2000). Students submit answers to
several questions before class, often addressing common
misconceptions, and the instructor uses the students’ pre-
lecture quiz answers to make last-minute adjustments to the
Address correspondence to: Diane O’Dowd (firstname.lastname@example.org).
© 2010 M. Moravec et al. CBE—Life Sciences Education © 2010 The
American Society for Cell Biology. This article is distributed by The
American Society for Cell Biology under license from the author(s). It
is available to the public under an Attribution–Noncommercial–Share
Alike 3.0 Unported Creative Commons License (http://creativecom-
CBE—Life Sciences Education
Vol. 9, 473–481, Winter 2010
day’s lecture material. The JiTT strategy has been success-
fully adopted for use in a broad range of disciplines (Novak
et al., 1999, 2004). A study done in an economics course
showed that completing JiTT exercises before class resulted
in a small positive effect on student exam performance
(Simkins and Maier, 2004). Results from a large nonmajors
biology course suggest that students prefer JiTT to tradi-
tional lectures (Marrs and Novak, 2004). These studies are
encouraging but the JiTT strategy, as the name implies,
involves substantial investment on the part of the faculty
member to evaluate student responses and revise lecture
material within hours of lecture. Faculty teaching large in-
troductory classes, particularly at research universities, have
little appetite for making major revisions to their traditional
lecture classes in the face of limited time, resources, and
reward (Justice et al., 2009). A less comprehensive but easier
to implement alternative is to move some knowledge-level
material from lecture to preclass exposure so students arrive
prepared to grapple with higher-level concepts during en-
gagement exercises already scheduled into the class. Theo-
retically, this could be accomplished by students completing
assigned readings before class but in our experience this
rarely occurs, even when coupled with preclass quizzes
worth a small number of points.
We therefore created preclass assignments designed to
help students learn knowledge-level material in preparation
for lecture. The assignments were presented in one of two
formats: a narrated PowerPoint video with a notetaking
sheet or a one-page worksheet. These had to be turned in but
did not involve instructor review of student responses. The
learn before lecture (LBL) assignments were coupled with
in-class active learning exercises in which students were
guided in applying their new knowledge to solve higher-
level problems. To determine whether this strategy was
effective in increasing learning, LBLs were included in a
small number of lectures (three of 30 periods) in 2009 to
replace material that was introduced as part of the lecture in
2007/2008. Performance of students on LBL-related exam
questions, matched by level and format, in 2009 was com-
pared with 2007/8. Performance on non-LBL exam ques-
tions was also compared.
The subjects for this study were students in introductory
biology (Bio 93) taught at University of California, Irvine
during fall quarter 2007 (Section A), 2008 (Sections A and B),
and 2009 (Sections A and B). Students were informed that
course material, including student responses on surveys,
homework, and exams, would be collected as part of an
ongoing science education study. The IRB-approved study
information sheet was posted on the course website. Stu-
dents were given the option of declining (anonymously) to
have their data included in the education studies. Students
with Family Educational Rights and Privacy Act holds and
those under the age of 18 were excluded from the study. The
demographic profiles and SAT scores of the 2007, 2008, and
2009 student cohorts were obtained from the registrar with-
out student identifiers. Incoming knowledge level of stu-
dents in 2008 was compared with 2009 by evaluating their
performance on 17 questions administered online in a pre-
class quiz. The questions were from a multiple-choice Intro-
ductory Molecular and Cell Biology Concept Assessment
developed at the University of Colorado (Jia Shi, personal
The 10-wk course was taught in three, 50-min classes per
week in a lecture hall (capacity 444) with fixed seating. Each
class period was divided into ?3 ? 10 min PowerPoint
guided lecture segments to deliver content separated by 5–7
min active-learning exercises. These included problem solv-
ing in small groups, clicker questions, class discussion,
and/or interactive physical demonstrations. In each year
two lecture sections, A and B, were taught on MWF, from
12:00 to 1:00 pm and 1:00 to 2:00 pm, respectively, in the
same room, by the same instructor team (O’Dowd and War-
rior). All students were required to attend a 50-min weekly
discussion section led by a graduate teaching assistant. The
graded elements in the class included two in-class quizzes, a
midterm exam, and a final exam (worth 85% of the total
points). The remaining 15% of the total points possible was
associated with online quizzes, clicker and discussion sec-
tion participation, and LBL activities (2009 only).
Learn before Lecture Procedure
For the 2009 class we used three LBLs to introduce material
that was previously covered in one content section (?10 min
of PowerPoint guided lecture) in each of three lectures in
2007/8. The LBLs were available online for download 2 d
before the related lecture. To encourage completion of the
LBLs, students received credit worth 1% of the total grade
for uploading the completed assignment and 0.25% for com-
pleting an online quiz covering the material, before the
associated class. Credit for the LBL assignment was awarded
if legible electronic files were uploaded on or before the
deadline (3 or 4 h before class).
We compared the effectiveness of LBLs presented in two
The worksheet directed students to complete a short reading
assignment and to answer associated questions. Students
downloaded the worksheet PDF file, handwrote their answers,
digital photo. The PDF file was chosen as the distribution
format to minimize “copy and paste” plagiarizing that occurs
more easily with Word documents. The second format was a
PowerPoint video consisting of the slides that were used in
class the previous year, narrated by the same faculty. The
videos were made using Camtasia Relay software (TechSmith
Corporation), a screen/sound capture program. Students
downloaded a PDF of related PowerPoint slides and wrote
notes on the slide PDF, as they would for a typical lecture
session. They digitally submitted their notes as scanned images
or a digital photo. In 2009, students in Section A were assigned
the worksheet format for the Organelle LBL (associated with
lecture 5) and the video format for the Inheritance LBL (asso-
ciated with lecture 18). Students in Section B were assigned the
video format for the Organelle LBL and the worksheet format
for the Inheritance LBL.
M. Moravec et al.
CBE—Life Sciences Education474
Anecdotal evidence from previous classes suggested that
students who make a cut-out model of a biological topic
perform better on a related exam question than those who
work only with a drawing on the same topic. We tested this
by making the Cloning LBL (associated with lecture 24) a
worksheet activity for both sections, where one group inter-
preted a drawing and filled in a worksheet and the other cut
out and constructed a plasmid. Half of the students in each
section were randomly assigned to the drawing or the cut-
out and construct worksheet LBLs. After each LBL activity,
all students completed the same preclass online quiz.
Each LBL homework assignment was associated with an
in-class active-learning exercise, where students were pre-
sented with an interactive demonstration, clicker questions,
problem-based small group discussion, and/or class discus-
sion. Novel research findings on the topic were also presented
moving the basic textbook level knowledge to the LBLs. The
overall experimental design is summarized in Table 1.
LBL Example: Organelle LBL5: Nucleus and
LBL learning goals are to be able to:
• Describe the basic structure and function of the nucleus
• Describe the role of nuclear pores in transport across
• Describe the relationship between free and bound ribo-
somes and proteins they synthesize
In-class Activity on Nuclear Pores (5 min)
• Clicker questions on transport through nuclear pores requir-
ing students to apply basic understanding of nuclear pore
trafficking to consequences of disrupting pore function.
• Tie this to data from a recent article in Cell (D’Angelo et
al., 2009) reporting that as cells age, the lack of renewal of
some nuclear pore components leads to loss of pore func-
tion. Nuclei in older cells become leaky and proteins nor-
mally excluded are found in the nucleus.
In-class Activity on Free versus Bound Ribosomes (5 min)
• Class demonstration that includes physical representation
of mRNAs, ribosomes in the cytosol, and ribosomes at-
tached to the rER. Student volunteers are asked to illus-
trate how the ribosomes interact with mRNAs encoding
secreted and cytosolic proteins.
• Students answer clicker questions (old exam questions)
about which ribosomes are synthesizing proteins that are
either cytosolic or secreted/transmembrane.
Matched Question Pairs
The LBLs were spread throughout the quarter and ad-
dressed a span of biological topics (organelles, inheritance,
and cloning). To determine the efficacy of LBLs in improv-
ing learning, we included questions on the final exam that
specifically addressed material associated with the LBLs.
The experiment included six question pairs: five multiple
choice and one free response. One question pair was iden-
tical between years (restriction enzymes) but the rest were
isomorphic variants between years, matched by topic and
presentation (Smith et al., 2009). One question pair (blood
type) was nearly identical in terms of the wording of the
stem and the answer choices. The ribosome question pair
had one variant in 2008 that was compared with two iso-
morphic variants in 2009, one very similar and the other
with changes in stem and answers choices. The remaining
three question pairs (multiple alleles, plasmids, and nuclear
transport) had changes in both the stem and the answer
choices but the thought process required to answer the
question correctly was the same for each question in the
pair. The question pairs were also matched by level on
Bloom’s scale [i.e., Knowledge (1), Comprehension (2), Ap-
plication (3); Bloom et al., 1956; Crowe et al., 2008]. Bloom’s
level rankings were based on consensus among three indi-
viduals either teaching or associated with the course, in
addition to the author of each question.
Isomorphic Ribosome Questions (see Supplemental
Material Appendix A for Complete List of Question
Fall 2008 Section A
A free ribosome that binds to an mRNA molecule coding for
a lysosomal proton pump in the lysosome membrane will:
a. cleave off the signal peptide region before starting protein
b. bind to the ER and synthesize the protein directly into
Table 1. Summary of LBL exercises and associated in-class activities
LBL Lecture ALecture B In-class activities Exam question topicsBloom’s
Organelle WorksheetVideo Recent research; interactive
LBL clicker activity; interactive
Real-world application; interactive
(1⁄2 CO,1⁄2 ID)
(1⁄2 CO,1⁄2 ID)
Bloom’s Level Comprehension ? 2 and Application ? 3. ID, Interpreted a drawing of a plasmid; CO, cut-out and constructed plasmid.
Learn before Lecture
Vol. 9, Winter 2010 475
c. bind to the ER and synthesize the protein into the rER
d. synthesize the protein in the cytosol and transport it to
e. synthesize the protein in cytosol and package it in vesicles
for transport to lysosome
Fall 2009 Section A
A free ribosome that binds to an mRNA molecule coding for
a potassium channel that is located in the membrane of an
a. synthesize the protein in the cytosol and insert it directly
into the axon membrane
b. synthesize the protein in the cytosol, package it in a
vesicle, and transport to axon
c. cleave off the signal peptide region before starting protein
d. bind to the ER and synthesize the protein into the rER
e. bind to the ER and synthesize the protein directly into the
Fall 2009 Section B
A proton pump molecule that is synthesized for addition to
a lysosome membrane will be made by:
a. a free ribosome which will bind to rough ER and insert it
into the rough ER membrane
b. a free ribosome and inserted directly into a mitochon-
c. a free ribosome and released into the cytosol
d. a bound ribosome on the rough ER and delivered for
exocytosis by a kinesin
e. a free ribosome which will then bind the rough ER and
insert it into the lumen
The Point Biserial correlation (PBS) was calculated post-
hoc as a quantitative measure of the consistency of each
LBL-related exam question with the whole test (Ding et al.,
2006). The PBS can vary between ?1 and 1. A positive PBS
indicates that students with high total scores are more likely
to answer the question correctly than students with low total
scores. A negative PBS indicates that student with low over-
all scores were more likely to get the correct answer and
implies potential problems with the wording of the ques-
tion. In general, values above 0.2 are considered a good
indication of a well-written exam question (Ding et al., 2006).
The PBS was calculated as:
where¯X1is the average total score for those students who
answered correctly,¯X is the average total score for all stu-
dents, ?xis the SD of the total score for all students, and P is
the proportion of correct responses. For all topic-matched
multiple-choice questions, the PBS values were above 0.3,
indicating that the questions in each pair were well matched
in their ability to discriminate stronger students from
weaker students (Supplemental Material, Appendix A).
PBS ?X?1? X?
1 ? P
Assessment of Learning Gains
The percentage of students who correctly answered each of
the six matched exam question pairs was compared between
2009 (with LBLs) and 2007/8 (no LBLs) students. Students
who did not complete an LBL assignment in 2009 were
omitted from the analysis of performance on the associated
exam questions. Average scores on all non-LBL exam ques-
tions were also compared between 2007/8 and 2009. The
LBL-related exam questions were worth ?8% and the non-
LBL exam questions were worth ?92% of the total exam
points in each year.
Student Participation and Feedback
The percentage of students who submitted complete LBL as-
signments (and LBL-related quizzes) was tracked throughout
were posted, students were asked to complete an anonymous
online survey to share their opinion about various aspects of
the course, including LBLs.
Graph Pad InStat software (v3.1a) was used for statistical
analysis. Specific tests applied for each comparison are de-
scribed along with levels of significance in the text or figure
To assess the effectiveness of moving several content seg-
ments covered in class in 2007/8 to preclass LBLs in 2009,
we first evaluated level of participation in the LBL. Both the
worksheet and video LBLs required completion and upload-
ing of an assignment. We found that a small point reward
(?1% of the final grade) was sufficient motivation for at
least 90% of the students to upload the assignments for each
LBL (Table 2).
Comparison of Student Learning with and without
Performance on Matched Exam Question Pairs
To determine whether LBL activities affected learning out-
comes we included six questions on the 2009 final exams,
two related to each of three LBL topics. Of these, five were
multiple-choice questions: one that was identical and four
that were isomorphic to questions from the 2007 or 2008
final exams. The percentage of students who correctly an-
Table 2. Percentage of students awarded points for completing LBL
activities and online quizzes of a total of 771 students
M. Moravec et al.
CBE—Life Sciences Education476
swered the multiple-choice questions for each topic was
significantly higher for students in 2009 compared with
2007/8 (Fisher’s exact test, p ? 0.001; Figure 1). The perfor-
mance differences were seen on question pairs at both the
level of comprehension (multiple alleles and restriction en-
zymes) and application (ribosomes, blood type, nuclear
transport; Figure 1).
The sixth isomorphic question pair was an application-
level short answer (plasmids). This question was worth five
points on the 2008 and 2009 final exams. There was no
significant difference in performance between the two years
with mean scores of 4.0 ? 0.04 (n ? 854; 2008) compared
with 3.9 ? 0.05 (n ? 709; 2009) (Mann–Whitney U, p ? 0.7).
The mean increase in percentage correct calculated for the
six matched questions pairs was 21.3 ? 7.5% (% correct
2009 ? % correct 2007/8).
Overall Academic Performance
To explore the possibility that differences in composition,
ability, and/or preparation of the 2009 compared with the
2007/8 students contributed to the significant differences in
performance on LBL exam questions, we further character-
ized the three cohorts. The preclass academic indices of the
cohorts were similar (Table 3). There was no difference in
performance on the preclass concept assessment between
2008 and 2009 (Table 3; 2007 was not given this test). In
addition, the SAT scores were not different across the
groups with the exception of verbal scores in 2007 which
were 3% lower than 2009. The demographics of the groups
were also similar, with no significant difference in gender
balance or ethnicity (Table 3). There were small variations in
the percentage of freshmen and biology majors from year to
Because the LBL-related exam questions accounted for
?8% of the total exam points, the majority of the exam
points were related to material presented to students in
the same manner (by the same faculty) in all three years.
Therefore we also compared the percentage of total non-
LBL exam points earned to determine whether the 2009
students were consistently outperforming the 2007/8 stu-
dents. This was not the case as the percentage of the
non-LBL exam points earned in 2009 was 2.6% higher than
2008 but 7.5% lower than 2007 (Figure 2). This range is
typical of the year-to-year variability due to slight differ-
ences in the difficulty of the non-LBL questions that are
changed each year.
The large and significant increase in mean performance on
the LBL-related matched questions pairs (21%) in contrast to
the ?3% increase in exam performance on non-LBL ques-
tions, and similarity in preclass academic indices and com-
position of the 2007, 2008, and 2009 cohorts, indicate the
majority of the increase in performance is associated with
LBL-related learning gains.
Student performance in 2009 (with LBLs) was significantly higher than
2007/8 (no LBLs) on four isomorphic and one identical (restriction
enzymes) multiple-choice exam question pairs. Multiple Alleles and
Restriction Enzyme question were Bloom’s level 2, comprehension.
Blood type, Ribosomes, and Nuclear Transport questions were
Bloom’s level 3, application. Percent correct represents the number of
students who answered each question correctly of the total number of
students who answered each question. *** p ? 0.001, Fisher’s exact test.
The total number of students answering each question is indicated by
numbers within each bar. These vary because there were two sections
in 2008 and 2009, and not all questions were included on the final
exams for each section. In addition, students who did not complete an
LBL for a particular topic were not included in the analysis of the
related LBL exam questions.
Increase in student performance associated with LBLs.
Table 3. Academic and demographic data for 2007 (Section A),
2008 (Sections A and B), and 2009 (Sections A and B)
(means ? SEM)
% Biology majors
% East Asian
— 42 ? 248 ? 3 T; p ? 0.16
556 ? 4* 565 ? 3 571 ? 3 K; p ? 0.01
614 ? 4 620 ? 3 626 ? 3 K; p ? 0.08
C; p ? 0.27
C; p ? 0.01
C; p ? 0.01
514751C-3 ? 5;
p ? 0.31
% Latino or Hispanic
% Other ethnicity
Tests used for comparisons: T, Student’s t test (two-tailed); K,
Kruskal-Wallis with Dunn’s Multiple Comparison’s test; C, ?2;
C-3 ? 5, ?2. Post hoc test p values were accepted as significant based
on a Bonferroni correction: * indicates significant difference between
2009 and 2007. Concept assessment reflects mean percent correct on
17 identical questions answered in a preclass online quiz by stu-
dents in 2008 and 2009. Demographic data and SAT scores provided
by the registrar without student identifiers from 2007 (n ? 438),
2008 (n ? 872), and 2009 (n ? 795).
Learn before Lecture
Vol. 9, Winter 2010 477
Comparison of Different LBL Formats on Exam
Performance in 2009
We hypothesized that worksheet activities requiring stu-
dents to read, answer questions, and/or draw diagrams
would result in higher learning gains than watching and
taking notes on a narrated PowerPoint video covering the
same material. To test this, we used a cross-over design
using the LBL on inheritance and the LBL on organelles.
There was no difference in performance on the exam ques-
tions between students assigned the video version of an LBL
and those assigned the worksheet version (Figure 3; Fisher’s
exact test, p ? 0.1). This indicates that both methods of
preclass exposure produce similar learning gains.
For the Cloning LBL, we compared the performance of
students who completed the drawing worksheet to those
who completed the cut-out worksheet, on a free response
exam question that required understanding of restriction
mapping (plasmids). Of five possible points, the mean scores
for the drawing (3.9 ? 0.07, n ? 368) and the cut-out groups
(3.9 ? 0.08, n ? 341) were not significantly different (Mann-
Whitney U, p ? 0.69), indicating that both methods are
equally effective for student learning.
Student Perception and Behavior
Students were asked to fill out a postclass anonymous sur-
vey to gather information on a variety of issues related to the
class including their experience with the LBLs. Although
there was no reward, 56% of the 2009 students (446/795)
completed the survey.
The majority (80%) of students who responded indicated
LBLs were helpful in learning the course material (Figure 4).
In addition, 73% of the respondents indicated they viewed
the LBL material one or more times after the assignment had
been turned in. Interestingly, when asked which LBL was
more effective in helping them learn the information, 50%
(220/446) indicated the worksheets and 50% (218/446) indi-
cated the video format. Students were also given an oppor-
tunity to provide additional comments about the LBLs. Of
those who added comments (82/446), some were negative
(13%), most commonly expressing frustration with the lo-
gistics of LBL submission. However, the majority (74%) of
comments were positive and pointed out the usefulness of
LBLs in preparing for lecture (e.g., “I thought it really gets
your mind warmed up before you have a lecture about
it”). Other comments indicated students felt the LBLs
helped them keep up with, review, and understand the
Our data demonstrate inclusion of LBLs that shifted presen-
tation of new material from lecture to preclass assignments
non-LBL exam questions. Distributions represent the percentage of
students who earned the indicated percentage of non-LBL exam
points in each year. Data were binned in 2% intervals. The mean in
2009 (61.3 ? 0.6%, n ? 771) was 2.6% higher than in 2008 (58.7 ? 0.5,
n ? 854) and 7.6% lower than in 2007 (69 ? 0.8, n ? 433) (p ? 0.01,
Dunn’s multiple comparisons tests, Kruskal–Wallis).
Performance of students in 2007, 2008, and 2009 on
gains. Comparison of performances on identical 2009 exam ques-
tions between video and worksheet LBL versions. Percent correct
represents the number of students who answered each question
correctly of the total number of students who answered each ques-
tion. The Multiple Alleles question was a Bloom’s Level 2, compre-
hension question. The Ribosome and Nuclear Transport questions
were Bloom’s level 3, application. There was no significant differ-
ence between video and worksheet LBLs for Multiple Alleles, Ribo-
somes, or Nuclear Transport questions (p ? 0.1, Fisher’s Exact Test).
Total number of students answering each question is indicated by
numbers within each bar.
Worksheet and video LBLs produced similar learning
ing course material. Graph shows percentage of students (of a total
of 446 respondents) who selected each of the indicated response
categories on an anonymous postclass survey.
Students indicate LBL assignments were helpful in learn-
M. Moravec et al.
CBE—Life Sciences Education478
but did not require instructor review, coupled with related
participatory exercises in class, resulted in significant in-
creases in student learning gains in a large introductory
biology course. Our findings also indicate that the learning
gains associated with LBLs did not come at the expense of
reduced learning outcomes on non-LBL material. The com-
bination of increased exam performance on LBL-related top-
ics and the high level of student satisfaction encourages us to
continue their use in our class.
Assessment Using Matched Pairs of Final-Exam
The LBLs in 2009 were used for only a small number of
topics, while the rest were taught with the same strategies
used in the previous years. This allowed us to examine the
effectiveness of LBLs, embedded in the same overall class
environment. While it is common to assess learning gains by
comparing student performance on identical questions be-
fore and after a new teaching strategy, these are most often
administered as ungraded surveys at the beginning and the
end of a class (Metz, 2008; Perry et al., 2008; Scholer and
Hatton, 2008). This was not possible because our study was
conducted over three years with different classes. In addi-
tion, we wanted to evaluate both LBL and non-LBL course
related learning outcomes under conditions where the per-
formance expectations and motivation were consistent, so
comparisons were all based on performance on exam ques-
tions. Our standard practice of posting answer keys to all
exams precluded extensive use of identical questions from
year to year. Therefore only one question pair was identical
and the other five were isomorphic, closely matched by
topic, presentation, and Bloom’s level (Smith et al., 2009). A
post hoc analysis also showed that all questions in each pair
were well matched in terms of their ability to discriminate
stronger from weaker students based on PBS values of 0.3 or
higher (Ding et al., 2006). This strongly suggests that the
large mean increase in performance on the matched final
exam question pairs that we report in this study reflects
increases in student learning associated with the LBLs. Fur-
ther support for this conclusion is that there was little or no
increase in the performance on non-LBL exam questions
between 2009 and the previous years, and the cohorts in all
years were similar in terms of their demographics and pre-
class academic indicators.
Choosing Material to Convert to an LBL
Our introductory biology course covers material from DNA
to organisms in 10 wk. The goal of moving material out of
lecture was not to increase the amount of material covered,
as the content is already broad and challenging. Instead, we
selected topics where the basic knowledge was explained
well in the textbook and developed preclass assignments
that guided students through the reading and interpretation
of diagram/figures. The LBL-related questions associated
with the largest performance increases were multiple alleles
(comprehension level) and nuclear transport (application
level). Historically these have been difficult topics for stu-
dents, and modifications in presentation of the material in
lecture over multiple years had resulted in little change in
understanding. The only LBL-related question (plasmids)
that did not result in a significant increase in performance
was one that the majority (80%) of students got correct
before LBLs were introduced, consistent with the interpre-
tation that presentation of this material in lecture was al-
ready quite effective. Our data thus indicate that the LBL
strategy is most effective for teaching core biological con-
cepts that are challenging for students to learn in a tradi-
tional lecture format. The increase in performance on both
comprehension- and application-level questions suggests
that exposure to new material before class along with revis-
iting this material during the more in-depth exploration of
the core concepts in class contributes to the learning gains.
Worksheet and Video Format LBLs
Previous studies have used homework assignments similar
to our worksheets and online video lectures to introduce
students to new material before class (Walvoord and Ander-
son, 1998; Collard et al., 2002; Klionsky, 2002; Keefe, 2003;
Allen and Tanner, 2005; Day and Foley, 2006). However,
there is little information regarding the comparative effec-
tiveness of these methods on student learning. In our study,
we found no difference in exam performance between stu-
dents who completed a worksheet LBL and those who
viewed a narrated video LBL on the same topic. This was
somewhat surprising because we predicted the worksheet
assignments would be more effective as they required more
active participation including writing answers to questions
and/or making drawings. We also saw no difference in
learning between those who completed the cut-out and
drawing versions of the third LBL assignment, again con-
trary to our prediction that actually making a model would
be more effective in reinforcing learning than analysis of a
The data demonstrate that both methods of preclass ex-
posure to the material are equally effective in producing
learning gains when coupled with the same in-class activi-
ties. However, we were unable to determine what percent-
age of the learning gains were associated with the preclass
exposure as opposed to the in-class activity, as the combi-
nation was part of the experimental design. It would be
interesting to explore the relative contributions in our set-
ting because a previous study showed that active-learning
exercises coupled with online activities were more effective
than active exercises alone (Riffell and Sibley, 2005).
Online Submission of LBL Assignments
In large classes like ours, collection and assessment of as-
signments on paper is not feasible with limited administra-
tive support. Therefore, we chose to have all LBL assign-
ments distributed and submitted electronically. Online
assignments can be submitted in multiple formats, including
Word documents or online forms (Schaeffer et al., 2001;
Allain and Williams, 2006). Our submission procedure re-
quired uploading of a digital image to our class website
drop box, so we were able to assess whether students com-
pleted drawings and/or made cut-out models. More than
90% of the students were able to accomplish this task by
scanning or photographing their completed worksheets. A
small percentage of students uploaded Word documents. In
a postclass survey approximately 5% of the students re-
Learn before Lecture
Vol. 9, Winter 2010479
ported dissatisfaction with the submission procedure, in-
cluding difficulty in accessing a digital camera or scanner.
Virtually all of our students have cell phones with digital
cameras, the more recent versions of which were used by a
number of students to produce adequate images of their
assignments. In coming years, high-resolution cell phone
cameras will provide most students with a relatively simple
method to create digital images of their completed assign-
ments. It is also worth noting that web-based reading quiz-
zes are relatively simple to set-up and administer through
readily available online course management systems. Al-
though we hypothesize that students learn more when they
have to write out something by hand, it would be interesting
to test the relative efficacy of LBLs with completely elec-
tronic reading quizzes.
Many faculty resist integration of active learning into their
large courses because restructuring can be time-consuming,
implementation administratively challenging, and there is
often little support or reward for these activities, particularly
in large research universities (Allen and Tanner, 2005; Jus-
tice et al., 2009). Adoption of incremental changes rather
than an abrupt class overhaul has been advocated as a more
feasible strategy for moving in the direction of creating
learner-centered environments in large classes (Bonwell and
Eison, 1991; Allen and Tanner, 2005). Our study included
only a small number of LBLs to evaluate effectiveness of this
specific tool as opposed to the effects of a global change in
learning environment. The data show this strategy can be
implemented incrementally and still result in significant
learning gains. Additional studies will be needed to deter-
mine whether increasing the number of LBLs results in
similar learning gains for each topic. It is possible that mak-
ing LBLs a routine part of each lecture would reduce the
novelty and potentially the effectiveness. We have found
that including a wide variety of teaching strategies is helpful
in keeping the diverse range of students that populate our
introductory classes engaged throughout the quarter.
For implementation we recommend starting with a small
number of topics that students have had difficulty grasping
from traditional lectures in the past. In our experience, trans-
forming small lecture segments into LBL worksheet activi-
ties takes less time than making narrated videos, and be-
cause both formats are equally effective instructors should
choose the format that is easiest. Once the activities have
been developed and implemented, they can be reused
and/or revised in the same way faculty would update their
established lectures. Administrative burden can be kept to a
minimum when assignment distribution, submission, and
point allocation are done using standard electronic course
management systems. In class, we do not reteach the mate-
rial in the LBL but instead focus on exploring the concepts in
more depth through active engagement exercises that in-
volve the students such as clicker questions and/or interac-
tive demonstrations (O’Dowd and Aguilar-Roca, 2009). Fac-
ulty teaching introductory classes who are involved in the
generation of primary research data can also provide large
numbers of students with meaningful access to the prime
commodity of a research university, new knowledge not yet
widely available. This may be especially important in the
sciences where early exposure to basic research can lead to
improved student success (Seymour et al., 2007; Wischusen
and Wischusen, 2007). This approach, involving a relatively
small initial investment, can help busy faculty transform
large lecture halls into more active learning environments
that support increased student learning gains.
We thank Dr. Rahul Warrior for participating in creating and im-
plementing LBLs and all of our students in Bio 93 2007–2009. This
research was funded by a grant from the Howard Hughes Medical
Institute (HHMI) Professor Program (to D.K.O.).
Allain, R., and Williams, T. (2006). The effectiveness of online home-
work in an introductory science class. J. Coll. Sci. Teach. 35, 28–30.
Allen, D., and Tanner, K. (2005). Infusing active learning into the
large-enrollment biology class: seven strategies, from the simple to
complex. Cell Biol. Educ. 4, 262–268.
Bloom, B. S., Krathwohl, D. R., and Masia, B. B. (1956). Taxonomy of
Educational Objectives: The Classification of Educational Goals, 1st
ed, New York: David McKay Company, Inc.
Bonwell, C. C., and Eison, J. A. (1991). Active Learning: Creating
Excitement in the Classroom. Ashe-Eric Higher Education Report
No 1, Washington, D.C: George Washington University, School of
Education and Human Development.
Chaplin, S. (2009). Assessment of the impact of case studies on
student learning gains in an introductory biology course. J. Coll. Sci.
Teach. 39, 72–79.
Collard, D. M., Girardot, S. P., and Deutsch, H. M. (2002). From the
textbook to the lecture: improving prelecture preparation in organic
chemistry. J Chem. Educ. 79, 520–523.
Crowe, A., Dirks, C., and Wenderoth, M. P. (2008). Biology in
bloom: implementing bloom’s taxonomy to enhance student learn-
ing in biology. CBE Life Sci. Educ. 7, 368–381.
D’Angelo, M. A., Raices, M., Panowski, S. H., and Hetzer, M. W.
(2009). Age-dependent deterioration of nuclear pore complexes
causes a loss of nuclear integrity in postmitotic cells. Cell 136,
Day, J. A., and Foley, J. D. (2006). Evaluating a web lecture inter-
vention in a human-computer interaction course. IEEE Trans. Educ.
Ding, L., Chabay, R., Sherwood, B., and Beichner, R. (2006). Evalu-
ating an electricity and magnetism assessment tool: brief electricity
and magnetism assessment. Phys. Rev. ST Phys. Educ. Res. 2, 1–7.
Dobson, J. L. (2008). The use of formative online quizzes to enhance
class preparation and scores on summative exams. Adv. Physiol.
Educ. 32, 297.
Freeman, S., O’Connor, E., Parks, J. W., Cunningham, M., Hurley,
D., Haak, D., Dirks, C., and Wenderoth, M. P. (2007). Prescribed
active learning increases performance in introductory biology. CBE
Life Sci. Educ. 6, 132–139.
Hake, R. R. (1998). Interactive-engagement versus traditional meth-
ods: a six-thousand-student survey of mechanics test data for intro-
ductory physics courses. Am. J. Phys. 66, 64–74.
Johnson, B. C., and Kiviniemi, M. T. (2009). The effect of online
chapter quizzes on exam performance in an undergraduate social
psychology course. Teach. Psychol. 36, 33–37.
M. Moravec et al.
CBE—Life Sciences Education 480
Justice, C., Rice, J., Roy, D., Hudspith, B., and Jenkins, H. (2009).
Inquiry-based learning in higher education: administrators’ per-
spectives on integrating inquiry pedagogy into the curriculum.
High. Educ. 58, 841–855.
Keefe, T. (2003). Enhancing a Face-to-Face Course With Online
Lectures: Instructional and Pedagogical Issues. Proceedings of the
Annual Mid-South Instructional Technology Conference, Mur-
freesboro, TN: Middle Tennessee State University.
Klionsky, D. J. (2002). Constructing knowledge in the lecture hall. J.
Coll. Sci. Teach. 31, 246–251.
Knight, J. K., and Wood, W. B. (2005). Teaching more by lecturing
less. Cell Biol. Educ. 4, 298–310.
Marrs, K. A., and Novak, G. (2004). Just-in-time teaching in biology:
creating an active learner classroom using the internet. CBE Life Sci.
Educ. 3, 49.
Metz, A. M. (2008). Teaching statistics in biology: using inquiry-
based learning to strengthen understanding of statistical analysis in
biology laboratory courses. CBE Life Sci. Educ. 7, 317–326.
Michael, J. (2006). Where’s the evidence that active learning works?
Adv. Physiol. Educ. 30, 159–167.
Narloch, R., Garbin, C. P., and Turnage, K. D. (2006). Benefits of
prelecture quizzes. Teach. Psychol. 33, 109–112.
Novak, G. M., Gavrin, A., and Patterson, E. T. (2004). Just-In-Time
teaching. http://webphysics.iupui.edu/jitt/jitt.html, last modified
2006 (accessed August 2010).
Novak, G. M., Gavrin, A., and Wolfgang, C. (1999). Just-in-Time
Teaching: Blending Active Learning With Web Technology, Upper
Saddle River, NJ: Prentice Hall.
Novak, G. M., and Patterson, E. T. (2000). The Best of Both Worlds:
WWW Enhanced in-Class Instruction. Proceedings of the IASTED
International Conference on Computers and Advanced Technology
in Education, Calgary, AB: ACTA Press.
O’Dowd, D. K., and Aguilar-Roca, N. (2009). Garage demos: using
physical models to illustrate dynamic aspects of microscopic bio-
logical processes. CBE Life Sci. Educ. 8, 118–122.
Perry, J., Meir, E., Herron, J. C., Maruca, S., and Stal, D. (2008). Evalu-
trees through diagramming tasks. CBE Life Sci. Educ. 7, 193–201.
Riffell, S., and Sibley, D. (2005). Using web-based instruction to
improve large undergraduate biology courses: an evaluation of a
hybrid course format. Comput. Educ. 44, 217–235.
Schaeffer, E., Bhargava, T., Nash, J., Kerns, C., and Stocker, S. (2001).
Innovation From Within the Box: Evaluation of Online Problem Sets
in a Series of Large Lecture Undergraduate Science Courses. Meet-
ing of the American Educational Research Association.
Scholer, A. M., and Hatton, M. (2008). An evaluation of the efficacy of a
laboratory exercise on cellular respiration. J. Coll. Sci. Teach. 38, 40–45.
Seymour, E., Hunter, A., Laursen, S. L., and DeAntoni, T. (2007).
Establishing the benefits of research experiences for undergraduates in
the sciences: first findings from a three-year study. Sci. Educ. 88,
Simkins, S., and Maier, M. (2004). Using just-in-time teaching tech-
niques in the principles of economics course. Soc. Sci. Comp. Rev. 22, 444.
Smith, M. K., Wood, W. B., Adams, W. K., Wieman, C., Knight,
J. K., Guild, N., and Su, T. T. (2009). Why peer discussion im-
proves student performance on in-class concept questions. Sci-
ence 323, 122.
Walvoord, B. E., and Anderson, V. J. (1998). Effective Grading: A
Tool for Learning and Assessment, 1st ed, San Francisco, CA: Jossey-
Wischusen, S. M., and Wischusen, E. W. (2007). Biology intensive
orientation for students (BIOS): a biology “boot camp”. CBE Life Sci.
Educ. 6, 172–178.
Learn before Lecture
Vol. 9, Winter 2010481