ArticlePDF Available

Testing in the Age of Active Learning: Test Question Templates Help to Align Activities and Assessments

Article

Testing in the Age of Active Learning: Test Question Templates Help to Align Activities and Assessments

Abstract and Figures

Many undergraduate biology instructors incorporate active learning exercises into their lessons while continuing to assess students with traditional exams. To better align practice and exams, we present an approach to question-asking that emphasizes templates instead of specific questions. Students and instructors can use these Test Question Templates (TQTs) to generate many variations of questions for pre-exam practice and for the exams themselves. TQTs are specific enough to show students which material they should master, yet general enough to keep the exact exam questions a surprise and easy to change from term to term. TQTs generate biology problems analogous to other STEM disciplines' standard problems whose general format is known to students in advance. TQTs thus help instructors ask more exam questions at the higher-order cognitive levels of Bloom's taxonomy, while empowering students to prepare actively and creatively for such questions. https://doi.org/10.21692/haps.2020.006
Content may be subject to copyright.
74 • HAPS Educator Journal of the Human Anatomy and Physiology Society Volume 24, Issue 1 April 2020
continued on next page
Testing in the Age of Active Learning: Test Question Templates Help
to Align Activities and Assessments
Gregory J. Crowther,1 Benjamin L. Wiggins,2 and Lekelia D. Jenkins3
1Department of Life Sciences, Everett Community College, Everett WA; gcrowther@everettcc.edu
2Department of Biology, University of Washington, Seattle WA; benlw@uw.edu
3School for the Future of Innovation in Society, Arizona State University, Tempe AZ; kiki.jenkins@asu.edu
Abstract
Many undergraduate biology instructors incorporate active learning exercises into their lessons while continuing to assess students
with traditional exams. To better align practice and exams, we present an approach to question-asking that emphasizes templates
instead of specic questions. Students and instructors can use these Test Question Templates (TQTs) to generate many variations
of questions for pre-exam practice and for the exams themselves. TQTs are specic enough to show students which material
they should master, yet general enough to keep the exact exam questions a surprise and easy to change from term to term. TQTs
generate biology problems analogous to other STEM disciplines’ standard problems whose general format is known to students
in advance. TQTs thus help instructors ask more exam questions at the higher-order cognitive levels of Bloom’s taxonomy, while
empowering students to prepare actively and creatively for such questions. https://doi.org/10.21692/haps.2020.006
Key words: study guides, problem sets, rubrics
Introduction
Six years ago, a comprehensive meta-analysis argued that
active learning was unequivocally superior to traditional
lecturing in terms of student learning gains in STEM courses
(Freeman et al. 2014). There is growing acceptance of this
conclusion among biology and STEM faculty, as reected in
faculty surveys (Patrick et al. 2016) and the expectation of
many search committees that teaching demonstrations should
include active-learning techniques (Smith et al. 2013). The
actual implementation of active-learning techniques may lag
behind perceived best practices (Stains et al. 2018), due in part
to many instructors’ belief that they must cover more content
than active learning can accommodate (Miller and Metz 2014,
Silverthorn 2020).
The momentum of the active learning movement does not
address the fact that the term “active learning” encompasses
many distinct teaching and learning strategies (Bonwell and
Eason 1991). In fact, in the above-mentioned meta-analysis
(Freeman et al. 2014), active learning was operationally
dened to include almost any student activity other than
reading, listening, or verbatim copying of notes. This
diversity of active learning options is, overall, a good thing,
since dierent instructional goals may be served best by
dierent approaches (Tanner 2013). However, this diversity
also raises the question of how such learning can best
be demonstrated in summative assessments, especially
traditional comprehensive fact-based tests, which are the
primary determinant of students’ nal grades in most STEM
courses (Goubeaud 2010, Momsen et al. 2010).
As STEM educators, we are interested in the extent to which
these traditional assessments align with active learning as
currently practiced in undergraduate courses (Pellegrino
2006, Reeves 2006). While we are unaware of comprehensive
empirical data on this issue, we suspect that the active
learning movement has not reformed testing to the same
extent that it has reformed classroom lectures. Traditional
tests may not be an ideal inventory of the fruits of active
learning; for instance, if active-learning activities help
students improve their higher-order cognition (HOC), such
improvements may not be captured by typical undergraduate
biology tests, which consist mostly of lower-order cognition
(LOC) questions (Momsen et al. 2010). Improving summative
assessments to better align with class assignments is a current
major eort of national directives like the Next Generation
Science Standards (NGSS Lead States 2013).
In considering the alignment between learning activities
and subsequent tests, it is helpful to apply the principle
of backwards design (Wiggins and McTighe 2005), which
advises teachers to rst dene how they want their students
to demonstrate mastery in summative assessments, and
then to design learning activities that lead naturally to those
assessments. Like most teachers, we want our students to be
able to solve problems that go beyond the recognition and
recitation of specic facts (Songer and Kali 2014). Backwards
design would suggest that we identify appropriate problems
on which students can be assessed, and then give the students
numerous opportunities (e.g., classroom activities and
homework assignments) to practice such problems prior to
formal summative assessments. The practice should resemble
the assessment, though not so closely that the practice gives
away the exact nature of the assessment, which would then
allow students to pre-prepare memorized answers without
75 • HAPS Educator Journal of the Human Anatomy and Physiology Society Volume 24, Issue 1 April 2020
continued on next page
Testing in t he Age of Active Learning: Test Question Templates Help to Al ign Act ivities and Assessments
necessarily understanding them. Best practices indicate that it
is useful to create opportunities for students to do the work of
cognition both in performance and in their own independent
practice (Pellegrino 2014). Specically, the goal is to build
novel, ecient assessments that reveal students’ abilities to
operate at HOC levels of engagement, as described in Bloom’s
taxonomy (Crowe et al. 2008, Anderson et al. 2001).
Introducing Test Question Templates
Here we describe an approach, which we call Test Question
Templates (TQTs), for improving the alignment of learning
activities and summative assessments. The word “template” is
used to indicate that a TQT is not itself a question, but rather
a question generator. A TQT can be used to generate a large
number of distinct questions that conform to the template
format (discussed below). The questions generated are
suitable for tests as well as pre-test practice.
At its core, a TQT denes a relationship between an Input (the
information given to the student) and an Output (what the
student will do with the information given). For maximum
clarity and transparency, a TQT should also include an Example
(a specic case of an Input and the corresponding requested
Output), a Key (a correct answer for the Example), and Other
Answers (some imperfect responses along with notes on
scoring them). Table 1 illustrates the structure of a TQT with
an example in short-answer format; however, TQTs can just as
easily be used to create multiple-choice questions, as shown in
the footnote to Table 1.
TQT element Example of element Rationale
Input: Information
to be given to the
student.
“Given a table of the genetic code and a point mutation
(reported in terms of the DNA coding strand OR the DNA
template strand OR the mRNA)...
This example indicates that students
should know how to use a table of the
genetic code, and should also be able to
interconvert between the two DNA strands
and mRNA.
Output: What the
student will do with
the information
given.
... determine the likelihood that the mutation will aect
the function of the corresponding protein.
This example indicates that students should
be able to convert codons to amino acids
before and after the mutation, and thus see
whether the amino acid changes (to another
amino acid, or to a stop codon).
Example: A
specic case of a
possible question
in the Input/Output
format.
“Example: In an exon of the coding region of the coding
strand for a particular gene, most people have the
codon 5’-AGG-3’, but Jesse has codon 5’-CGG-3’. Is the
corresponding protein likely to function dierently in Jesse
than in most other people? Show your work and explain
your reasoning.”*
An Example is vital for helping students see
how a general Input-Output pair can be
translated into a specic test question. The
Example should ask for the same Output
format (e.g., short answer or multiple-choice
selection) as the test will.
Key: An answer to
the Example that
would earn full
credit.
“Old DNA coding strand codon AGG => mRNA codon AGG
=> amino acid Arg. New DNA coding strand codon CGG =>
mRNA codon CGG => amino acid Arg. Since the old codon
and the new codon both code for the same amino acid,
Arg (arginine), no change in protein function is expected.
In addition to being a rudimentary rubric,
students can use the Key for pre-test
practice. If possible, put the Key in a
separate le and encourage students to
work through the Example before checking
the Key.
Other Answers:
examples of
answers that would
earn partial credit,
with explanations of
the scoring.
Example A: The amino acid changes from Ser to Ala,
so function might be impaired.” [Shows understanding
that a change in amino acid may alter function, but the
conversions to amino acids were done wrong.]
Example B: “Even though the amino acid is Arg in both
cases, the change in DNA and RNA may cause problems.
[The codons were translated correctly, but the answer does
not convey that amino acid sequence dictates function.]
This optional TQT component can help
students recognize and avoid common
mistakes.
Table 1: The structure and function of a Test Question Template
*A multiple-choice version of this example might ask the same question and might provide choices like the following:
(A) The amino acid changes from Ser to Ala, so the protein’s function might be impaired.
(B) The amino acid changes to a stop codon, so the protein is likely nonfunctional.
(C) The change in DNA codon did not change the corresponding amino acid, so the protein’s function should remain the same. [correct]
(D) The change in DNA sequence might interfere with transcription.
Additional TQTs for sophomore-level A&P courses -- with multiple-choice and short-answer examples -- are provided in the Appendix
76 • HAPS Educator Journal of the Human Anatomy and Physiology Society Volume 24, Issue 1 April 2020
continued on next page
Testing in t he Age of Active Learning: Test Question Templates Help to Al ign Act ivities and Assessments
TQTs are given to students prior to tests, and thus constitute a
kind of study guide. In general, study guides and other practice
problems, whether delivered in class or not, are useful to students
in agging certain information as important (Lieu et al. 2017).
However, for a typical practice question, once students have
arrived at a good answer, it is unclear what additional steps (if
any) they should take to prepare for a test. Should they commit
this particular answer to memory? Should they think of variations
on the original question, and, if so, which aspects of the original
question should remain xed, and which should be varied? Or
should they just move on to the next question in order to get
through as many questions as possible?
TQTs avoid such dilemmas by explicitly showing students
the relationship between the practice questions and the test
questions. If an instructor delineates key course content in the
form of a TQT, the students can use the TQT to practice for the
test, and the instructor can use the same TQT to generate actual
test questions. Therefore, the alignment between the practice
and the test is excellent, even though the exact details of the
test questions are appropriately hidden from students and can
easily be changed in subsequent iterations of the course. This
close alignment should benet all students, but especially at-risk
groups such as rst-generation college students, who might
otherwise struggle to understand the instructor’s expectations
for tests (Wright et al. 2016). Furthermore, students who are
language learners are likely to benet from increasingly explicit
modes of practice similar in format to the assessments used
(Abedi, 2001).
Table 2 compares TQTs with two common study question formats:
the “fact check” and the “mini-essay. To be clear, we nd all three
formats useful. However, as detailed in Table 2, we believe that
TQTs have several advantages: they are arguably more specic
than mini-essays, more general than fact checks, and more
transparent and adaptable than either of these other formats.
Fact Check Mini-Essay Test Question Template
Description Students are asked to name one or
more correct specic facts.
Students are asked to write
a short correct narrative
(describe a process, compare
two things, etc.).
Given some specic information
(textual or graphical), students
answer a question about that
information.
Typical cognitive level in Bloom’s
taxonomy
1 (Knowledge),
2 (Comprehension)
2 (Comprehension),
3 (Application),
4 (Analysis),
5 (Synthesis),
6 (Evaluation)
2 (Comprehension),
3 (Application),
4 (Analysis),
5 (Synthesis),
6 (Evaluation)
Nervous System example (for
sophomore-level A&P)
What are the two general types of
Na+ channels found in the cell mem-
branes of neurons, and in which
parts of the neuron (dendrites, soma,
axon) is each type located?
Explain how an electrical
signal is passed from one
neuron to another.
Given a graph of membrane
potential versus time at the
axon hillock of a neuron, identify
the EPSPs, IPSPs, and action
potentials.
Cardiovascular System example
(for sophomore-level A&P)
List the cardiac structures
specialized for electrical conduction,
in the order that they are
depolarized.
Why do the pacemaker
cells of the heart depolarize
spontaneously? Explain in
terms of ion channels.
Given a specic alteration in the
cardiac conduction pathway,
explain which ECG time intervals
would be most aected, and in
which direction (increased or
decreased).
Specicity: Is it usually clear which
specic facts are needed to answer
the stated question?
Yes Maybe Yes
Generality: Does the question
usually highlight a theme of general
importance?
Maybe Yes Ye s
Transparency: Does the study
question familiarize students with
the format of test questions?
Maybe Maybe Yes
Adaptability: Does the question
usually permit many variations
appropriate for student practice and
testing?
No No Yes
Table 2: A comparison of Test Question Templates with conventional practice question formats
77 • HAPS Educator Journal of the Human Anatomy and Physiology Society Volume 24, Issue 1 April 2020
continued on next page
Testing in t he Age of Active Learning: Test Question Templates Help to Al ign Act ivities and Assessments
Tips for creating good TQTs
Those interested in the TQT format may wonder how they
might create TQTs suitable for their own courses.
TQTs may be created for any topic in which many possible
Inputs connect to many possible Outputs. Even a very
straightforward template – for example, “Given the name
of an organelle, summarize its function in a few words”
may qualify as a TQT according to the denition outlined in
Table 1, and may provide useful transparency to students.
However, our main interest is in TQTs that encourage HOC,
and that, therefore, are most appropriate for courses whose
Learning Objectives (LOs) go beyond the mastery of factual
details. If the LOs ask students only to identify, list, or describe,
verbs commonly associated with the Knowledge and
Comprehension levels of Bloom’s taxonomy, there will not
be many ways to ask about this information, and thus not
enough variations to generate TQTs that require HOC. On the
other hand, LOs that ask students to analyze, critique, interpret,
or predict, verbs associated with the upper levels of Bloom’s
taxonomy, may be more readily translated into TQTs that
require HOC.
An alternative to thinking in terms of LO verbs per se is the
following; just about all HOC involves combining or reconciling
two or more sources of information. Often, previously
mastered background information (obtained from “source 1”,
often the textbook or teacher) is used to interpret a brand-new
example (from “source 2”, often the test itself). For the example
in Table 1, background information on the genetic code (from
source 1) is juxtaposed with the specics of Jesse’s DNA (from
source 2). If we identify types of information that may be
juxtaposed in numerous interesting ways, such that students
need to think analytically, rather than simply memorizing the
outcomes of all possible combinations, we may be able to craft
a TQT that requires HOC.
An example of this “combinatorial” approach, showing how
several aspects of the integumentary system might be
combined in a TQT, is illustrated in Table 3. A conventional
study question might ask students about relationships
between UV light and other factors, with each relationship
considered independently. The question can be made into a
TQT by asking students to apply those known relationships to
a new-to-them scenario about a particular patient. Only in the
TQT version of the question do interesting interactions among
the factors emerge, e.g., high melanin levels are benecial
in that they protect folate supplies and protect against skin
cancer, but are undesirable in that they reduce endogenous
vitamin D production and thus calcium absorption (HHMI
Biointeractive 2015).
Conventional study question Test Question Template
Example
Indicate the impact (stimulatory, inhibitory,
or neither) that UV light has on skin cancer
incidence, melanin levels in the skin, rate
of endogenous vitamin D production, rate
of dietary calcium absorption, and plasma
folate levels.
Given the results of an interview and physical exam
of a patient -- including information on her natural
melanin levels, sunlight exposure, and dietary
(calcium, vitamin D, folate) habits and needs -- oer
her sound medical or nutritional advice.
Comments on example
This is a straightforward ve-part question:
what eect does UV light have on each of
ve other variables?
This template covers the same variables as the
conventional question, but is more interesting
because there are more than ve possible
combinations to consider.
Number of possible
variations of question Few Many
“Memorizability” of answers
to all possible questions High Low
Importance of reasoning
ability in answering the
question
Low High
Table 3: Test Question Templates may motivate students to pursue true understanding rather than pure memorization
78 • HAPS Educator Journal of the Human Anatomy and Physiology Society Volume 24, Issue 1 April 2020
continued on next page
Testing in t he Age of Active Learning: Test Question Templates Help to Al ign Act ivities and Assessments
While the TQT example in Table 3 is a clinical one, some clinical
issues cannot be readily translated into TQTs. For example, if
a sophomore-level physiology course covers only a couple of
neuromuscular disorders, it would not make much sense to
have a TQT of the form of, “given some symptoms, diagnose a
patient’s neuromuscular disorder.” And if a course mentions
numerous disorders, but only does so in passing, students
may not have the knowledge needed to think analytically
about the disorders. Thus, while clinical applications are
often interesting to students and good for stimulating critical
thinking, instructors who create clinical TQTs should take care
not to presume background knowledge or analytical skills that
their students do not yet have.
Whether clinically focused or not, most instructors probably
have existing practice questions that can be converted into
TQTs. The key is to recognize questions that represent specic
examples of a general pattern. For example, imagine that your
study guide asks students to calculate cardiac output from
an end-systolic volume of 60 mL, an end-diastolic volume of
140 mL, and a heart rate of 50 beats per minute. Presumably
you would like your students to be able to solve any similar
problem involving the cardiac output equation. This
expectation can be made more transparent by giving students
a TQT such as the following: “Given values for three of the
following four variables -- end-systolic volume, end-diastolic
volume, heart rate, and cardiac output -- solve for the fourth
variable.The original study-guide problem could then serve
as an example of a question generated by this TQT.
An additional consideration is that, ideally, a TQT would reect
the type of active learning activity that was originally used to
teach the content. For example, if students initially learned
about histology by examining microscope slides, an ideal TQT
might also involve the analysis of new (but related) microscope
slides. However, if this is not feasible, the TQT could instead
involve electronic images and/or text descriptions.
These last two examples -- the cardiac output calculation
and the histology images -- illustrate the fact that simple
mathematical analyses and gures are often an excellent basis
for TQTs. For equations and graphs, the exact numbers and
curve shapes can be varied endlessly, allowing students to get
unlimited practice on important mathematical relationships.
This also furthers the important goal of integrating more
quantitative skill development into biology (Brewer and Smith
2011). Likewise, for qualitative gures, there are numerous
versions that students have not seen before, yet should be
able to analyze after previous experience with similar gures.
A variation on the use of qualitative gures is the following,
which we have borrowed from the “Public Exam” system
(Wiggins 2019). Give students a specic gure to study in
advance, often a complex but important one (e.g., of the blood
clotting cascade), along with examples of questions that could
be asked about the gure. Assure students that they will have
a copy of the gure during the test. The clear message to
students is that this gure should be understood in depth, but
does not need to be memorized. The advantage to instructors,
in this case, is that they do not need to search for novel gures;
they can simply use existing gures that are already central to
their lessons.
Supporting students’ use of TQTs
Many undergraduate students think of biology as an endless
series of specic, unique questions, each with its own specic,
unique answer. Since TQTs, by contrast, involve more general
patterns of questions, students deserve explicit guidance on
this alternative approach. If TQTs will be used to generate test
questions, they should be introduced and explained during
formative assessments. Cooperative learning formats such as
think-pair-share (Mazur 1996), and jigsaws such as those used
in Theobald et al. 2017, can be especially benecial because
students can help each other with the format of the questions,
as well as the relevant content. Students who encounter TQTs
early and often will be well-equipped to handle TQT-generated
questions on actual tests.
Even if the TQT format is explained carefully, with examples,
many students may simply study the instructor-provided
examples without creating their own additional examples.
However, such students would miss a primary advantage of
TQTs, namely, the opportunity to create their own additional
practice questions. For this reason, students should be
explicitly assigned the task of creating novel TQT-based
questions. This assignment can benet from a group-learning
context; students can check each other’s creations, try to solve
the ones that seem most plausible as test questions, and grade
each other’s answers. Instructors should monitor these eorts,
if possible, to ensure that no group is straying too far from the
original intent of each TQT.
Preliminary student feedback on TQTs
In order to examine student views on topics covered in this
paper, we administered a brief questionnaire to students who
had taken a course that used TQTs. (The Everett Community
College IRB armed that this survey was exempt from formal
IRB review.) Of the 76 students enrolled in two courses, 35
opted to read this manuscript and respond to the questionnaire.
The questionnaire consisted of three open-ended questions:
1. In your view, what are the main CLAIMS (central
assertions or arguments) being made by this paper?
Please list at least two claims.
2. To what extent do you agree or disagree with the claims
listed in the previous response? Please be assured that it
is OK to disagree! Explain why you agree and/or disagree
with each claim.
3. Aside from the claims covered by the previous two
questions, what other comments (if any) do you have
about this paper?
79 • HAPS Educator Journal of the Human Anatomy and Physiology Society Volume 24, Issue 1 April 2020
continued on next page
Testing in t he Age of Active Learning: Test Question Templates Help to Al ign Act ivities and Assessments
We coded students’ responses to the rst question as high,
moderate, or low with regard to their level of comprehension
of the manuscript. We excluded from the full analysis
any student whose responses reected a low level of
comprehension, in that the response did not identify a claim of
the paper or lacked sucient detail to assess comprehension.
We then coded the remaining 30 student responses with
the qualitative data analysis software program, MAXQDA,
applying a grounded theory approach to text analysis (Strauss
and Corbin 1998), which is an iterative inductive process of
coding concepts that arise from the data and linking these
concepts into themes. Our analysis yielded four themes which
consistently arose across student responses.
The most general of these four themes was the idea that
active learning in general (not necessarily TQTs in particular) is
helpful for student learning. For example, one student wrote,
“We need more active learning in classrooms so students
better understand material and not just memorize facts.
Two additional themes centered more specically on TQTs.
These themes were, rst, that TQTs especially help students
learn new material more deeply, and, second, that TQTs
are useful for reviewing previously covered material in
preparation for exams. The rst theme was exemplied by
student comments such as the following: “Incorporating TQTs
… will better promote active learning in students by not just
memorizing facts given, but to further understand the context
and apply it to deeper thinking problems.The second theme,
focusing more on exam preparation, was echoed in comments
such as, “As a student, when I have sample question [sic] that
include the same material that will be on the exam I study
more and make sure that I actually know the material even
if the sentence gets rearranged or numbers get changed. It
makes it easy for students to know exactly what to study and
focus more on, instead of studying the whole book. I think
its [sic] really unrealistic when a teacher makes you study
absolutely everything they covered in class.
All of the students agreed in part or in whole with the claims
they identied in the manuscript. The few who agreed
only in part largely expressed an additional theme, namely,
that students may struggle with how to use TQTs when rst
introduced to them. One representative comment was the
following: TQTs can be good as long as students understand
how to solve them and understand them. And so I believe
for TQTs to work, it would need to be integrated in school
learning…. It can be very frustrating and confusing when
given something like a TQTs question on the exam and you
never came across that type of question before.” This insight,
that even an otherwise helpful format such as a TQT can be
confusing in the absence of repeated exposure, underscores
our suggestion above that students be given explicit,
extensive training in TQTs well before the rst exam.
Other approaches
As noted earlier, promoting HOC, both during personal
practice and during summative assessments, is considered
an educational best practice (Pellegrino 2014). The literature
in developing assessments that promote higher-order
thinking and learning in the K-12 system is rich (e.g., Darling-
Hammond and Adamson 2014). TQTs thus represent one
of many possible ways of aligning practice and assessment
of HOC. Analogous approaches in elds that habitually use
algorithmic or equation-based calculations may routinely
do TQT-like work without explicitly stating it. Specically,
physics and engineering exams commonly include questions
on the application of formulae to physical problems. The TQT
framework may help to make the design of these assessments
more transparent for students. Learning progressions are
an eective way to formalize assessment-ready pieces of
curriculum by investigating how students proceed towards
mastery (Scott et al. 2019). Development of learning
progressions is laborious but highly useful and will achieve
goals overlapping with those of TQTs. Other formulations
of exam-presentation tools for students, such as the “Public
Exam” system (Wiggins 2019), may also be useful to educators
looking to move beyond traditional exam styles.
Summary
Biology instructors want to help their students go beyond
isolated facts to master fundamental, general patterns and
skills. In-depth active-learning activities are great for inspiring
deep learning, but students may not fully engage in these
activities if they do not see strong connections between these
learning activities and subsequent high-stakes exams. TQTs
represent an approach, newly formulated, yet similar to other
existing frameworks, to bridge the gap between practice
questions and exam questions. By improving the transparency
of instructors’ exam-writing, TQTs help instructors and students
avoid the cat-and-mouse game of “Whats going to be on the
exam?” and instead provide students with rich opportunities
for creative, analytical practice with the course’s most
important material.
Acknowledgments
Mentorship of GJC by LDJ was supported in part by a PALM
Fellowship (PI: Susan Wick, University of Minnesota). We thank
Susan Wick for encouragement and comments on a draft of
this paper, and Subin Hyun for illustrations in the Appendix.
We also thank Jonathan Pottle for feedback on GJCs ideas for
an “A&P manifesto,” which indirectly fueled the writing of the
present article.
80 • HAPS Educator Journal of the Human Anatomy and Physiology Society Volume 24, Issue 1 April 2020
continued on next page
Testing in t he Age of Active Learning: Test Question Templates Help to Al ign Act ivities and Assessments
About the Authors
GregCrowtherteaches human anatomy and human
physiology at Everett Community College. When time permits,
he studies the incorporation of the arts into STEM education.
Kiki Jenkins is based at Arizona State University, where she
teaches various interdisciplinary courses and conducts
research on marine conservation, marine technology, and the
role of science dance in social change.
Ben Wiggins is the Manager of Instruction at UW Biology,
where his research focuses on active learning in large
classrooms. He teaches in molecular biology, civilizational
biology, and science teaching methods.
Literature cited
Abedi J and Lord C. 2001. The language factor in mathematics
tests. Appl Meas Educ 14(3): 219-234.
Anderson LW, Krathwohl DR, Airasian PW, Cruikshank KA,
Mayer RE, Pintrich, PR, ... & Wittrock MC. 2001. A taxonomy
for learning, teaching, and assessing: A revision of Bloom’s
taxonomy of educational objectives, abridged edition.
White Plains, NY: Longman.
Bonwell CC and Eison JA. 1991. Active learning: creating
excitement in the classroom. ASHE-ERIC Higher Education
Reports. Washington, DC: The George Washington
University, School of Education and Human Development.
Brewer CA, Smith D. 2011. Vision and change in undergraduate
biology education: a call to action. Washington, DC:
American Association for the Advancement of Science.
Crowe A, Dirks C, Wenderoth MP. 2008; Biology in bloom:
implementing Bloom’s taxonomy to enhance student
learning in biology. CBE Life Sci Educ 7(4):368-381.
Darling-Hammond L, Adamson F. 2014. Beyond the bubble
test: How performance assessments support 21st century
learning. San Francisco, CA: John Wiley & Sons.
Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N,
Jordt H, Wenderoth MP. 2014. Active learning increases
student performance in science, engineering, and
mathematics. Proc Natl Acad Sci U S A 111(23):8410-8415.
Goubeaud K. 2010. How is science learning assessed at the
postsecondary level? Assessment and grading practices in
college biology, chemistry and physics. J Sci Educ Technol
19(3): 237-245.
HHMI Biointeractive. 2015. The biology of skin color. https://
www.biointeractive.org/classroom-resources/biology-
skin-color.
Lieu R, Wong A, Aserad A, Shaer JF. 2017. Improving exam
performance in introductory biology through the use of
preclass reading guides. CBE Life Sci Educ 16(3): ar46.
Mazur E. 1996. Peer instruction: A user’s manual. Boston:
Pearson.
Miller CJ, Metz MJ. 2014. A comparison of professional-level
faculty and student perceptions of active learning: its
current use, eectiveness, and barriers. Adv Physiol Educ
38(3): 246-252.
Momsen JL, Long TM, Wyse SA, Ebert-May D. 2010. Just the
facts? Introductory undergraduate biology courses focus
on low-level cognitive skills. CBE Life Sci Educ 9(4):435-440.
NGSS Lead States. 2013. Next generation science standards:
For states, by states. Washington, DC: National Academies
Press.
Patrick LE, Howell LA, Wischusen W. 2016. Perceptions of active
learning between faculty and undergraduates: Diering
views among departments. J STEM Educ Innov Res 17(3):
55-63.
Pellegrino JW. 2006. Rethinking and redesigning curriculum,
instruction and assessment: What contemporary research
and theory suggests. Chicago: National Center on
Education and the Economy.
Pellegrino JW. 2014. A Learning sciences perspective on the
design and use of assessment in education. In: Sawyer RK,
editor. The Cambridge handbook of the learning sciences.
Cambridge, UK: Cambridge University Press.
Reeves TC. 2006. How do you know they are learning? The
importance of alignment in higher education. Int J Learn
Technol 2(4): 294-309.
Scott EE, Wenderoth MP, Doherty JH. 2019. Learning
Progressions: An Empirically Grounded, Learner-Centered
Framework to Guide Biology Instruction. CBE Life Sci Educ
18(4): es5.
Silverthorn DU. 2020. When active learning fails … and what
to do about it. In: Mintzes JJ, Walter EM, editors. Active
learning in college science: The case for evidence-based
practice. Berlin: Springer, p. 985-1001.
Songer NB, Kali Y. 2014. Science education and the learning
sciences as coevolving species. In: Sawyer RK, editor.
The Cambridge handbook of the learning sciences.
Cambridge, UK: Cambridge University Press.
Smith MK, Wenderoth MP, Tyler M. 2013. The teaching
demonstration: what faculty expect and how to prepare
for this aspect of the job interview. CBE Life Sci Educ 12(1):
12-18.
Stains M et al. 2018. Anatomy of STEM teaching in North
American universities. Science 359: 1468-1470.
Strauss AL and Corbin J. 1998. Basics of qualitative research:
Techniques and procedures for developing grounded
theory. Thousand Oaks, CA: SAGE Publications, Inc.
81 • HAPS Educator Journal of the Human Anatomy and Physiology Society Volume 24, Issue 1 April 2020
Testing in t he Age of Active Learning: Test Question Templates Help to Al ign Act ivities and Assessments
Tanner KD. 2013. Structure Matters: Twenty-One Teaching
Strategies to Promote Student Engagement and Cultivate
Classroom Equity. CBE Life Sci Educ 12(3): 322-331.
Theobald EJ, Eddy SL, Grunspan DZ, Wiggins BL, Crowe AJ.
2017. Student perception of group dynamics predicts
individual performance: Comfort and equity matter. PLoS
ONE 12(7): e0181336.
Wiggins B. 2019. The Public Exam System: Simple Steps to
More Eective Tests. https://www.coursehero.com/faculty-
club/classroom-tips/benjamin-wiggins/.
Wiggins G, McTighe J. 2005. Understanding by Design [2nd
edition]. Alexandria, VA: Association for Supervision and
Curriculum Development (ASCD).
Wright CD, Eddy SL, Wenderoth MP, Abshire E, Blankenbiller
M, Brownell SE. 2016. Cognitive Diculty and Format of
Exams Predicts Gender and Socioeconomic Gaps in Exam
Performance of Students in Introductory Biology Courses.
CBE Life Sci Educ 15: 1-16.
Appendix Hyperlink
Follow the link below to view and download the pdf
appendix to this article.
https://www.hapsweb.org/page/crowtherappendix
Back to TOC
... Here our framework for promoting knowledge transfer is that of Test Question Templates. TQTs have been described in detail (15); in brief, a TQT directly bundles a learning outcome (LO) with multiple specific examples of how that LO might be assessed. While most courses come with LOs and practice questions, the links between them are often unclear to students, and links between practice questions and actual test questions are often murky as well. ...
... This lesson includes four TQTs (15), which provide opportunities for formative assessment while also foreshadowing subsequent summative assessment. Having an explicit template for possible test questions empowers students to keep practicing a given type of problem -creating their own additional examples, if desired -until they are satisfied with their understanding. ...
... The lesson might also be considered inclusive in the sense that TQTs potentially provide students with transparent alignment of learning activities and test questions (15). This transparency of expectations should be especially helpful to students who are unfamiliar or uncomfortable with high-stakes testing, students who lack test-savvy study partners, and students for whom reading and writing in English is difficult. ...
... The primary purpose of learning objectives is to explicitly indicate using measurable verbs and phrases what the student must do to demonstrate learning. It is essential to align practice and exams with learning objectives (Crowther et al., 2020). With these, students can solve problems beyond recognizing and reciting specific facts (Songer and Kali 2014). ...
Article
Full-text available
Competence of Filipino Secondary Science Teachers in Developing Self-Learning Modules. Objectives: This study investigates the competence of Filipino secondary science teachers in developing self-learning modules (SLMs) in terms of intellectual property compliance, learning competencies, instructional design and organization, instructional quality, assessment, and readability. Methods: Data were gathered from 68 Filipino teachers using simple random sampling and clustered sampling techniques. To analyze the data, frequency count and percentage, mean and standard deviation, and one-way Analysis of Variance (ANOVA) were utilized. Findings: Results revealed that science teachers were highly competent in developing SLMs across all six variables. Furthermore, there was a statistically significant difference in the competence in developing SLMs in terms of intellectual property compliance based on their highest educational attainment and in terms of the aspects of learning competencies, instructional design, and organization, and assessment based on their sex. Conclusions: Teachers' competence is essential in improving the quality of performance in the learning process. Abstrak: Kompetensi Guru IPA SMA di Filipina dalam Mengembangkan Modul Belajar Mandiri. Tujuan: Penelitian ini menyelidiki kompetensi guru IPA SMA di Filipina dalam mengembangkan modul belajar mandiri dalam hal kepatuhan kekayaan intelektual, kompetensi pembelajaran, desain dan pengorganisasian pembelajaran, kualitas pembelajaran, penilaian, dan keterbacaan. Metode: Data dikumpulkan dari 68 guru Filipina menggunakan teknik simple random sampling dan clustered sampling. Untuk menganalisis data, digunakan frekuensi dan persentase, mean dan standar deviasi, dan Analisis Varians satu arah (ANOVA). Temuan: Hasil mengungkapkan bahwa guru sains sangat kompeten dalam mengembangkan SLM di semua enam variabel. Selanjutnya, terdapat perbedaan yang signifikan secara statistik dalam kompetensi pengembangan SLMs dalam hal kepatuhan kekayaan intelektual berdasarkan pencapaian pendidikan tertinggi mereka dan dari aspek kompetensi pembelajaran, desain pembelajaran, dan organisasi, dan penilaian berdasarkan jenis kelamin mereka. Kesimpulan: Kompetensi guru sangat penting dalam meningkatkan kualitas kinerja dalam proses pembelajaran. Kata kunci: pendidikan IPA, modul belajar mandiri, kompetensi guru.
... These presentations not only attract potential Fellows but also provide instructors with specific, practical ways to embed AL in their courses. PALM projects also have been published in a variety of venues targeting instruction (23)(24)(25)(26)(27), including in peer-reviewed journals like CourseSource, The Journal of College Science Teaching, and Advances in Physiology Education. ...
Article
Full-text available
Perspectives A large body of data suggests that implementing active learning practices in a STEM classroom contributes to increased success in both achievement of student learning outcomes and retention of students. Despite these findings, significant barriers exist for instructors implementing active learning strategies in their undergraduate classrooms. These barriers can be effectively addressed by providing sustained support to instructors and postdoctoral trainees interested in implementing active learning strategies in their teaching practice. The Promoting Active Learning and Mentoring (PALM) network attains this objective by connecting instructors interested in learning more about active learning (Fellows) with individuals who have extensive expertise related to this practice (mentors). These facilitated connections occur in the form of active mentorship for a year or more, virtual journal clubs, and biannual gatherings of PALM Fellows and mentors. Here, we describe the foundation on which PALM was built and explain how a successful mentor-ship program can pave the way for educators to adapt and implement evidence-based practices like active learning in a college classroom.
Conference Paper
Character and learning outcomes are aspects of student competency achievement, which have an important position as benchmarks for educational development in schools. There are many steps taken by the teacher to improve students’ character and learning outcomes at school. This study aims to identify the enhancement of character and student learning outcomes by using a local wisdom-based neuroscience approach. The population of this study was taken at a high school located in Sima Regency. The research sample was 50 students of class X divided into two groups, namely the experimental class group and the control class group. The instrument used was in the form of test and non-test. The results showed that the character and student learning outcomes obtained a higher percentage value in the experimental class than those obtained in the control class. This shows that using a neuroscience approach based on local wisdom effectively improves student character and learning outcomes.
Article
Gross anatomy is considered by many the backbone of medical education. While learning anatomy has reputation of requiring mainly rote memorization, modern day anatomy education often involves instruction and assessment at cognitive levels that foster higher‐order thinking. In many instances, these higher‐order anatomical concepts are taught to graduate students in healthcare related fields, such as medicine. At this level, students are expected to apply and analyze anatomical information since that is what will ultimately be expected of them as professionals. In contrast, undergraduate anatomy education is typically more introductory in nature and often takes place in the setting of a large‐enrollment course that serves as a prerequisite for many health sciences degree programs. In this study, variables related to the assessment of higher‐order concepts in clinical anatomy were compared between first year medical students and undergraduate students enrolled in an upper‐level human gross anatomy course. Results demonstrate that undergraduate students perform lower than medical students overall, but the degree of difference in how they perform on higher‐ versus lower‐order questions is comparable. The most notable exception is on practical examinations, where undergraduate students tend to struggle more with applying and analyzing information. Exploration of additional variables provide insight into how the cognitive level being assessed affects the time it takes to answer a question and how different practical examination question types and formats influence student performance. Findings presented in this study have implications for designing anatomy courses and underscore the importance of blueprinting assessments.
Article
Full-text available
Vision and Change challenged biology instructors to develop evidence-based instructional approaches that were grounded in the core concepts and competencies of biology. This call for reform provides an opportunity for new educational tools to be incorporated into biology education. In this essay, we advocate for learning progressions as one such educational tool. First, we address what learning progressions are and how they leverage research from the cognitive and learning sciences to inform instructional practices. Next, we use a published learning progression about carbon cycling to illustrate how learning progressions describe the maturation of student thinking about a key topic. Then, we discuss how learning progressions can inform undergraduate biology instruction, citing three particular learning progressions that could guide instruction about a number of key topics taught in introductory biology courses. Finally, we describe some challenges associated with learning progressions in undergraduate biology and some recommendations for how to address these challenges.
Article
Full-text available
Active learning in college classes and participation in the workforce frequently hinge on small group work. However, group dynamics vary, ranging from equitable collaboration to dysfunctional groups dominated by one individual. To explore how group dynamics impact student learning, we asked students in a large-enrollment university biology class to self-report their experience during in-class group work. Specifically, we asked students whether there was a friend in their group, whether they were comfortable in their group, and whether someone dominated their group. Surveys were administered after students participated in two different types of intentionally constructed group activities: 1) a loosely-structured activity wherein students worked together for an entire class period (termed the ‘single-group’ activity), or 2) a highly-structured ‘jigsaw’ activity wherein students first independently mastered different subtopics, then formed new groups to peer-teach their respective subtopics. We measured content mastery by the change in score on identical pre-/post-tests. We then investigated whether activity type or student demographics predicted the likelihood of reporting working with a dominator, being comfortable in their group, or working with a friend. We found that students who more strongly agreed that they worked with a dominator were 17.8% less likely to answer an additional question correct on the 8-question post-test. Similarly, when students were comfortable in their group, content mastery increased by 27.5%. Working with a friend was the single biggest predictor of student comfort, although working with a friend did not impact performance. Finally, we found that students were 67% less likely to agree that someone dominated their group during the jigsaw activities than during the single group activities. We conclude that group activities that rely on positive interdependence, and include turn-taking and have explicit prompts for students to explain their reasoning, such as our jigsaw, can help reduce the negative impact of inequitable groups. © 2017 Theobald et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Article
Full-text available
There have been numerous calls recently to increase the use of active learning in university science, technology , engineering, and math (STEM) classrooms to more actively engage students and enhance student learning. However, few studies have investigated faculty and student perceptions regarding the effectiveness of active learning or the barriers to its implementation. Previous work surveying a single class in a single department has suggested that faculty and students have different perceptions of the effectiveness of active learning strategies and the barriers faculty face when implementing these teaching strategies. We expand on these previous findings by surveying a larger and more diverse sample of students and faculty in a college consisting of five departments. We find that students and faculty agree that active learning techniques are useful, effective, and should be implemented more widely, but disagree on the percentage of class time currently devoted to active learning. When we parsed the data by department, more nuanced perceptions became apparent. The perceived barriers to implementing teaching reform differed in importance by department and in some departments relatively few faculty had observed or used active learning. Our findings suggest that advocates of teaching reform must recognize that not all departments or institutions face the same barriers to implementing curricular changes.
Article
Full-text available
Recent reform efforts in undergraduate biology have recommended transforming course exams to test at more cognitively challenging levels, which may mean including more cognitively challenging and more constructed-response questions on assessments. However, changing the characteristics of exams could result in bias against historically underserved groups. In this study, we examined whether and to what extent the characteristics of instructor-generated tests impact the exam performance of male and female and middle/high- and low-socioeconomic status (SES) students enrolled in introductory biology courses. We collected exam scores for 4810 students from 87 unique exams taken across 3 yr of the introductory biology series at a large research university. We determined the median Bloom’s level and the percentage of constructed-response questions for each exam. Despite controlling for prior academic ability in our models, we found that males and middle/high-SES students were disproportionately favored as the Bloom’s level of exams increased. Additionally, middle/high-SES students were favored as the proportion of constructed-response questions on exams increased. Given that we controlled for prior academic ability, our findings do not likely reflect differences in academic ability level. We discuss possible explanations for our findings and how they might impact how we assess our students.
Article
Full-text available
Active learning is an instructional method in which students become engaged participants in the classroom through the use of in-class written exercises, games, problem sets, audience-response systems, debates, class discussions, etc. Despite evidence supporting the effectiveness of active learning strategies, minimal adoption of the technique has occurred in many professional programs. The goal of this study was to compare the perceptions of active learning between students who were exposed to active learning in the classroom (n = 116) and professional-level physiology faculty members (n = 9). Faculty members reported a heavy reliance on lectures and minimal use of educational games and activities, whereas students indicated that they learned best via the activities. A majority of faculty members (89%) had observed active learning in the classroom and predicted favorable effects of the method on student performance and motivation. The main reported barriers by faculty members to the adoption of active learning were a lack of necessary class time, a high comfort level with traditional lectures, and insufficient time to develop materials. Students hypothesized similar obstacles for faculty members but also associated many negative qualities with the traditional lecturers. Despite these barriers, a majority of faculty members (78%) were interested in learning more about the alternative teaching strategy. Both faculty members and students indicated that active learning should occupy portions (29% vs. 40%) of face-to-face class time.
Article
High-structure courses or flipped courses require students to obtain course content before class so that class time can be used for active-learning exercises. While textbooks are used ubiquitously in college biology courses for content dissemination, studies have shown that students frequently do not read their textbooks. To address this issue, we created preclass reading guides that provided students with a way to actively engage with the required reading for each day of class. To determine whether reading guide completion before class is associated with increased performance, we surveyed students about their use of reading guides in two sections of a large-enrollment (400+ students) introductory biology course and used multiple linear regression models to identify significant correlations. The results indicated that greater than 80% of students completed the reading guides before class and that full completion of the reading guides before class was significantly positively correlated with exam performance. Reading guides in most cases were used similarly between different student groups (based on gender, ethnicity, and aptitude). These results suggest that optional preclass reading guides may help students stay on track to acquire course content in introductory biology and thus result in improved exam performance.
Book
Performance assessment is a hot topic in school systems, and educators continue to analyze its costs, benefits, and feasibility as a replacement for high-stakes testing. Until now, researchers and policymakers have had to dig to find out what we know and what we still have to learn about performance assessment. Beyond the Bubble Test: How Performance Assessments Support 21st Century Learning synthesizes the latest findings in the field, and not a moment too soon. Statistics indicate that the United States is in danger of falling behind if it fails to adapt to our changing world. The memory and recall strategies of traditional testing are no longer adequate to equip our students with the skills they need to excel in the global economy. Instead teachers need to engage students in deeper learning, assessing their ability to use higher-order skills. Skills like synthesizing information, understanding evidence, and critical problem-solving are not achieved when we teach to multiple-choice exams. Examples in Beyond the Bubble Test paint a useful picture of how schools can begin to supplement traditional tests with something that works better. This book provides new perspectives on current performance assessment research, plus an incisive look at what's possible at the local and state levels. Linda Darling-Hammond, with a team of leading scholars, bring together lessons learned, new directions, and solid recommendations into a single, readily accessible compendium. Beyond the Bubble Test situates the current debate on performance assessment within the context of testing in the United States. This comprehensive resource also looks beyond our U.S. borders to Singapore, Hong Kong, and other places whose reform-mindedness can serve as an example to us.
Chapter
From teachers’ classroom quizzes and mid-term or final exams, to nationally and internationally administered standardized tests, assessments of students’ knowledge and skills have become a ubiquitous part of the educational landscape. Assessments of school learning provide information to help educators, administrators, policy makers, students, parents, and researchers judge the state of student learning and make decisions about implications and actions. The specific purposes for which an assessment will be used are important considerations in all phases of its design. For example, assessments that instructors use in classrooms to assist or monitor learning typically need to provide more detailed information than assessments whose results will be used by policy makers or accrediting agencies. Assessment to assist learning. In the classroom context, instructors use various forms of assessment to inform day-to-day and month-to-month decisions about next steps for instruction, to give students feedback about their progress, and to motivate students (e.g., Black & Wiliam, 1998; Wiliam, 2007). One familiar type of classroom assessment is a teacher-made quiz, but assessment also includes more informal methods for determining how students are progressing in their learning, such as classroom projects, feedback from computer-assisted instruction, classroom observation, written work, homework, and conversations with and among students - all interpreted by the teacher in light of additional information about the students, the schooling context, and the content being studied.