A preview of this full-text is provided by Springer Nature.
Content available from International Journal of Technology and Design Education
This content is subject to copyright. Terms and conditions apply.
Vol.:(0123456789)
International Journal of Technology and Design Education (2022) 32:1191–1205
https://doi.org/10.1007/s10798-020-09639-1
1 3
Learning byevaluating (LbE) throughadaptive comparative
judgment
ScottR.Bartholomew1 · NathanMentzer2· MatthewJones3· DerekSherman4·
SwetaBaniya5
Accepted: 11 November 2020 / Published online: 21 November 2020
© Springer Nature B.V. 2020
Abstract
Traditional efforts around improving assessment often center on the teacher as the evalua-
tor of work rather than the students. These assessment efforts typically focus on measuring
learning rather than stimulating, promoting, or producing learning in students. This paper
summarizes a study of a large sample of undergraduate students (n = 550) in an entry-level
design-thinking course who engaged with Adaptive Comparative Judgment (ACJ), a form
of assessment, as a learning mechanism. Following random assignment into control and
treatment sections, students engaged in identical activities with the exception of a 20-min-
ute intervention we call learning by evaluating (LbE). Prior to engaging in a Point Of View
(POV) creation activity, treatment group students engaged in LbE by viewing pairs of pre-
viously-collected POV statements through ACJ; in each case they viewed two POV state-
ments side-by-side and selected the POV statement they believed was better. Following this
experience, students created their own POV statements and then the final POV statements,
from both the control and treatment students, were collected and evaluated by instructors
using ACJ. In addition, qualitative data consisting of student comments, collected during
ACJ comparisons, were coded by the researchers to further explore the potential for the
students to use class knowledge while engaging in the LbE review of peer work. Both the
quantitative and qualitative data sets were analyzed to investigate the impact of the LbE
activity. Consistent with other ACJ research findings, significant positive learning gains
were found for students who engaged in the intervention. Researchers also noted that these
findings did not indicate the actual quality of the assignments, meaning the while students
who engaged in the LbE intervention were better than their peers, they were not necessar-
ily “good” at the assignment themselves. Discussion of these findings and areas for further
inquiry are presented.
Keywords Adaptive comparative judgment· Assessment· Design-thinking· Learning by
evaluating· Peer assessment
* Scott R. Bartholomew
scottbartholomew@byu.edu
Extended author information available on the last page of the article
Content courtesy of Springer Nature, terms of use apply. Rights reserved.