Scott Bartholomew

Scott Bartholomew
Brigham Young University - Provo Main Campus | BYU · School of Technology

PhD. Curriculum & Instruction
Teacher Educator, STEM Researcher, Fulbright Scholar

About

76
Publications
33,673
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
418
Citations
Citations since 2016
72 Research Items
414 Citations
2016201720182019202020212022020406080100120
2016201720182019202020212022020406080100120
2016201720182019202020212022020406080100120
2016201720182019202020212022020406080100120
Introduction
Scott Bartholomew currently works at Brigham Young University in the Technology & Engineering Studies program. Scott does research in Technology & Engineering Teacher Education and prepares future educators. Scott's research interests include Adaptive Comparative Judgment, Design Assessment, Design Portfolios, and Teacher Preparation.
Additional affiliations
September 2013 - present
Utah State University
Position
  • Technology & Engineering Education Teacher
Description
  • Communication Technologies Computer-Aided Drafting & Design Architectural CAD Energy, Power, & Transportation Technologies

Publications

Publications (76)
Article
Full-text available
Developing computational thinking (CT) skills at an early age can help develop literacy, science, and mathematics skills; however, CT instruction in grades K-2 remains limited. This study examined the perceptions of 45 K-2 teachers from 30 school districts before and after a CT professional development (PD) experience. The PD included two online se...
Article
The project allowed students to see how literacy, technology, and engineering concepts could all play a role in many different aspects of academic and everyday life.
Article
Curriculum, legislation, and standards across the nation are quickly evolving to incorporate computer science and computational thinking concepts into K-12 classrooms. For example, many states have passed legislation requiring computer science to be included in every school’s curriculum. Most states, however, report high shortages of qualified comp...
Article
Artificial intelligence (AI) is a rapidly developing field with growing importance in engineering, mainly as it serves to better understand and manipulate big data. However, the literature is sparse on how to educate engineers on the use of AI applications. For many years, a role-playing simulation known as the ‘Beer Distribution Game' has been a s...
Article
Full-text available
Iteration and improvement are important attributes of design, tacitly indicating that failure is also a part of the process. There are different conceptions of failure in engineering contexts than in other academic settings. Therefore, for beginning designers, these failure experiences may be perceived as mishaps, lowering confidence or interest. G...
Conference Paper
Full paper can be accessed at - https://peer.asee.org/41253. In 2020, the International Technology and Engineering Educators Association (ITEEA) published Standards for Technological and Engineering Literacy: The Role of Technology and Engineering in STEM Education (STEL). These standards open with a clear rationale why all Pk-12 students should st...
Conference Paper
Full-text available
Previous work has demonstrated the capacity for positively influencing student learning by engaging them in evaluation of previously submitted work as an intentional priming exercise. This exercise has been referred to as Learning by Evaluating (LbE). Nuances such as who should complete the evaluations, what should be evaluated, how often, and when...
Article
Full-text available
Traditional efforts around improving assessment often center on the teacher as the evaluator of work rather than the students. These assessment efforts typically focus on measuring learning rather than stimulating, promoting, or producing learning in students. This paper summarizes a study of a large sample of undergraduate students (n = 550) in an...
Article
Full-text available
Adaptive Comparative Judgment (ACJ), an approach to the assessment of open-ended problems which utilizes a series of comparisons to produce a standardized score, rank order, and a variety of other statistical measures, has demonstrated high levels of reliability and validity and the potential for application in a wide variety of areas. Further, res...
Article
Full-text available
This study discusses a writing assessment for State University’s first year integrated experience of Technology 120: Design Thinking in Technology and English 106: First-Year Writing and the implications it provides for curriculum design. The study used Adaptive Comparative Judgment to determine whether integrated research essays that utilized desi...
Article
Background and Context Hour of Code is one of the largest educational campaigns implemented. It exposes millions of learners, to an hour of computer science fundamentals. With such a large impact, a large number of research articles have been published on the topic. This research ranges from reports of experiments testing the efficacy of Hour of Co...
Article
Full-text available
Adaptive comparative judgment (ACJ) is a holistic judgment approach used to evaluate the quality of something (e.g., student work) in which individuals are presented with pairs of work and select the better item from each pair. This approach has demonstrated high levels of reliability with less bias than other approaches, hence providing accurate v...
Article
The single biggest factor influencing people's lives in 2020-21 has been the COVID-19 pandemic. The global pandemic has disrupted almost every aspect of daily life and people are trying to respond to a variety of challenges that range from medical concerns and infection to social distancing and feelings of loneliness and isolation. Arming students...
Preprint
Full-text available
Coding is an essential 21st-century literacy, becoming increasingly compulsory across educational systems worldwide. Consequently, there has been an explosion of tools designed to teach coding to children. In this study, we attempted to curate a list of tools whose primary intent is to enable and teach children to code. From this list, we developed...
Article
Full-text available
Traditional Accounting curricula include instruction on the preparation of audit documentation, however, experiential instruction for students on the process of review-both as a reviewer and reviewee-is often scarce or missing. This study investigated a classroom intervention that engaged undergraduate students in peer-review activities to gauge ho...
Article
Full-text available
The development of the entrepreneurial mindset is not only important for engineering graduates who desire to start a new company, but also important for graduates who desire to get a job within a company. Multi-, inter-, and transdisciplinary engineering graduates can use their entrepreneurial mindset to create optimal designs and bring value to a...
Article
Full-text available
This study investigated the perceptions of primary-school children related to each of the Science, Technology, Engineering, and Mathematics (STEM) areas and, building on research related to the potential for literature to influence perceptions, the impact of an eight-week STEM literature intervention on their perceptions. During the STEM literature...
Chapter
While Science, Technology, Engineering, and Mathematics (STEM) education is being lobbied as a mechanism for teaching students both conceptual knowledge and procedural skills, the assessment of these skills can be difficult. The open-ended nature of STEM education assignments contributes to challenges related to reliability, validity, and feasibili...
Data
An Engineering Performance Matrix (EPM) is a conceptual model to demonstrate ways in which the content identified in the framework can be used to guide engineering instruction and serve as an assessment blueprint for the development of engineering literacy and competence. EPMs are intended to provide teachers with a sharper understanding of how sub...
Book
Download @ https://www.p12engineering.org/framework - Engineering touches every aspect of human life, from providing access to clean drinking water to 5G telecommunications and vaccine development. Yet few young people ever encounter the subject in school or graduate with the foundational skills and knowledge to pursue engineering studies and car...
Article
Full-text available
Efforts to revamp existing educational systems and approaches result in various new models in STEM education. Accordingly, a new urban public charter school, located in a Midwest state in the United States, was established in partnership with the state's land-grant university to create a transformed integrated STEM educational environment character...
Article
Full-text available
In this lesson, students engage in analyzing generated data from OceanTracks.org (Figure 1), identifying patterns and creating a data visualization model based on their own research of scientific data in order to inform design decisions. Students consider the interactions from their chosen combination of variables involving marine wildlife movement...
Article
Full-text available
Adaptive comparative judgment (ACJ) has proven to be a valid, reliable, and feasible method for assessing student performance in open-ended design scenarios. In addition to the use of ACJ for purely assessment and evaluation, research has demonstrated an opportunity to identify the design values of judges involved with the ACJ process. The potentia...
Article
Full-text available
In order to feed the world’s growing population, farmers will need to produce 70% more food by 2050 than they did in 2006 (Bruinsma, 2009). To meet this demand, farmers and agriculture companies are turning to Internet of Things (IoT) technologies and data visualization to optimize analytic capabilities and ultimately, enhance their production thro...
Article
Considering the challenges associated with the teaching of engineering design, and recognizing potential differences in design values of individuals from various backgrounds, this study investigated the utility of adaptive comparative judgment (ACJ) as a method for informing the teaching and practice of engineering design. The authors investigated...
Conference Paper
Design process representations often attempt to show the iterative pattern of design through a circular or spiral representation. Expert designers iterate, constantly refining their understanding of both the design problem and solution. In other words, a designer’s ability to manage the design process—plan, reflect, and incorporate new insights—may...
Article
Research on children's experiences with designing has emphasized cognitive processes, self-efficacy, and outcomes related to designing. However, efforts have been limited towards identifying approaches children use while designing and making decisions related to design. This study, which incorporated a qualitative analysis of children's design port...
Article
Full-text available
Computational thinking is a problem-solving technique that has traditionally been employed by computer scientists to develop computer applications. However, computational thinking practices are now believed to be applicable to a variety of other fields (Google for Education, 2018), specifically those related to engineering and technology. According...
Article
Full-text available
In this article, we discuss how students can work in an engineering design setting to explore potential solutions for combatting veteran homelessness through the creation of small, scale-model “tiny homes” (Figures 1 and 2). This lesson, which integrates a socially relevant context, will provide students the opportunity to connect with core enginee...
Article
Full-text available
While design-based pedagogies have increasingly been emphasized, the assessment of design projects remains difficult due to the large number of potentially “correct” solutions. Adaptive comparative judgment (ACJ), an approach based on assessors/judges working through a series of paired comparisons and selecting the better of two items, has demonstr...
Article
Improving graphics education may begin with understanding best practices for providing, receiving, and improving formative feedback. Challenges related to anonymity, efficiency, and validity in peer critique settings all contribute to a difficult-to-implement process. This research investigates university-level computer graphics students while enga...
Chapter
As mobile devices have become increasingly ubiquitous in society, the potential for impacting student learning in classroom settings has been the subject of debate. Instant information access and collaboration have been mentioned as potential benefits of mobile device access. However, the potential for mobile devices to foster self-directed learnin...
Article
STEM education practices and approaches have been emphasized in recent years at the elementary school level. The emphasis on STEM integration at the elementary level has stressed learning, motivation, and 21st-century skills as positive outcomes. Despite this emphasis, elementary level teacher assessment practices for open-ended STEM design challen...
Article
Full-text available
This study examined the design cognition and achievement results of both kindergarten and fourth grade students engaged in engineering design-based instructional activities. Relationships between design cognition and student grade level, as well as quality of student work, were investigated. 30 concurrent think-aloud protocols were collected from i...
Article
Full-text available
This study investigated the design cognition and performance results of secondary and post-secondary engineering students while engaged in an engineering design task. Relationships between prototype performance and design cognition were highlighted to investigate potential links between cognitive processes and success on engineering design problems...
Conference Paper
Full-text available
Adaptive comparative judgment (ACJ) has proven to be a valid, reliable, and feasible method for assessing student performance in open-ended design scenarios. In addition to the use of ACJ for purely assessment and evaluation, research has demonstrated an opportunity to identify the design values of judges involved with the ACJ process and feed that...
Article
Full-text available
Open-ended design problems have become an important component in our educational landscape (Grubbs & Strimel, 2015; Jonassen, Strobel, & Lee, 2006; NRC, 2012; Strimel, 2014a). The ability of students to confront open-ended problem scenarios, think creatively, and produce novel designs have all been lauded as necessary skills for today’s 21st centur...
Conference Paper
Full-text available
Understanding the best practices of providing, receiving, and improving the formative feedback process in design is critical to improving student creative graphics education. Situated in a university-level computer graphics course this research studied the impacts on student performance of students engaged in adaptive comparative judgment (ACJ), as...
Conference Paper
Full-text available
This project details the results from first-year undergraduate engineering students, engineering instructors, and industry professionals collaborating to assess student design projects. Each group (students, instructors, and industry professionals) used adaptive comparative judgment to rank the final projects from a first-year engineering course de...
Conference Paper
Full-text available
This study investigated student capabilities in recognizing and predicting the relative quality of their own work and the work of their peers after participating in the Adaptive Comparative Judgment (ACJ) assessment process. Students participating in this study estimated their own ACJ-ranking for their design project in conjunction with ACJ exercis...
Conference Paper
Full-text available
As technological advances connect countries from across the world, preparing students to contribute to an internationally connected society is paramount. An understanding of the various cultures, traditions, values, and educational practices is necessary for a more-fully integrated and preparatory curriculum. Specifically, we contend that in the ar...
Conference Paper
Full-text available
The evolution of advanced technology systems and the labour market for future engineers and designers are a global matter. In light of this reasoning, a global perspective on technology education becomes even more important. Assessment is key in order to bridge teaching and learning and an international perspective is needed for understanding diffe...
Conference Paper
Full-text available
While current initiatives in P-12 engineering education are promising, a clear vision and roadmap eludes educators. Little is known about how children progress through engineering learning. Few curricula have explored and investigated how an articulate P-12 engineering program may contribute the general literacy of our children. The Advancing Excel...
Article
Full-text available
Innovation capacity continues to be a national concern as U.S. competitiveness and prosperity in a global economy depends on developing and maintaining competitive industries and a strong innovation-capable workforce (National Academy of Engineering, 2015; National Academy of Engineering, 2017; The White House, 2016; OECD, 2007). Innovation, consid...
Article
Full-text available
The shortage of teachers prepared to teach career and technical education (CTE) courses, or willing to work in certain locations (e.g. inner city, rural), has motivated the desire to explore solutions such as alternative routes to teacher licensure (ARL). Most ARL programs bypass colleges of education and provide a different approach to preparing i...
Article
Full-text available
While research into the effectiveness of open-ended problems has made strides in recent years, less has been done around the assessment of these problems. The large number of potentially-correct answers makes this assessment difficult. Adaptive Comparative Judgment (ACJ), an approach based on assessors/judges working through a series of paired comp...
Presentation
Dr Eva Hartell provided practical examples on how to embed formative assessment in classroom practice and elaborate on her research with comparative judgement in STEM, and provided results from the ongoing study with Dr Greg Strimel and Dr Scott Bartholomew.
Article
We investigated the use of adaptive comparative judgment to evaluate the middle school student learning, engagement, and experience with the design process in an open-ended problem assigned in a technology and engineering education course. Our results indicate that the adaptive comparative judgment tool effectively facilitated the grading of the st...
Article
Full-text available
This article examines the use of an alternative form of assessment for engineering design projects called adaptive comparative judgment (ACJ). The researchers employed an ACJ tool to evaluate undergraduate engineering student design projects in an effort to examine its' reliability, validity, and utility in comparison with traditional assessment te...
Article
Full-text available
Discussion surrounding the inclusion of mobile devices in K-12 classrooms has escalated since the early 2000s, and the literature base dedicated to mobile devices, mobile-learning, and e-learning has likewise grown. The majority of the research related to mobile devices and their inclusion in educational settings has largely revolved around efficac...
Conference Paper
Full-text available
This paper reports the commonalities, and differences, in teachers' assessment practices in middle-school technology/engineering (TE) education, in Sweden, the United Kingdom, and the United States of America-three countries where STEM-education is strongly emphasized among stakeholders while also challenged in terms of teaching and learning opport...
Presentation
Full-text available
Title of Session: A Look into Engineering Design Thinking: Elementary, Secondary, and Post-Secondary Abstract: As initiatives to integrate engineering at the P-12 level continue to increase, it has become evident there is broad societal agreement for fostering a student’s engineering design abilities. However, engineering design can be challenging...
Article
Full-text available
Today’s students are growing up in a world of constant connectivity, instant information, and ever-changing technological advancements. The increasingly ubiquitous nature of mobile devices among K–12 students has led many to argue for and against the inclusion of these devices in K–12 classrooms. Arguments in favor cite instant access to informatio...
Presentation
Full-text available
Open-ended design challenges are common in engineering, technology, and design programs (Kimbell, 2007; Reeve, 2015). However, these problems can be challenging to assess using traditional rubrics, which are problematic in terms of subjectivity and reliability (Bartholomew, 2017; Kimbell, 2007, 2012a, 2012b; Pollit 2004, 2012). This can be especial...
Conference Paper
Full-text available
Adaptive comparative judgment (ACJ), a relatively new approach to assessment, has proven valid, reliable, and feasible for the assessment of open-ended design problems. The use of ACJ for assessment has shown positive results in various countries around the world. The potential for ACJ, as a tool for international collaboration in assessment, has n...
Conference Paper
Full-text available
One of the fundamental advantages behind Adaptive Comparative Judging (ACJ) is that it is easier and more accurate to comparison judge a series of products, and to develop a rank order of achievement, than it is to score products using a more subjective method or rubric approach. Research in the field of comparative judging has shown very high leve...
Conference Paper
Full-text available
This evidence-based practice paper examines the use of an alternative form of assessment for engineering design projects called adaptive comparative judgment (ACJ). The authors employed ACJ to assess undergraduate engineering student design projects and compared the results to traditional marking assessment techniques. The authors sought to examine...
Article
Full-text available
The interest in engineering education for K-12 students has been rising (Carr, Bennett IV, & Strobel, 2012; Grubbs & Strimel, 2015; Strimel, Grubbs, & Wells, 2016), and the importance of engineering education is discussed as early as the elementary school level (Hegedus, 2014). Petroski (2003) claims that children are ready to learn engineering bec...
Article
Full-text available
Time: Five hours Lesson Overview/Purpose: Students will engage in an engineering challenge to design a " release-mechanism " for a quadcopter enabling it to deliver supplies to a remote village with difficult access. This will require students to learn the basic principles of flight, practice flying a quadcopter, and research the geological and cli...
Article
Full-text available
Scientific discoveries are a driver for advancing our technological world (ITEA/ITEEA; 2000/2002/2007). As more knowledge is acquired through scientific inquiry, people can better design and develop technological inventions and innovations (Knowles, Kelley, & Hurd, 2016). In turn, these novel technologies can aid in making new scientific discoverie...