Valerie J. Shute

Valerie J. Shute
Florida State University | FSU · Department of Educational Psychology and Learning Systems

PhD

About

193
Publications
317,617
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
11,203
Citations
Additional affiliations
May 2007 - present
Florida State University
April 2001 - May 2007
June 1986 - June 1999

Publications

Publications (193)
Chapter
Engagement is a strong predictor of learning in educational contexts, but the definition of engagement can vary from study to study, with small differences in definition leading to substantial differences in findings. In addition, students frequently employ strategies in online learning systems that the system designers may not have expected, which...
Article
Full-text available
The current study investigated students' gameplay behavioral patterns as a function of in-game learning supports delivery timing when played a computer-based physics game. Our sample included 134 secondary students (M = 14.40, SD = .90) from all over the United States, who were randomly assigned into three conditions: receiving instructional videos...
Article
Five types of affective supports were designed to induce an appropriate emotional regulation strategy in players of an educational video game. These supports were based on the emotional regulation strategies of situation selection, situation modification, attentional deployment, cognitive change, and response modulation. A series of qualitative stu...
Chapter
Full-text available
Learning analytics (LA) dashboards refer to digital tools designed to help learners keep track of their progress and goals. There is growing interest and research around the topic of LA dashboards in online learning environments, with many lessons to be learned by educational game developers and researchers. However, we need more research in this a...
Article
Background and Context: The Inclusive Assessment of Computational Thinking (CT) designed for accessibility and learner variability was studied in over 50 classes in US schools (grades 3-8). Objective: The validation studies of IACT sampled thousands of students to establish IACT’s construct and concurrent validity as well as test-retest reliability...
Article
Creativity is one of the most essential skills for success in life in our dynamic, complex world. For instance, we are currently facing major problems with the COVID-19 pandemic, which requires creative thinking for solutions. To increase the pool of creative thinkers, we need tools that can assess and support people's creativity. With advances in...
Article
Collaborative problem solving (CPS) is an essential skill for the 21st century workforce, but remains difficult to assess. Understanding how CPS skills affect CPS performance outcomes can inform CPS training, task design, feedback design, and automated assessment. We investigated CPS behaviors (individually and in co-occurring patterns) in 101 (N =...
Article
Educational video games are hypothesized to be good environments for promoting learning; however, research on conceptual learning from games is mixed. We tested whether embedding a learning support in the form of short animations illustrating physics concepts that can be used to aid gameplay improved learning. Ninety-six 7th to 11th grade students...
Article
Full-text available
Over three years, our research team has designed various learning supports for promoting content knowledge and solving game levels. In this case study, we examined the optimal design and the evaluation of learning support videos for a physics educational game. Often studies focus on investigating the effects of research-based principles without a s...
Article
Full-text available
In-game learning supports aim to help students solve game levels (i.e., game-related supports), and connect to underlying content (i.e., content-related and hybrid supports). Students with different levels of prior knowledge may have different needs for in-game supports. In this study, we designed a 2D physics game with game-related, content-relate...
Article
Persistence is an important part of student success—both in and out of school. To enhance persistence, we first need to assess it accurately. Digital games can be used as vehicles for measuring and enhancing persistence. The purpose of this study is to test the effects of (a) game-level characteristics (i.e., game mechanics and conceptual difficult...
Article
Full-text available
In this study, we investigated the validity of a stealth assessment of physics understanding in an educational game, as well as the effectiveness of different game‐level delivery methods and various in‐game supports on learning. Using a game called Physics Playground, we randomly assigned 263 ninth‐ to eleventh‐grade students into four groups: adap...
Article
We examined the use and effectiveness of an incentive system—one of the five elements of a theory-based motivational architecture in educational games that we proposed—in a computer-based physics game on students’ learning and performance. The incentive system’s purpose was to motivate students to access learning supports designed to facilitate con...
Conference Paper
Full-text available
To achieve inclusion of diverse professionals in STEM fields, educators must try to get more children, particularly underrepresented minorities, excited about STEM areas such as physics (Shute et al., 2020a). As researchers, we understand our educational responsibility in supporting active learning for a wide range of learners. In this study, we ex...
Article
Creativity has been of research interest to psychologists dating back many decades and is currently recognized as one of the essential skills needed to succeed in our complex, interconnected world. One medium that has affordances to assess and support creativity in young people is video games. In this paper, we briefly discuss the literature on vid...
Article
In honor of Jim Greer, we share our recent work—a design and development study of various learning supports embedded within the game Physics Playground. This 2-dimensional computer game is designed to help students learn Newtonian physics and uses stealth assessment to measure, in real-time, their physics understanding. The game operates according...
Article
Collaborative problem solving (CPS) is a crucial 21st century skill; however, current technologies fall short of effectively supporting CPS processes, especially for remote, computer-enabled interactions. In order to develop next-generation computer-supported collaborative systems that enhance CPS processes and outcomes by monitoring and responding...
Chapter
Full-text available
Well-designed digital games hold promise as effective learning environments. However, designing games that support both learning and engagement without disrupting flow [1] is quite tricky. In addition to including various game design features (e.g., interactive problem solving, adaptive challenges, and player control of gameplay) to engage players,...
Conference Paper
We adopt a multimodal approach to investigating team interactions in the context of remote collaborative problem solving (CPS). Our goal is to understand multimodal patterns that emerge and their relation with collaborative outcomes. We measured speech rate, body movement, and galvanic skin response from 101 triads (303 participants) who used video...
Conference Paper
Full-text available
Understanding why students quit a level in a learning game could inform the design of appropriate and timely interventions to keep students motivated to persevere. In this paper, we study student quitting behavior in Physics Playground (PP)-a Physics game for secondary school students. We focus on student cognition that can be inferred from their i...
Article
Collaborative problem solving (CPS) is an essential skill in the 21st century. There is a need for an appropriate framework and operationalization of CPS to guide its assessment and support and across multiple domains. Accordingly, we synthesized prior research on CPS to construct a generalized CPS competency model (i.e., skills and abilities) cons...
Chapter
Full-text available
Digital, educational games have many promises (e.g., increasing students’ content knowledge as well as competencies like problem solving, spatial skills, and persistence). However, there are challenges to overcome before using these games more broadly in educational settings. One challenge involves identifying effective, theoretically based learnin...
Chapter
A common skepticism about educational games is that learning and play are frequently not well integrated—the skill or content to be used and learned lacks a semantic or meaningful relation with the fantasy and challenge elements and can be easily swapped without influencing gameplay. In this chapter, we describe and analyze design challenges associ...
Chapter
There are two important design issues related to game-based learning (GBL) in school settings: (a) the intrinsic integration of content-related tasks in gameplay and (b) the real-time capture and analysis of in-game performance data. In this chapter, we describe an integrative design approach that is aimed to interweave game-based task design with...
Chapter
Games are not just a vehicle to enhance learning but a new way of understanding and organizing learning. The starting point of an optimal integration of learning and play in the game setting is to identify the salient elements of learning itself and the inherent learning processes in gameplay. In this introductory chapter, we discuss relevant theor...
Chapter
In this concluding chapter, we discuss an evolving, experiential design framework for game-based learning platforms, by synthesizing the salient design problem-solving events, experiences, and solution-exploration findings reported in the previous chapters. It is not aimed to be prescriptive or exhaustive, but acts as a starting point for specifyin...
Chapter
Our phenomenological examination of learning game design is situated in a four-year, longitudinal design-based research project that encompasses iterative design processes to develop, refine, and study a game-based learning platform called E-Rebuild. This chapter presents an introductory overview of the four facets of the interdisciplinary educatio...
Chapter
In this chapter, we provide a reflective and analytical description of the interdisciplinary design activities of E-Rebuild, identify driving design questions and salient design patterns that capture and frame the essence of E-Rebuild development, and discuss distilled meta-generalizations that help to decompose the interdisciplinary learning game...
Chapter
The role of support for game-based learning cannot be overemphasized. It remains inconclusive as to what, when, and how support for learning should be designed and implemented to foster learners’ extended engagement, in-game performance, and game-based disciplinary knowledge learning and transfer. In this chapter, we review prevalent support featur...
Chapter
Full-text available
According to Pearson’s Law: “That which is measured improves.” But some important constructs, like problem-solving skill, are generally viewed as very difficult to measure for a variety of reasons (e.g., lack of a clear and agreed-upon definition, psychological and/or statistical multidimensionality of the construct, subjectivity of scoring, and so...
Conference Paper
Full-text available
Identifying struggling students in real-time provides a virtual learning environment with an opportunity to intervene meaningfully with supports aimed at improving student learning and engagement. In this paper, we present a detailed analysis of quit prediction modeling in students playing a learning game called Physics Playground. From the interac...
Conference Paper
Full-text available
Digital games have evolved as an engaging medium for learning. This paper studies the interaction between game design and student affective experience. Data from 137 students playing a learning game was analyzed to identify the factors correlating with student frustration. Results suggest that in this well-designed game, difficulty associated with...
Chapter
Full-text available
Immersive Environments (IEs) hold many promises for learning. They represent an active approach to learning and are intended to facilitate better, deeper learning of competencies relevant for success in today’s complex, interconnected world. To harness the power of these environments for educational purposes (i.e., to support learning), we need val...
Chapter
Full-text available
Digital games are very popular in modern culture. We have been examining ways to leverage these engaging environments to assess and support important student competencies, especially those that are not optimally measured by traditional assessment formats. In this chapter, we describe a particular approach for assessing and supporting student learni...
Chapter
In this chapter, we describe the importance of assessing and developing conscientiousness in students and how we are approaching this challenge. After discussing the benefits conscientiousness has for learning, we describe the process we are using to create a valid stealth assessment of conscientiousness. We then discuss the current state of this w...
Article
Full-text available
This paper examines the growing field of computational thinking (CT) in education. A review of the relevant literature shows a diversity in definitions, interventions, assessments, and models. After synthesizing various approaches used to develop the construct in K-16 settings, we have created the following working definition of CT: The conceptual...
Book
Full-text available
Lacking a digital crystal ball, we cannot predict the future of education or the precise instructional role games will have going forward. Yet we can safely say that games will play some role in the future of K-12 and higher education, and members of the games community will have to choose between being passive observers or active, progressive cont...
Conference Paper
Full-text available
Educators are increasingly using games as a method for enabling engagement and learning in students, but research has suggested potentially inconsistent outcomes for the use of these digital tools. One explanation for these mixed findings may be different preferred playstyles of game players, such as Bartle's (1996) player taxonomies. This research...
Chapter
Full-text available
In this chapter we discuss challenges with traditional assessment when researchers deal with hard-to-measure constructs such as systems-thinking, creativity, or persistence. We then describe how the frameworks of evidence-centered design and stealth assessment may be used to develop assessments that can be embedded in video games to make valid infe...
Conference Paper
Full-text available
The educational system in the U.S. needs a "paradigm change" (Reigeluth & Karnopp, 2013) or an "educational sea change" (Shute, 2007). Schools should prepare students for the information age society in which we are living (Reigeluth & Karnopp, 2013; Shute, 2007). There are many lessons we, as educators and instructional designers, can learn from ga...
Article
We used stealth assessment, embedded in a game called Use Your Brainz (a slightly modified version of Plants vs. Zombies 2), to measure middle-school students’ problem solving skills. We began by developing a problem solving competency model based on a review of the relevant literature. We then identified in-game indicators that would provide evide...
Article
Affect detection is a key component in intelligent educational interfaces that respond to students' affective states. We use computer vision and machine-learning techniques to detect students' affect from facial expressions (primary channel) and gross body movements (secondary channel) during interactions with an educational physics game. We collec...
Article
Full-text available
Review of computer-based assessment for learning in elementary and secondary education Assessment should not merely be done to students; rather, it should also be done for students, to guide and enhance their learning. NCTM (2000). Assessment refers to not only systematically collecting and analyzing information about a learner—what people normally...
Article
Full-text available
Designing, developing, and administering assessments has remained fairly unchanged across the past century. However, recent developments in instructional technology, learning science theory, and advances in the design of assessments necessitate a newfound perspective on assessment. The objective of the present paper is to review the topic of assess...
Chapter
Full-text available
A number of assessment experts have advanced a vision of assessment in schools that relies on ongoing, performance-based, formative assessment. While there are many potential benefits of such a system, it also raises concerns about assessment quality. This chapter is a review of the current state of the evidence for psychometric properties undergir...
Chapter
Full-text available
Many games offer players opportunities to express their creativity, from posting their clever solutions in online forums, to beating particular boss levels, to creating their own levels using built-in level editors. Yet there is scant evidence supporting the link between video games and creativity. This is partially due to the difficulty of measuri...
Article
Full-text available
Game-based assessment (GBA) is a specific use of educational games that employs game activities to elicit evidence for educationally valuable skills and knowledge. While this approach can provide individualized and diagnostic information about students, the design and development of assessment mechanics for a GBA is a nontrivial task. In this artic...
Data
Full-text available
Conference Paper
Full-text available
This paper discusses multimodal affect detection from a fusion of facial expressions and interaction features derived from students' interactions with an educational game in the noisy real-world context of a computer-enabled classroom. Log data of students' interactions with the game and face videos from 133 students were recorded in a computer-ena...
Article
Full-text available
Stealth assessment provides an innovative way to assess and ultimately support knowledge, skills, and other personal attributes within learning and gaming environments without disrupting students' flow. In this paper, the authors briefly discuss two challenges they encountered during the development of stealth assessments in two past projects (i.e....
Article
This study investigated the relationships among incoming knowledge, persistence, affective states, in-game progress, and consequently learning outcomes for students using the game Physics Playground. We used structural equation modeling to examine these relations. We tested three models, obtaining a model with good fit to the data. We found evidenc...
Article
Abstract Educators today are increasingly interested in using game-based assessment to assess and support students' learning. In the present study, we investigated how changing a game design element, linearity in gameplay sequences, influenced the effectiveness of game-based assessment in terms of validity, reliability, fairness, learning, and enjo...
Conference Paper
Full-text available
Increased attention to the relationships between affect and learning has led to the development of machine-learned models that are able to identify students' affective states in computerized learning environments. Data for these affect detectors have been collected from multiple modalities including physical sensors, dialogue logs, and logs of stud...
Conference Paper
The goal of this paper was to explore the possibility of generalizing face-based affect detectors across multiple days, a problem which plagues physiological-based affect detection. Videos of students playing an educational physics game were collected in a noisy computer-enabled classroom environment where students conversed with each other, moved...
Article
Full-text available
Affect detection is a key component in developing intelligent educational interfaces that are capable of responding to the affective needs of students. In this paper, computer vision and machine learning techniques were used to detect students' affect as they used an educational game designed to teach fundamental principles of Newtonian physics. Da...
Article
Full-text available
Physics Playground is an educational game that supports physics learning. It accepts multiple solutions to most problems and does not impose a stepwise progression through the content. Assessing student performance in an open-ended environment such as this is therefore challenging. This study investigates the relationships between student action se...
Article
Full-text available
In this chapter, we describe the processes of designing and validating game-based learning assessment and/or support in two different games—Portal 2 (by Valve Corporation) and Earthquake Rebuild. The games represent cases of possible game-based learning (i.e., domain-generic and domain-specific) and provide good vehicles for testing the design deci...
Article
Full-text available
The study explored instructional benefits of a storyline gaming characteristic (GC) on learning effectiveness, efficiency, and engagement with the use of an online instructional simulation for graduate students in an introductory statistics course. A storyline is a game-design element that connects scenes with the educational content. In order to e...
Article
Full-text available
One of the key ideas of evidence-centered assessment design (ECD) is that task features can be deliberately manipulated to change the psychometric properties of items. ECD identifies a number of roles that task-feature variables can play, including determining the focus of evidence, guiding form creation, determining item difficulty and discriminat...
Article
Full-text available
In this study, we tested 77 undergraduates who were randomly assigned to play either a popular video game (Portal 2) or a popular brain training game (Lumosity) for 8 hours. Before and after gameplay, participants completed a set of online tests related to problem solving, spatial skill, and persistence. Results revealed that participants who were...