ArticlePublisher preview available

The effectiveness of adaptive versus non‐adaptive learning with digital educational games

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

For the training of academic skills, digital educational games with integrated adaptivity are promising. Adaptive games are considered superior to non‐adaptive games, because they constantly assess children's performance, and accordingly adapt the difficulty of the tasks corresponding to the children's individual level. However, empirical evidence with regard to the effectivity of adaptive compared to non‐adaptive games is limited. A study was conducted with 191 children from the third year of Kinder garten who were enrolled in one of three conditions, that is, playing an adaptive version of the reading game (RG), a non‐adaptive version of the RG or training with pen‐and‐paper exercises. In all three conditions, children trained emergent reading (phonological awareness and letter knowledge) once a week for 30 min over a period of 5 weeks. Children's performance on cognitive (phonological awareness, letter knowledge, reading fluency) and non‐cognitive (motivation, self‐concept) factors was assessed. Results revealed a significant improvement in phonological awareness and letter knowledge in all conditions. However, no differences between the conditions were observed with respect to children's improvement on phonological awareness and letter knowledge or on their post‐test scores for reading fluency. With regard to motivation and self‐concept, again, no differences in these non‐cognitive factors were observed across conditions.
This content is subject to copyright. Terms and conditions apply.
ARTICLE
The effectiveness of adaptive versus non-adaptive learning
with digital educational games
Stefanie Vanbecelaere
1,2
| Katrien Van den Berghe
2
| Frederik Cornillie
2
|
Delphine Sasanguie
3
| Bert Reynvoet
3
| Fien Depaepe
1,2
1
Faculty of Psychology and Educational
Sciences, Centre for Instructional Psychology
and Technology, KU Leuven, Belgium
2
ITEC, IMEC Research Group, KU Leuven,
Belgium
3
Faculty of Psychology and Educational
Sciences, Brain & Cognition, KU Leuven,
Belgium
Correspondence
Stefanie Vanbecelaere, Faculty of Psychology
and Educational Sciences, KU Leuven Kulak,
Etienne Sabbelaan 51, 8500 Kortrijk, Belgium.
Email: stefanie.vanbecelaere@kuleuven.be
Funding information
VLAIO Flanders Innovation &
Entrepreneurship
Peer Review
The peer review history for this article is
available at https://publons.com/publon/10.
1111/jcal.12416.
Abstract
For the training of academic skills, digital educational games with integrated adaptiv-
ity are promising. Adaptive games are considered superior to non-adaptive games,
because they constantly assess children's performance, and accordingly adapt the dif-
ficulty of the tasks corresponding to the children's individual level. However, empiri-
cal evidence with regard to the effectivity of adaptive compared to non-adaptive
games is limited. A study was conducted with 191 children from the third year of
Kinder garten who were enrolled in one of three conditions, that is, playing an adap-
tive version of the reading game (RG), a non-adaptive version of the RG or training
with pen-and-paper exercises. In all three conditions, children trained emergent read-
ing (phonological awareness and letter knowledge) once a week for 30 min over a
period of 5 weeks. Children's performance on cognitive (phonological awareness, let-
ter knowledge, reading fluency) and non-cognitive (motivation, self-concept) factors
was assessed. Results revealed a significant improvement in phonological awareness
and letter knowledge in all conditions. However, no differences between the condi-
tions were observed with respect to children's improvement on phonological aware-
ness and letter knowledge or on their post-test scores for reading fluency. With
regard to motivation and self-concept, again, no differences in these non-cognitive
factors were observed across conditions.
KEYWORDS
adaptive learning, digital game-based learning, emergent reading, intervention, preschool
1|THEORETICAL BACKGROUND
1.1 |Educational games to support learning
Very early in education, children need to learn important skills such
as reading and math. Persistent training of these academic skills is
required in order to prevent children from later learning difficulties
as it is proved that early reading and math skills are predictive for
children's future academic achievement (Melby-Lervåg, Lyster, &
Hulme, 2012; Sasanguie, Göbel, Moll, Smets, & Reynvoet, 2013).
One opportunity to establish practice in the classroom is the use of
digital educational games which are specifically designed to contrib-
ute to educational purposes (Jamshidifarsani, Garbaya, Lim,
Blazevic, & Ritchie, 2019; Szűcs & Myers, 2017). Specific character-
istics such as immediate feedback, meaningful and engaging con-
texts, interaction, modality and adaptive practice are promising
features in order to enable learning (Wouters & van Oostendorp,
2013). Because digital educational games are not only informative
but also entertaining, they are assumed to impact learners' cognitive
as well as non-cognitive learning outcomes (All, Nuñez Castellar, &
Van Looy, 2015; Wouters, van Nimwegen, van Oostendorp, & van
Der Spek, 2013).
Received: 23 July 2019 Revised: 30 October 2019 Accepted: 1 December 2019
DOI: 10.1111/jcal.12416
502 © 2019 John Wiley & Sons Ltd J Comput Assist Learn. 2020;36:502513.wileyonlinelibrary.com/journal/jcal
... In this paper, we focus on the adaptation of difficulty, a common adaptation approach in game-based learning (Liu, Moon, Kim, & Dai, 2020). Although previous research has shown that adaptation is a promising instructional technique in game-based learning (Wouters & Oostendorp, 2017), several studies have revealed that difficulty adaptation does not always have positive effects on motivational outcomes (e.g., Shute et al., 2021;Van Oostendorp, Van Der Spek, & Linssen, 2014;Vanbecelaere et al., 2020). These findings are inconsistent with the theoretical propositions of seminal motivational theories suggesting that balancing the difficulty of the game with the player's skill level should positively affect motivational outcomes (Csikszentmihalyi, 1990;Hattie, Hodis, & Kang, 2020;Ryan & Deci, 2020). ...
... However, only a limited number of studies have examined the effects of difficulty adaptation on motivational outcomes (for a review, see Ninaus & Nebel, 2021;Liu et al., 2020;Sajjadi, Ewais, & De Troyer, 2022). Further, several studies have revealed that difficulty adaptation does not always have positive effects on motivational outcomes (Orvis, Horn, & Belanich, 2008;Sampayo-Vargas et al., 2013;Shute et al., 2021;Van Oostendorp et al., 2014;Vanbecelaere et al., 2020). The measured motivational outcomes of these studies included, for example, engagement (Orvis et al., 2008), training motivation (Van Oostendorp et al., 2014), and situational interest (Vanbecelaere et al., 2020). ...
... Further, several studies have revealed that difficulty adaptation does not always have positive effects on motivational outcomes (Orvis, Horn, & Belanich, 2008;Sampayo-Vargas et al., 2013;Shute et al., 2021;Van Oostendorp et al., 2014;Vanbecelaere et al., 2020). The measured motivational outcomes of these studies included, for example, engagement (Orvis et al., 2008), training motivation (Van Oostendorp et al., 2014), and situational interest (Vanbecelaere et al., 2020). The non-significant results of these interventions might be partly explained by rather small sample sizes (Orvis et al., 2008;Van Oostendorp et al., 2014) or the relatively short duration of the interventions (Sampayo-Vargas et al., 2013). ...
Article
Research has shown that difficulty adaptation is a promising instructional technique in digital game-based learning. Nevertheless, although the strength and direction of difficulty adaptation can affect motivational outcomes, these effects remain insufficiently examined in game-based learning. This within-subject study examined how the strength and direction of difficulty adaptation affected motivational outcomes in digital game-based math learning. The participants were 167 European (masked for peer review) fifth-graders who studied fractions with the anonymized (name of the game has been anonymized for peer review) game. The game included 144 tasks, half were adapted according to participants’ playing performance. Situational interest and perceived difficulty were measured several times with in-game self-report items during the intervention. The manipulation check confirmed that difficulty adaptation was implemented successfully as task correctness and perceived difficulty changed according to the strength and direction of adaptation. Regarding motivational outcomes, two-way repeated-measures ANOVAs showed that the difficulty adaptation increased situational interest, but only when the task difficulty was substantially adapted downwards. Contrary to our expectations, a substantial upwards adaptation of the task difficulty significantly decreased situational interest. Minor adaptation of difficulty did not affect situational interest. The current study contributes to the field of adaptive digital learning environments by highlighting the effects of the strength and direction of difficulty adaptation on motivational outcomes. Theoretical, practical, and methodological implications of the findings are discussed.
... Evaluadores/as capacitados/as y supervisados/as por el equipo de investigación Programa de entrenamiento: Tutores/as sin experiencia docente previa Sesiones diarias de 20 minutos, cinco días por semana, por 10 semanas Forma modificada de Interactive Book Reading, actividad utilizada en la intervención del GC. (Vanbecelaere et al., 2019) Sílaba Fonema ...
Article
Phonological awareness is a strong predictor of reading and writing skills development. Several programs have been developed and tested for the stimulation of phonological awareness, but the degree of variation among them makes it difficult to compare the different methods. A synthesis of the literature on phonological awareness stimulation programs is needed to examine the effectiveness of PA programs while considering the variability that exists between the methodologies used. This systematic review aimed to 1) synthesize the literature on PA stimulation programs in typically developing children; 2) examine the effectiveness of PA programs; 3) critically appraise the methodology of PA stimulation programs. Central, Medline, Pubmed, Scopus, and Web of Knowledge were used to conduct an extensive literature search. A total of 10 articles met the eligibility criteria and were included in the present study. Results showed that, in general, the phonological awareness stimulation programs carried out in the analyzed research were effective. However, the quality of the methodology varied significantly across studies, showing in some cases a lack of detail in the inclusion criteria, limited training of the professionals who carried out the PA programs, an inconsistency in parent involvement, and a lack of follow-up. Guidelines for future research are discussed to enhance the methodological quality of this line of research and reduce the risk of bias.
... To perform personalized learning, identifying children's learning trajectories and dynamic problem-solving processes in advance is crucial (Lin et al., 2013). In DGBL, to systematically support children's personalized learning, emerging research has incorporated adaptivity in games (Vanbecelaere et al., 2020). Here, adaptivity refers to the systematic and dynamic delivery of game-based instructional activities through ongoing and in-situ learner analyses . ...
Article
Full-text available
In this study, we have proposed and implemented a sequential data analytics (SDA)-driven methodological framework to design adaptivity for digital game-based learning (DGBL). The goal of this framework is to facilitate children’s personalized learning experiences for K–5 computing education. Although DGBL experiences can be beneficial, young children need personalized learning support because they are likely to experience cognitive challenges in computational thinking (CT) development and learning transfer. We implemented the educational game Penguin Go to test our methodological framework to detect children’s optimal learning interaction patterns. Specifically, using SDA, we identified children’s diverse gameplay patterns and inferred their learning states related to CT. To better understand children’s gameplay performance and CT development in context, we used qualitative data as triangulation. We discuss adaptivity design based on the children’s gameplay challenges indicated by their gameplay sequence patterns. This study shows that SDA can inform what in-game support is necessary to foster student learning and when to deliver such support in gameplay. The study findings suggest design guidelines regarding the integration of the proposed SDA framework.
... Adaptivity responds to the user's profile [83,84] and activity that is tracked and recorded on a database, activating programmatic flags to adapt to the choices [85]. This aspect is essential to guide players through the game and make the learning more effective [86][87][88]. The least intrusive adaptation is to use NPCs to provide the necessary feedback [76], so that the immersive experience will not decrease. ...
Article
Full-text available
Serious games have to meet certain characteristics relating to gameplay and educational content to be effective as educational tools. There are some models that evaluate these aspects, but they usually lack a good balance between both ludic and learning requirements, and provide no guide for the design of new games. This study develops the Gaming Educational Balanced (GEB)Model which addresses these two limitations. GEB is based on the Mechanics, Dynamics and Aesthetics framework and the Four Pillars of Educational Games theory. This model defines a metric to evaluate serious games, which can also be followed to guide their subsequent development. This rubric is tested with three indie serious games developed using different genres to raise awareness of mental illnesses. This evaluation revealed two main issues: the three games returned good results for gameplay, but the application of educational content was deficient, due in all likelihood to the lack of expert educators participating in their development. A statistical and machine learning validation of the results is also performed to ensure that the GEB metric features are clearly explained and the players are able to evaluate them correctly. These results underline the usefulness of the new metric tool for identifying game design strengths and weaknesses. Future works will apply this metric to more serious games to further test its effectiveness and to guide the design of new serious games.
... These are challenges that have been addressed in past work on adaptive assessment, and they are definitely possible to address here, but careful effort will be needed to do so. Moreover, previous work in the context of game-based learning adaptivity has in many cases reported null results, indicating no differences between adaptive level navigation and linear student navigation [72], [73]. In our specific case, the value of adaptation in Shadowspect puzzle selection has not yet been experimentally demonstrated, and thus it remains as a direction for future work. ...
Article
Full-text available
Previous research and experiences have indicated the potential that games have in educational settings. One of the possible uses of games in education is as game-based assessments (GBA), using game tasks to generate evidence about skills and content knowledge that can be valuable. There are different approaches in the literature to implement the assessment machinery of these GBA, all of them having strengths and drawbacks. In this paper, we propose using multivariate Elo-based learner modeling, as we believe it has a strong potential in the context of GBA for three aims: 1) to simultaneously measure students competence across several knowledge components in a game, 2) to predict task performance, and 3) to estimate task difficulty within the game. To do so, we present our GBA Shadowspect, which is focused on solving geometry puzzles, and we depict our implementation using data collected from several high schools across the USA. We obtain high performing results (AUC of 0.87) and demonstrate that the model enables analysis of how each student's competency evolves after each puzzle attempt. Moreover, the model provides accurate estimations of each task's difficulty, enabling iterative improvement of the game design. This study highlights the potential that multivariate Elo-based learner modeling has within the context of GBA, sharing lessons learned, and encouraging future researchers in the field to consider this algorithm to build their assessment machinery.
... Nevertheless, a digital game's effectiveness during the learning process does not depend only on its compelling and immersive characteristics. It is generally acknowledged that if the game can successfully distinguish each learner's characteristics and adapt its content to them, its profits through its employment in learning are significantly enhanced [9][10][11][12]. These characteristics may be related to the educational content, e.g. they may concern the learner's preferences regarding the learning style or the selection of exercises and activities adapted to her/his knowledge level. ...
Article
Nowadays, the use of educational games is gaining substantial popularity. Games offer immersive and fascinating environments that render them a powerful tool for achieving high-quality learning outcomes, like advancing students’ education through engaging activities. However, one crucial enhancement is that they should take into consideration important aspects of each individual student and adapt to them accordingly for improving the educational results. Given the above, the present work considers an educational adventure game that offers cognitive-based adaptivity in its educational content and scenarios. More specifically, it adapts both the difficulty level of the educational content and its plot dynamically to the knowledge level and learning needs of each individual student. The adaptation is realized using a fuzzy student model that detects the current cognitive state of the trainee and decides about the way in which the game’s plot has to be modified. It determines if its scenario will be dynamically extended or not, aiming to assist the trainee overcome her/his weaknesses and improve her/his educational performance. The gain of this is that through the plot’s adaptation, the game motivates the learners/players to be involved actively in the learning process and provides educational content tailored to their knowledge level and weaknesses, enhancing the educational results. The contribution of the game to the educational process and outcomes was thoroughly evaluated. The evaluation results indicate a high acceptance rate of the game by learners and teachers and underline its effectiveness in the educational results.
... This learning aims to adapt the learning experience by making changes dynamically, based on interactions and input provided by the learner (Somyürek, 2015). Adaptive behaviour allows teachers to develop learning models based on learners' knowledge, characteristics, and capacities (Vanbecelaere et al., 2020). It has been proven in general that mastery of speaking in a foreign language takes place better through a participatory process, exploration of adapted materials and a lively learning atmosphere (Wloka & Winiwarter, 2021). ...
Article
Full-text available
Language teaching has developed with the support of techniques, media, and learning approaches that complement each other. However, the need to develop linguistic skills never arrest and need to be improved continuously. One of the models developed in the early 21st century is adaptive learning, but this learning tends to be more widely applied outside of language teaching. Therefore, This study aims to find concrete forms of adaptive learning that can be applied in appropriately developing speaking skills in German. A qualitative, descriptive approach is employed in this study, focusing on an inductive inquiry into phenomena and fact-finding on adaptive learning. The learning source employed in the learning is a coursebook called Netzwerk Deutsch als Fremdsprache.To bridge the rigidity of the content and the less flexible learning aspects delivered by the coursebook, we tried to integrate it with the campus internal learning management system Besmart-Moodle. There were 12 students (initials P1-12) who participated in learning and practising speaking-oriented learning. We collected data using video recordings, digital questionnaires and field notes. Afterwards, we analyzed participants' conduct and responses using coding procedures to formulate categories relevant to adaptive learning forms. The results showed that most participants viewed this learning positively, and it supported improving their speaking skills effectively. In addition, for both students and lecturers, this learning model thoroughly impacted the learning process concerning certain linguistic facets and enriched previous research.
Chapter
Digital games for learning have been introduced as a motivating way for children to learn as they can provide immediate feedback, embed the learning content in an attractive narrative, and adapt item difficulty to learners’ performance in the game. Although studies showed positive benefits of using digital educational games compared to other teaching methods, a systematic assessment of the effectiveness of digital games for learning is lacking. In this paper, we reflect on two intervention studies in which we investigated the effectiveness of digital games for learning with integrated adaptivity. Based on this work, we propose an extension of the framework of All and colleagues [5] for assessing digital games for learning including cognitive, noncognitive, and efficiency outcomes. We added aspects of study design that need to be considered such as learner, intervention, and measurement characteristics. The validation and adoption of this framework may contribute to more standardized procedures to assess the effectiveness of digital games for learning.KeywordsDigital games for learningEffectivenessMethodologies
Article
Full-text available
This study aims to examine adaptability for educational games in terms of adaptation elements, components used in creating user profiles, and decision algorithms used for adaptation. For this purpose, articles and full-text papers in Web of Science, Google Scholar, and Eric databases between 2000–2021 were searched using the keywords "educational games", "serious games", "game-based learning", "adapt*", "player modeling", "user modeling". After applying the inclusion and exclusion procedures of studies accessed in the search, 26 studies were included in the study. The studies were analyzed in line with the themes determined for the components used in the adaptation of educational games. According to the results, adaptive educational game design was made for a wide variety of fields such as programming teaching, physics, mathematics, computational thinking, and logic. As for adaptive factors; It was determined that adaptations were made for the game, educational content, interface, and non-player character (NPC) behaviors. It is understood that pre-game adaptation and in-game adaptation methods are used as adaptation types. Finally, it is seen that Bayesian networks, artificial neural networks, fuzzy logic, deep learning, item response theory, and decision tree methods are preferred as decision systems in the adaptation process. The findings of this literature review can facilitate the design process by providing a roadmap for researchers interested in adaptive educational game design.
Article
Full-text available
Most studies of children’s attitudes to mathematics have dealt with children in second grade or later, and have suggested that attitudes deteriorate, and anxiety increases with age. The present study investigated attitudes to mathematics in 67 English and 49 Chinese children at the end of their first year of school. The participants were given Thomas and Dowker’s (2000) Mathematics Attitude Questionnaire, which uses pictorial rating scales to assess primary school children’s mathematics anxiety, liking for mathematics, unhappiness at poor performance in mathematics, and self-rating in mathematics. They were also given the British Abilities Scales Basic Number Skills test. Attitudes were generally positive, though not more so than previously found for older primary school children. The Chinese children performed better in the arithmetic test and also rated themselves higher than the English children, but did not differ in other attitudes. Self-rating in mathematics and lack of unhappiness at poor performance were associated with better performance in the English group. There were no significant relationships between attitudes and performance in the Chinese group. Implications of the findings are discussed.
Article
Full-text available
Electronic learning systems have received increasing attention because they are easily accessible to many students and are capable of personalizing the learning environment in response to students’ learning needs. To that end, using fast and flexible algorithms that keep track of the students’ ability change in real time is desirable. Recently, the Elo rating system (ERS) has been applied and studied in both research and practical settings (Brinkhuis & Maris, 2009; Klinkenberg, Straatemeier, & van der Maas in Computers & Education, 57, 1813–1824, 2011). However, such adaptive algorithms face the cold-start problem, defined as the problem that the system does not know a new student’s ability level at the beginning of the learning stage. The cold-start problem may also occur when a student leaves the e-learning system for a while and returns (i.e., a between-session period). Because external effects could influence the student’s ability level during the period, there is again much uncertainty about ability level. To address these practical concerns, in this study we propose alternative approaches to cold-start issues in the context of the e-learning environment. Particularly, we propose making the ERS more efficient by using an explanatory item response theory modeling to estimate students’ ability levels on the basis of their background information and past trajectories of learning. A simulation study was conducted under various conditions, and the results showed that the proposed approach substantially reduces ability estimation errors. We illustrate the approach using real data from a popular learning platform.
Conference Paper
Full-text available
One of the challenges with Computer Science serious games is ensuring they are suitable for learners of di�erent levels of ability and knowledge. To address this challenge, we propose a new methodology for incorporating adaptive gameplay and content into existing non-adaptive serious programming games. Our methodology includes four phases: (1) Identifying an existing game that is suitable for adaptation; (2) Modeling the gameplay tasks and the in-game assessment of learning; (3) Building the adaptation into the existing code base; (4) Evaluating the new adaptive serious game in comparison to the original game with respect to learning and engagement.
Article
Full-text available
The development of games for people with learning disabilities is one way to enhance the quality of learning and respond to the need for inclusive special educational support. Recently, game researchers have highlighted the need for paying more attention to identifying the game design choices that can strengthen learning. This paper reviews recent studies in the field of games that aim at supporting people with difficulties in learning, particularly in basic reading and maths skills. We identify the major characteristics and learning outcomes of the reviewed studies, as well as key design principles that have been used in games for enhancing basic reading and maths skills. The results show that people with specific learning difficulties have positive improvements in the quality of learning. We also found specific gamification elements that have been used to promote the learning of basic reading and maths skills. However, we call for research, which would explicitly examine the effects of game design choices on learning. Currently, the studies that address learning disabilities do not specifically define which kind of games and game design the results refer to, while game design studies do not clarify how these games influence learning. Thus, there is a need to rethink previous empirical studies on game settings for people with learning difficulties via advancing the role of game design in empirical intervention studies.
Article
Research suggests that gains in executive function (EF) skills training are strongest when task difficulty increases progressively, yet findings on the effectiveness of adaptive approaches for EF training are inconsistent. This study compared the effectiveness of an adaptive vs a non-adaptive version of a digital game designed to train the EF sub-skill of shifting. Results showed increases in shifting skills for all learners between pretest and posttest measures, with adolescents scoring higher than pre-adolescents and early adolescents on posttest measures. Data analysis uncovered a trend suggesting that the adaptive treatment may be more effective than the non-adaptive treatment for adolescents. User logs showed that adaptivity helped customize players’ gameplay based on their performance, by making game play easier for younger learners, and making game play more difficult for older learners. Results support the use of digital games to train EF for a broad range of learners.
Article
Digital games (e.g., video games or computer games) have been reported as an effective educational method that can improve students' motivation and performance in mathematics education. This meta-analysis study (a) investigates the current trend of digital game-based learning (DGBL) by reviewing the research studies on the use of DGBL for mathematics learning, (b) examines the overall effect size of DGBL on K-12 students' achievement in mathematics learning, and (c) discusses future directions for DGBL research in the context of mathematics learning. In total, 296 studies were collected for the review, but of those studies, only 33 research studies were identified as empirical studies and systematically analyzed to investigate the current research trends. In addition, due to insufficient statistical data, only 17 out of the 33 studies were analyzed to calculate the overall effect size of digital games on mathematics education. This study will contribute to the research community by analyzing recent trends in significant DGBL research, especially for those who are interested in using DGBL for mathematics education.
Article
2016 Elsevier GmbHA popular suggestion states that an evolutionarily grounded analogue magnitude representation, also called an approximate number system (ANS) or ‘number sense’ underlies human mathematical knowledge. During recent years many studies aimed to train the ANS with the intention of transferring improvements to symbolic arithmetic. Here we critically evaluate all published studies. We conclude that there is no conclusive evidence that specific ANS training improves symbolic arithmetic. We provide a citation analysis demonstrating that highly controversial results often get cited in support of specific claims without discussion of controversies. We suggest ways to run future training studies so that clear evidence can be collected and also suggest that data should be discussed considering both supporting and contrary evidence and arguments.