Conference PaperPDF Available

Uncovering Gender and Problem Difficulty Effects in Learning with an Educational Game

Authors:
Bruce McLaren
Bruce McLaren
Bruce McLaren
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Rosta Farzan at University of Pittsburgh
Rosta Farzan
Deanne Adams
Deanne Adams
Deanne Adams
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Richard Mayer at University of California, Santa Barbara
Richard Mayer
Show all 5 authorsHide
Download full-text PDF
Read full-text
Download citation

Abstract and Figures

A prior study showed that middle school students who used the educational game Decimal Point achieved significantly higher gain scores on immediate and delayed posttests of decimal understanding than students who learned with a more conventional computer-based learning tool. This paper reports on new analyses of the data from that study, providing new insights into the benefits of the game. First, females benefited more than males from the game. Second, students in the game condition performed better on the more difficult intervention problems. This paper presents these new analyses and discusses why the educational game might have led to these results.
Figures - uploaded by Rosta Farzan
Author content
All figure content in this area was uploaded by Rosta Farzan
Content may be subject to copyright.
Content uploaded by Rosta Farzan
Author content
All content in this area was uploaded by Rosta Farzan on Jan 10, 2019
Content may be subject to copyright.
Uncovering Gender and Problem Difficulty
Effects in Learning with an Educational Game
Bruce McLaren
1(&)
, Rosta Farzan
2
, Deanne Adams
3
,
Richard Mayer
4
, and Jodi Forlizzi
1
1
Carnegie Mellon University, Pittsburgh, PA, USA
bmclaren@cs.cmu.edu
2
University of Pittsburgh, Pittsburgh, PA, USA
3
University of Notre Dame, South Bend, IN, USA
4
University of California, Santa Barbara, Santa Barbara, CA, USA
Abstract. A prior study showed that middle school students who used the
educational game Decimal Point achieved significantly higher gain scores on
immediate and delayed posttests of decimal understanding than students who
learned with a more conventional computer-based learning tool. This paper
reports on new analyses of the data from that study, providing new insights into
the benefits of the game. First, females benefited more than males from the
game. Second, students in the game condition performed better on the more
difficult intervention problems. This paper presents these new analyses and
discusses why the educational game might have led to these results.
Keywords: Educational games Mathematics learning Educational data
mining
1 Introduction
Research is still needed to determine the conditions under which game-based learning
can be effective [1]. A meta-review of over 1000 educational game studies advises that
more value-added studies of educational games be conducted, that is, research that
carefully identifies the features and conditions that lead to the successes and failures of
educational games [2]. This paper is a step in that direction. Using a successful edu-
cational game, Decimal Point, we investigate the conditions that lead to learning. In a
prior study Decimal Point was shown to lead to more learning and was more enjoyable
to students than a more conventional computer-based learning tool [3]. In this paper,
we report on new analyses that shed light on who benefitted from the game and under
what conditions.
Decimal Point (Fig. 1) is a single-player game based on an amusement park
metaphor, targeted at middle-school students learning decimals. Students play a series
of mini-games in different theme areas of the amusement park that are targeted at
decimal misconceptions. There is no scoring and no leader board; students simply
make their way through the park and are congratulated upon finishing.
©Springer International Publishing AG 2017
E. Andréet al. (Eds.): AIED 2017, LNAI 10331, pp. 540–543, 2017.
DOI: 10.1007/978-3-319-61425-0_59
The “Space Raider”mini-game of Fig. 2is targeted at the common misconception
in which students think longer decimals are larger than shorter decimals (e.g.,
0.634 > 0.82). The student tries to shoot the alien ships in the requested order (i.e.,
smallest to largest). If they make mistakes, they are prompted to correct their solution
by dragging and dropping the decimals to the correct sequence. The various
mini-games challenge students with other types of decimal problems, as well, including
placing a point on a number line and adding decimals. After playing a mini-game and
correctly solving the problem, the student is prompted to explain his or her solution [4],
by choosing possible self-explanations from a multiple-choice list.
As a comparison to the game, students use a conventional, non-game version of the
decimal instructional materials that employs a more standard user interface for solving
decimal problems. As with the mini-games, after solving a problem the student is
prompted to explain his or her solution in the same way that the mini-games prompt
students for self-explanation.
A classroom study of Decimal Point is presented in [3]. The study involved more
than 150 sixth grade students at two schools, comparing students who played the
Decimal Point game to learn decimals with students who learned decimals with the
more conventional computer-based learning tool. The same 48 decimal problems were
presented to students in the same order across the conditions, except that students in the
game condition solved problems using the mini-games, while students in the non-game
condition solved the problems using the conventional instructional software. Students
in both conditions took a pretest, posttest, and delayed posttest (comprising 61 items).
A survey, completed after the intervention, had 11 5-point Likert scale questions
(“Strongly agree”(1) to “Strongly disagree”(5)) related to the categories of Lesson
Enjoyment, Ease of Interface, and Feelings of Math Efficacy. Summary of the results
[3]: students in the game condition learned significantly more and had significantly
more positive feelings about their experience. Also, low prior knowledge learners
benefited significantly more from the game.
Fig. 1. Map of the Decimal Point game Fig. 2. The “Space Raider”mini-game
Uncovering Gender and Problem Difficulty Effects in Learning 541
2 Exploring Gender and Problem Difficulty Effects
In this work, we raised new research questions and conducted new analyses of the data
from our prior study. RQ1: Is the learning benefit of playing the Decimal Point game
more, less, or the same for female students as for male students? RQ2: Did the
Decimal Point game lead to students performing better, and potentially learning more,
from the more difficult problems in the intervention? We wondered whether females, in
particular, might benefit from the game. One could argue that games are more likely to
benefit males who more frequently identify themselves as gamers and are more fre-
quent game players [5]. Yet, there is evidence that gender does not play a role,
especially when games rely on fostering intrinsic motivation [6]. It is also important to
understand how games facilitate learning of more complicated materials. As the dif-
ficulty level of problems grows, engagement with the materials might drop. Thus,
engagement might be more important for difficult problems. Games could help by
providing a more engaging way to grapple with difficult problems.
157 students participated in the study with the gender distribution as follows:
Game (70) –39 females, 31 males; Non-Game (87) –49 females, 38 males.
To address RQ1 we conducted a regression analysis to predict the relationship
between the intervention and learning outcomes for female vs. male students. The
results of the main effect in the regression model confirmed the prior learning results
cited above. The results for immediate and delayed posttests by gender are summarized
in Fig. 3(a) and (b). In terms of both immediate posttest and delayed posttest, there is a
significant interaction effect of game condition with gender.
Even though both male and female students performed significantly better under the
game condition on the immediate posttest, the effect size for the female students is
higher (d= .59 vs. d= .39). For the delayed posttest, while male students did not
perform differently with or without game, female students performed significantly
better under the game condition (d= .71).
To address RQ2,wefirst conducted a subjective evaluation with 3 middle school
math teachers of problem difficulty. The teachers rated the 48 intervention problems on
a 5-point Likert scale: 1 - “Very Easy”;5-“Very Difficult”. Using this data, we judged
(a) Immediate posttest (b) Delayed posttest
Fig. 3. Interaction effect of game with gender on learning outcomes
542 B. McLaren et al.
a problem as difficult if the average
rating of the 3 teachers was above 3, and
easy if the average rating was less than
or equal 3. 27 of the problems were
judged as easy, 21 as difficult. To then
assess the impact of the game on diffi-
cult vs. easy problems, we conducted a
regression analysis of the relationship
between the intervention conditions on
the number of errors students made on
each problem.
There is a significant interaction of the game condition and the difficulty level of the
problem on the number of errors students make at the problem level. The interaction
effect is presented in Fig. 4. For the easy problems, the game does not influence the
number of errors the students make; however, for difficult problems, the game leads to
students making significantly fewer errors (d= .09).
Our new data analyses answered our two new research questions. First, females
benefited more from the game than males. This result may be related to the fact that
game achievement is not a focus of Decimal Point. Prior research suggests that male
players can be particularly attracted to games of achievement, while achievement does
not appear to be a key factor in engaging female players [7]. Second, the game appears
to have made difficult problems more tractable, as the game group made significantly
fewer errors on the difficult problems in the intervention than the non-game
group. Better performance on the difficult problems may be a result of students’
higher level of engagement with the game. Games may be a way to engage students in
continuing to higher levels of mastery, even in the face of difficult problems.
References
1. Mayer, R.E.: Computer Games for Learning: An Evidence-Based Approach. MIT Press,
Cambridge (2014)
2. Clark, D.B., Tanner-Smith, E.E., Killingsworth, S.S.: Digital games, design, and learning: a
systematic review and meta-analysis. Rev. Ed. Res. 86(1), 79–122 (2016)
3. McLaren, B.M., Adams, D.M., Mayer, R.E., Forlizzi, J.: A computer-based game that
promotes mathematics learning more than a conventional approach. Int. J. Game Based Learn.
(IJGBL) 7(1), 36–56 (2017). doi:10.4018/IJGBL.2017010103
4. Chi, M.T.H.: Self-explaining expository texts: the dual processes of generating inferences and
repairing mental models. In: Glaser, R. (ed.) Advances in Instructional Psychology, pp. 161–
238. Lawrence Erlbaum Associates, Inc., Mahwah (2000)
5. Lenhart, A., Smith, A., Anderson, M., Duggan, M., Perrin, A.: Teens, Technology and
Friendships. Pew Research Center (2015)
6. Habgood, M.P.J., Ainsworth, S.: Motivating children to learn effectively: exploring the value
of intrinsic integration in educational games. J. Learn. Sci. 20(2), 169–206 (2011)
7. Inkpen, K.: Three important research agendas for educational multimedia: Learning, children,
and gender. In: AACE World Conference on Educational Multimedia and Hypermedia, vol.
97, pp. 521–526 (1997)
Fig. 4. Interaction of difficulty level of problem
with number of errors on each problem
Uncovering Gender and Problem Difficulty Effects in Learning 543
... Digital learning games are instructional platforms that allow students to both engage playfully and learn (Gee, 2007;Mayer, 2014Mayer, , 2019. In recent studies, some learning games have been shown to be more effective for girls than for boys, both in terms of learning and affective outcomes (Arroyo et al., 2013;McLaren et al., 2022;McLaren, Farzan, et al., 2017;Nguyen et al., 2022). These results suggest that games may be particularly effective for reducing gender disparities in math outcomes, including lower levels of motivation and math self-confidence among girls compared to boys (Else-Quest et al., 2010;Reilly et al., 2019). ...
... Our current work explores these questions through studies with the digital learning game Decimal Point, which teaches decimal numbers and operations to middle school students . While Decimal Point has been the subject of many classroom studies over the past 10 years, studies that have explored a variety of gamebased learning questions, one highly consistent finding that emerged across studies is that girls have learned more than boys from the game after using Decimal Point to learn decimals McLaren, Farzan, et al., 2017;Nguyen et al., 2022). ...
... Prior studies have shown that girls learn more from Decimal Point than boys McLaren, Farzan, et al., 2017;Nguyen et al., 2022), and we expect to replicate this effect. If girls are benefitting from the game because they enjoy the game narrative more than boys, then we predict that boys and students with more masculine-typed characteristics will learn more than girls and students with more feminine-typed characteristics in Ocean Adventure, which was specifically designed to be more appealing to boys. ...
Using a Multi-Dimensional Model of Gender to Assess Learning with Different Game-Based Learning Narratives
Conference Paper
Full-text available
  • Oct 2024
Digital learning games can help address gender disparities in math by promoting better learning experiences and outcomes for girls. However, there is a need for more research to understand why some digital learning games might be especially effective for girls studying mathematics. In this study, we assess two possible pathways: that girls might benefit from math games because they reduce the anxiety and evaluation apprehension that girls are more likely to experience when doing math; and that girls might benefit from math games when they enjoy the narrative and thus experience greater engagement. To evaluate these pathways, our work uses multiple dimensions of gender (e.g., gender identity and gender-typed interests, activities, and traits) and surveys of affective experiences to examine the impact of three learning systems with identical learning content: a digital learning game, Decimal Point, that has consistently led to better learning for girls over boys; a new masculine-typed game, Ocean Adventure, developed based on a survey of over 300 students; and a conventional tutoring system. We predicted that girls and students with stronger feminine-typed characteristics would experience less math anxiety in both Decimal Point and Ocean Adventure compared to the tutor. We also predicted that girls and students with stronger feminine-typed characteristics would experience greater engagement and learning with Decimal Point while boys and students with stronger masculine-typed characteristics would experience greater engagement and learning with Ocean Adventure. Consistent with predictions, students with stronger feminine-typed characteristics experienced less anxiety and evaluation apprehension in both games compared to the tutor. This suggests that math learning games may provide a way to address these negative affective experiences. In terms of our measures of engagement, we found that students with stronger masculine-typed characteristics reported greater experience of mastery in the masculine Ocean Adventure; however, this was the only indicator that the more masculine narrative of Ocean Adventure led to different experiences based on gender. This suggests that narrative alone may not have a strong enough effect on students based on gender, especially when other game features are kept constant. Contrary to our predictions, there were no effects of gender identity or condition on learning outcomes, although both masculine-typed and feminine-typed characteristics were negatively associated with learning. Overall, these results point to the value of a multi-dimensional model of gender in assessing learning with a game, the important role learning games can have in reducing math anxiety and evaluation apprehension for girls and students with feminine-typed characteristics, and the nuanced effects of game narratives on experiences with game based learning.
View
... Compounding this issue, many studies have not considered whether digital learning games work in the same ways (and with comparable effectiveness) for different sub-groups of overall student populations. Though some studies have looked at whether games work equally well for different groups (e.g., female vs. male students- Papastergiou, 2009;Chung & Chang, 2017;McLaren et al., 2017b;Tsai, 2017; students of different races- Shin et al., 2012;Kao & Harrell, 2015), this remains a small proportion of the studies on learning in games. Furthermore, as noted by Dele-Ajayi et al. (2018), only a small number of the studies that do check for differences in learning or engagement in terms of student group membership continue on to explicitly investigate why and how these differences are seen. ...
... Given the influence of design features on the choice to game the system, and Decimal Point's overall effect on the prevalence of gaming, there is some reason to anticipate gender differences in gaming the system within Decimal Point. In fact, Decimal Point has led to consistently better learning for female students than male students McLaren et al., 2017bMcLaren et al., , 2022bNguyen et al., 2022). One possible hypothesis is that this may be because female students are more engaged-and thus may game the system less often-than male students when playing Decimal Point. ...
... Research on gender differences in digital learning game outcomes has shown mixed results, with an overall pattern suggesting female students benefit more. Female students have been shown to enjoy learning games more (Adamo-Villani et al., 2008;Chung & Chang, 2017), to be more likely to find a learning game worth playing (Joiner et al., 2011), and to achieve better learning outcomes (Khan et al., 2017a;Klisch et al., 2012;McLaren et al., 2017b;Nguyen et al., 2022;Tsai, 2017). However, other studies report no gender differences in outcomes (Chang et al., 2014;Clark et al., 2011;Dorji et al., 2015;Manero et al., 2016;Papastergiou, 2009). ...
Gaming the system mediates the relationship between gender and learning outcomes in a digital learning game
Article
Full-text available
  • Aug 2024
  • INSTR SCI
Digital learning games have been increasingly adopted in classrooms to facilitate learning and to promote learning outcomes. Contrary to common beliefs, many digital learning games can be more effective for female students than male students in terms of learning and affective outcomes. However, the in-game learning mechanisms that explain these differences remain unclear. In the current study, we re-analyze three retrospective data sets drawn from three studies conducted in different years. These data sets, which involved 213, 197, and 287 students, were collected from a digital learning game that teaches late elementary and middle school students decimal concepts. We re-analyzed these data sets to understand how female and male students differ in the rates of gaming the system, a behavioral measure that reflects a form of disengagement while playing the game. Rates of gaming the system are compared between female and male students within each of the game’s two core instructional activities (i.e. problem-solving and self-explanation) as well as tested in a game vs. non-game condition. We found that female students game the system significantly less than male students in the self-explanation step in the game condition, in all three studies. This difference in the rates of gaming mediates the relationship between gender and learning outcomes, a pattern in which female students tend to learn more than male students, across all three studies. These results suggest that future design iterations of the game could focus on reducing gaming behaviors for male students, which might improve learning outcomes for female students as well. Understanding gender-based differences in game behaviors can inform future game design to promote better learning outcomes for all students.
View
... The Decimal Point students also reported enjoying their experience significantly more than the tutored students, according to a post-game questionnaire. An analysis of learning outcomes by binary gender identity revealed that girls learned significantly more from the game than boys [18]. We subsequently performed experiments with different versions of the game and consistently uncovered the same finding: girls learned significantly more from the game than boys regardless of alterations [3,19]. ...
... In other words, if the self-explanation step in the game was creating the gender effect, then we would no longer expect to see a gender difference in the game when self-explanation was removed. We did not expect to see a gender difference in the non-game platform, regardless of the presence or absence of selfexplanation prompts, as prior research revealed no gender differences in the nongame [18]. ...
... Second, we failed to replicate the gender effect on learning outcomes, although we replicated results showing that boys outperformed girls on the pretest. Girls have learned more than boys across many previous studies with Decimal Point, although this was not the case in the non-game tutor [3,18]. We are unsure why this result failed to replicate in the game condition with self-explanation. ...
Understanding Gender Effects in Game-Based Learning: The Role of Self-Explanation
Conference Paper
Full-text available
  • Jul 2024
We conducted a 2x2 study comparing the digital learning game Decimal Point to a comparable non-game tutor with or without self-explanation prompting. We expected to replicate previous studies showing the game improved learning compared to the non-game tutor, and that self-explanation prompting would enhance learning across platforms. Additionally, prior research with Decimal Point suggested that self-explanation was driving gender differences in which girls learned more than boys. To better understand these effects, we manipulated the presence of self-explanation prompts and incorporated a multidimensional gender measure. We hypothesized that girls and students with stronger feminine-typed characteristics would learn more than boys and students with stronger masculine-typed characteristics in the game with self-explanation condition, but not in the game without self-explanation or in the non-game conditions. Results showed no advantage for the game over the non-game or for including self-explanation, but an analysis of hint usage indicated that students in the game conditions used (and abused) hints more than in the non-game conditions, which in turn was associated with worse learning outcomes. When we controlled for hint use, students in the game conditions learned more than students in the non-game tutor. We replicated a gender effect favoring boys and students with masculine-typed characteristics on the pretest, but there were no gender differences on the posttests. Finally, results indicated that the multidimensional framework explained variance in pretest performance better than a binary gender measure, adding further evidence that this framework may be a more effective, inclusive approach to understanding gender effects in game-based learning.
View
... While conducting a study exploring the use of different types of prompted self-explanation in the context of a digital learning game, Decimal Point , we also investigated whether the game led to gender differences, as found in prior studies with the game (McLaren, Farzan, et al. 2017;Nguyen et al., 2022). Prompted self-explanation is used to promote learning by encouraging students to self-explain what they are studying or how they have solved a problem (Wylie & Chi, 2014). ...
... This question is motivated by the consistent gender effect that has been uncovered in prior studies with Decimal Point, regardless of how we have altered the game intervention (McLaren, Farzan, et al. 2017;Nguyen et al., 2022). Thus, as in these earlier studies, we hypothesized for the present study that female students would learn more than male students from playing the Decimal Point game. ...
... Why do female students consistently benefit more from the Decimal Point game than male students? In prior work, we suggested that Decimal Point, which doesn't display scores or otherwise stress math achievement or competition, may have engaged female students more than male students (McLaren, Farzan, et al., 2017). Some prior work has suggested that male students may be more engaged with achievement-and competitionoriented games (Dele-Ajayi et al., 2018). ...
A Digital Learning Game for Mathematics that Leads to Better Learning Outcomes for Female Students: Further Evidence
Article
  • Sep 2022
Stereotypes about men being better than women at mathematics appear to influence female students’ interest and performance in mathematics. Given the potential motivational benefits of digital learning games, it is possible that games could help to reduce math anxiety, increase self-efficacy, and lead to better learning outcomes for female students. We are exploring this possibility in our work with Decimal Point, a digital learning game that scaffolds practice with decimal operations for 5th and 6th grade students. In several studies with various versions of the game, involving over 800 students across multiple years, we have consistently uncovered a learning advantage for female students with the game. In our most recent investigation of this gender effect, we decided to experiment with a central feature of the game: its use of prompted self-explanation to support student learning. Prior research has suggested that female students might benefit more from self-explanation than male students. In the new study, involving 214 middle school students, we compared three versions of self-explanation in the game – menu-based, scaffolded, and focused – each presenting students with a different type of prompted self-explanation after they solved problems in the game. We found that the focused approach led to more learning across all students than the menu-based approach, a result reported in an earlier paper. In the additional results reported in this paper, we again uncovered the gender effect – female students learned more from the game than male students, regardless of the version of self-explanation – and also found a trend in which female students made fewer self-explanation errors, suggesting they may have been more deliberate and thoughtful in their self-explanations. This self-explanation finding is a possible key to further investigation into how and why we see the gender effect in Decimal Point.
View
... In the process of educational reforms, digitization and AI have become key enablers, which open up new possibilities for the further development of the learning process and the provision of students with high-quality, complete educational experiences. (4) In order to elevate the importance of education to the next level, pre-and university students need to prepare themselves to face the challenges in the classrooms and the current workplace environment. Traditional paradigms in education often do not address the full potential of the students and stifle creativity; this is a problem that also has to be solved. ...
Integrating Artificial Intelligence into Collaborative Teaching Strategies: A Case Study in Smart Classrooms
Article
Full-text available
  • Feb 2025
The purpose of this study is to explore multiple possibilities for applying collaborative learning methodologies in smart classrooms through the use of artificial intelligence. AI technologies allow educators to increase student interest, promote teamwork, and individualize learning for each learner. In this case study, the focus will be placed on AI technologies, including virtual assistants, real-time data analytics, and adaptive learning platforms, to enhance collaborative learning processes. The study shows that collaborative teaching implemented with the assistance of artificial intelligence enhances the performance of students as they are capable of understanding the content thoroughly and addressing the shortcomings of conventional learning methods. Consideration is also given to the various strategies that may be adopted when it comes to the implementation of artificial intelligence in cooperative learning contexts.
View
... In this regard, most studies perform media comparisons to investigate if IVR lessons outperform similar lessons provided via less immersive media such as desktop VR (e.g., , video (e.g., Meyer et al., 2019), or PowerPoint slides (e.g., Parong & Mayer, 2018). These studies consistently find that IVR leads to higher levels of enjoyment (e.g., McLaren et al., 2017) and presence (e.g., Makransky & Lilleholt, 2018;, as well as in some instances higher intrinsic motivation to learn a subject (e.g., Parong & Mayer, 2018;Zhao et al., 2020) or higher selfefficacy (e.g., Meyer et al., 2019). Although recent meta-analyses report that IVR lessons lead to advantages on learning outcomes with small effect sizes compared to non-immersive learning approaches (Coban et al., 2022;Wu et al., 2020), there is great variability across studies with media comparison studies reporting positive (e.g., , mixed (e.g., Jensen & Konradsen, 2018), and negative (e.g., Parong & Mayer, 2018) effects. ...
Facilitating learning in immersive virtual reality: Segmentation, summarizing, both or none?
Article
Full-text available
  • Sep 2022
  • J COMPUT ASSIST LEAR
Introduction This study investigates the effectiveness of the segmentation principle from the cognitive theory of multimedia learning as well as the effectiveness of the generative learning strategy of summarization in immersive virtual reality (IVR) within a sample of preadolescents. Although previous research has supported the effectiveness of these instructional methods in multimedia learning, it remains unclear whether segmentation, summarization or the combination of both are superior to a stand‐alone IVR lesson in facilitating learning. Methods To address this gap, 190 sixth to seventh grade students learned about the human body in an IVR lesson. Students were randomly assigned to one of four experimental conditions: (a) an IVR lesson divided into four segments (segmentation condition); (b) an IVR lesson, where students summarized after the entire lesson (summarizing condition); (c) an IVR lesson presented in four segments, where students summarized after each segment (combined condition); and (d) an IVR lesson without any manipulation (control condition). Results Results indicated that, compared to the control condition, adding segmentation or summarization to an IVR lesson leads to better transfer, but not to acquiring more factual knowledge. Combining the two methods did not improve learning. Conclusion The findings support the evidence that choosing appropriate instructional methods for IVR lessons can foster transfer.
View
View
Decimal Point: A Decade of Learning Science Findings with a Digital Learning Game
Chapter
  • Dec 2024
The McLearn Lab at Carnegie Mellon University (CMU) first designed and developed the artificial intelligence (AI) in education learning game, Decimal Point, in 2013 and 2014 to support middle school children learning decimals and decimal operations. Over a period of 10 years, the McLearn Lab has run a series of classroom experiments with the game, involving over 1,500 elementary and middle school students. In these studies, we have explored a variety of game-based learning and learning science principles and issues, such as whether the game leads to better learning—demonstrated learning gains from a pretest to a posttest and/or a delayed posttest—than a more traditional online instructional approach; whether giving students more agency leads to more learning and enjoyment; whether students benefit from hints and error messages provided during game play; and what types of prompted self-explanation lead to the best learning and enjoyment outcomes. A fascinating finding also emerged during the variety of experiments we conducted: the game consistently led to a gender effect in which girls learned more from the game than boys. In this chapter I will discuss the current state of digital learning games, how we designed and developed Decimal Point, the technology it is built upon—including AI techniques—and the key results of the various experiments we’ve conducted over the years. I conclude by discussing the important game-based learning take-aways from our studies, what we have learned about using a digital learning game as a research platform for exploring learning science principles and issues; and exciting future directions for this line of research.
View
View
The Impact of Gender in Learning With Games: A Consistent Effect in a Math Learning Game
Article
Full-text available
  • Jan 2022
There is an established gender gap in middle school math education, where female students report higher anxiety and lower engagement, which negatively impact their performance and even long-term career choices. This work investigates the role of digital learning games in addressing this issue by studying Decimal Point, a math game that teaches decimal numbers and operations to 5th and 6th graders. Through data from four published studies of Decimal Point, involving 624 students in total, the authors identified a consistent gender difference that was replicated across all studies – male students tended to do better at pretest, while female students tended to learn more from the game. In addition, female students were more careful in answering self-explanation questions, which significantly mediated the relationship between gender and learning gains in two out of four studies. These findings show that learning games can be an effective tool for bridging the gender gap in middle school math education, which in turn contributes to the development of more personalized and inclusive learning platforms.
View
A Computer-Based Game that Promotes Mathematics Learning More than a Conventional Approach
Article
Full-text available
  • Jan 2017
Excitement about learning from computer-based games has been papable in recent years and has led to the development of many educational games. However, there are relatively few sound empirical studes in the scientific literature that have shown the benefits of learning mathematics from games as opposed to more traditional approaches. The empirical study reported in this paper provides evidence that a mathematics educational game can provide superior learning opportunities, as well as be more engaging. In a study involving 153 students from two middle schools, 70 students learned about decimals from playing an educational game-Decimal Point-whereas 83 students learned the same content by a more conventional, computer-based approach. The game led to significantly better gain scores in solving decimal problems, on both an immediate (d = .43) and delayed (d = .37) posttest and was rated as significantly more enjoyable (d = .95). Low prior knowledge students especially benefitted from the game. This paper also summarizes the game's design characteristics.
View
Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis
Article
Full-text available
  • Apr 2015
In this meta-analysis, we systematically reviewed research on digital games and learning for K-16 students. We synthesized comparisons of game versus nongame conditions (i.e., media comparisons) and comparisons of augmented games versus standard game designs (i.e., value-added comparisons). We used random-effects meta-regression models with robust variance estimates to summarize overall effects and explore potential moderator effects. Results from media comparisons indicated that digital games significantly enhanced student learning relative to nongame conditions ( = 0.33, 95% confidence interval [0.19, 0.48], k = 57, n = 209). Results from value-added comparisons indicated significant learning benefits associated with augmented game designs ( = 0.34, 95% confidence interval [0.17, 0.51], k = 20, n = 40). Moderator analyses demonstrated that effects varied across various game mechanics characteristics, visual and narrative characteristics, and research quality characteristics. Taken together, the results highlight the affordances of games for learning as well as the key role of design beyond medium.
View
Motivating Children to Learn Effectively: Exploring the Value of Intrinsic Integration in Educational Games
Article
Full-text available
  • Apr 2011
  • J LEARN SCI
The concept of intrinsic motivation lies at the heart of the user engagement created by digital games. Yet despite this, educational software has traditionally attempted to harness games as extrinsic motivation by using them as a sugar coating for learning content. This article tests the concept of intrinsic integration as a way of creating a more productive relationship between educational games and their learning content. Two studies assessed this approach by designing and evaluating an educational game called Zombie Division to teach mathematics to 7- to 11-year-olds. Study 1 examined the learning gains of 58 children who played either the intrinsic, extrinsic, or control variants of Zombie Division for 2 hr, supported by their classroom teacher. Study 2 compared time on task for the intrinsic and extrinsic variants of the game when 16 children had free choice of which game to play. The results showed that children learned more from the intrinsic version of the game under fixed time limits and spent 7 times longer playing it in free-time situations. Together, these studies offer evidence for the genuine value of an intrinsic approach for creating effective educational games. The theoretical and commercial implications of these findings are discussed.
View
Computer Games for Learning: An Evidence-Based Approach
Book
  • Jul 2014
A comprehensive and up-to-date investigation of what research shows about the educational value of computer games for learning. Many strong claims are made for the educational value of computer games, but there is a need for systematic examination of the research evidence that might support such claims. This book fills that need by providing, a comprehensive and up-to-date investigation of what research shows about learning with computer games. Computer Games for Learning describes three genres of game research: the value-added approach, which compares the learning outcomes of students who learn with a base version of a game to those of students who learn with the base version plus an additional feature; the cognitive consequences approach, which compares learning outcomes of students who play an off-the-shelf computer game for extended periods to those of students who do not; and the media comparative approach, which compares the learning outcomes of students who learn material by playing a game to those of students who learn the same material using conventional media. After introductory chapters that describe the rationale and goals of learning game research as well as the relevance of cognitive science to learning with games, the book offers examples of research in all three genres conducted by the author and his colleagues at the University of California, Santa Barbara; meta-analyses of published research; and suggestions for future research in the field. The book is essential reading for researchers and students of educational games, instructional designers, learning-game developers, and anyone who wants to know what the research has to say about the educational effectiveness of computer games.
View
View
Three Important Research Agendas for Educational Multimedia: Learning, Children, and Gender
Article
  • Dec 1997
: This paper discusses three often-overlooked characteristics of educational multimedia systems that are important for human-computer interaction (HCI) researchers: (a) learning environments have very different goals and approaches than do workplace environments, so HCI research must investigate issues within the context of learning; (b) what works for adults will not necessarily work for children, so HCI research must develop usability guidelines that are appropriate for children; (c) girls and boys interact differently with technology, so HCI research must understand how each gender interacts with computers. We illustrate each of these issues with examples from our own research and explain how the characteristics of educational multimedia systems can lead to computing environments that do not hinder children's learning and do not exclude any children on the basis of gender. Keywords: Human-Computer Interaction (HCI), children, learning, education, gender, educational multimedia, int...
View
Teens, Technology and Friendships
  • A Lenhart
  • A Smith
  • M Anderson
  • M Duggan
  • A Perrin
Lenhart, A., Smith, A., Anderson, M., Duggan, M., Perrin, A.: Teens, Technology and Friendships. Pew Research Center (2015)