Fig 3 - uploaded by Rosta Farzan
Content may be subject to copyright.
Interaction effect of game with gender on learning outcomes

Interaction effect of game with gender on learning outcomes

Source publication
Conference Paper
Full-text available
A prior study showed that middle school students who used the educational game Decimal Point achieved significantly higher gain scores on immediate and delayed posttests of decimal understanding than students who learned with a more conventional computer-based learning tool. This paper reports on new analyses of the data from that study, providing...

Context in source publication

Context 1
... address RQ1 we conducted a regression analysis to predict the relationship between the intervention and learning outcomes for female vs. male students. The results of the main effect in the regression model confirmed the prior learning results cited above. The results for immediate and delayed posttests by gender are summarized in Fig. 3(a) and (b). In terms of both immediate posttest and delayed posttest, there is a significant interaction effect of game condition with ...

Similar publications

Article
Full-text available
It is a worrying factor the results presented in the process of reading the 5th grade students of the brazilian schools. Many students indicate weaknesses in reading skills. Thus, it becomes important to resort to the theories of learning presented by the authors: Jean Piaget (1896-1980), Lev Vygotsky (1896-1934) and Paulo Freire (1921-1997), in an...
Chapter
Full-text available
Learning analytics (LA) dashboards refer to digital tools designed to help learners keep track of their progress and goals. There is growing interest and research around the topic of LA dashboards in online learning environments, with many lessons to be learned by educational game developers and researchers. However, we need more research in this a...
Conference Paper
Full-text available
Early Childhood Education (PAUD) requires an interactive and attractive learning system. The PAUD learning method is also capable of stimulating the knowledge development of the students, especially in the cognitive aspect. Such an aspect enters into an educational game where there is an element of education obtained from something that exists and...
Conference Paper
Full-text available
Assessment is used by a teacher in order to measure students' ability in learning process. There are many kinds of assessment which can be employed to assess students in the classroom. The objective of this research is to describe assessment used for English teaching in high school. The design of this research was a naturalistic study. Object of th...

Citations

... Digital learning games are instructional platforms that allow students to both engage playfully and learn (Gee, 2007;Mayer, 2014Mayer, , 2019. In recent studies, some learning games have been shown to be more effective for girls than for boys, both in terms of learning and affective outcomes (Arroyo et al., 2013;McLaren et al., 2022;McLaren, Farzan, et al., 2017;Nguyen et al., 2022). These results suggest that games may be particularly effective for reducing gender disparities in math outcomes, including lower levels of motivation and math self-confidence among girls compared to boys (Else-Quest et al., 2010;Reilly et al., 2019). ...
... Our current work explores these questions through studies with the digital learning game Decimal Point, which teaches decimal numbers and operations to middle school students . While Decimal Point has been the subject of many classroom studies over the past 10 years, studies that have explored a variety of gamebased learning questions, one highly consistent finding that emerged across studies is that girls have learned more than boys from the game after using Decimal Point to learn decimals McLaren, Farzan, et al., 2017;Nguyen et al., 2022). ...
... Prior studies have shown that girls learn more from Decimal Point than boys McLaren, Farzan, et al., 2017;Nguyen et al., 2022), and we expect to replicate this effect. If girls are benefitting from the game because they enjoy the game narrative more than boys, then we predict that boys and students with more masculine-typed characteristics will learn more than girls and students with more feminine-typed characteristics in Ocean Adventure, which was specifically designed to be more appealing to boys. ...
Conference Paper
Full-text available
Digital learning games can help address gender disparities in math by promoting better learning experiences and outcomes for girls. However, there is a need for more research to understand why some digital learning games might be especially effective for girls studying mathematics. In this study, we assess two possible pathways: that girls might benefit from math games because they reduce the anxiety and evaluation apprehension that girls are more likely to experience when doing math; and that girls might benefit from math games when they enjoy the narrative and thus experience greater engagement. To evaluate these pathways, our work uses multiple dimensions of gender (e.g., gender identity and gender-typed interests, activities, and traits) and surveys of affective experiences to examine the impact of three learning systems with identical learning content: a digital learning game, Decimal Point, that has consistently led to better learning for girls over boys; a new masculine-typed game, Ocean Adventure, developed based on a survey of over 300 students; and a conventional tutoring system. We predicted that girls and students with stronger feminine-typed characteristics would experience less math anxiety in both Decimal Point and Ocean Adventure compared to the tutor. We also predicted that girls and students with stronger feminine-typed characteristics would experience greater engagement and learning with Decimal Point while boys and students with stronger masculine-typed characteristics would experience greater engagement and learning with Ocean Adventure. Consistent with predictions, students with stronger feminine-typed characteristics experienced less anxiety and evaluation apprehension in both games compared to the tutor. This suggests that math learning games may provide a way to address these negative affective experiences. In terms of our measures of engagement, we found that students with stronger masculine-typed characteristics reported greater experience of mastery in the masculine Ocean Adventure; however, this was the only indicator that the more masculine narrative of Ocean Adventure led to different experiences based on gender. This suggests that narrative alone may not have a strong enough effect on students based on gender, especially when other game features are kept constant. Contrary to our predictions, there were no effects of gender identity or condition on learning outcomes, although both masculine-typed and feminine-typed characteristics were negatively associated with learning. Overall, these results point to the value of a multi-dimensional model of gender in assessing learning with a game, the important role learning games can have in reducing math anxiety and evaluation apprehension for girls and students with feminine-typed characteristics, and the nuanced effects of game narratives on experiences with game based learning.
... Compounding this issue, many studies have not considered whether digital learning games work in the same ways (and with comparable effectiveness) for different sub-groups of overall student populations. Though some studies have looked at whether games work equally well for different groups (e.g., female vs. male students- Papastergiou, 2009;Chung & Chang, 2017;McLaren et al., 2017b;Tsai, 2017; students of different races- Shin et al., 2012;Kao & Harrell, 2015), this remains a small proportion of the studies on learning in games. Furthermore, as noted by Dele-Ajayi et al. (2018), only a small number of the studies that do check for differences in learning or engagement in terms of student group membership continue on to explicitly investigate why and how these differences are seen. ...
... Given the influence of design features on the choice to game the system, and Decimal Point's overall effect on the prevalence of gaming, there is some reason to anticipate gender differences in gaming the system within Decimal Point. In fact, Decimal Point has led to consistently better learning for female students than male students McLaren et al., 2017bMcLaren et al., , 2022bNguyen et al., 2022). One possible hypothesis is that this may be because female students are more engaged-and thus may game the system less often-than male students when playing Decimal Point. ...
... Research on gender differences in digital learning game outcomes has shown mixed results, with an overall pattern suggesting female students benefit more. Female students have been shown to enjoy learning games more (Adamo-Villani et al., 2008;Chung & Chang, 2017), to be more likely to find a learning game worth playing (Joiner et al., 2011), and to achieve better learning outcomes (Khan et al., 2017a;Klisch et al., 2012;McLaren et al., 2017b;Nguyen et al., 2022;Tsai, 2017). However, other studies report no gender differences in outcomes (Chang et al., 2014;Clark et al., 2011;Dorji et al., 2015;Manero et al., 2016;Papastergiou, 2009). ...
Article
Full-text available
Digital learning games have been increasingly adopted in classrooms to facilitate learning and to promote learning outcomes. Contrary to common beliefs, many digital learning games can be more effective for female students than male students in terms of learning and affective outcomes. However, the in-game learning mechanisms that explain these differences remain unclear. In the current study, we re-analyze three retrospective data sets drawn from three studies conducted in different years. These data sets, which involved 213, 197, and 287 students, were collected from a digital learning game that teaches late elementary and middle school students decimal concepts. We re-analyzed these data sets to understand how female and male students differ in the rates of gaming the system, a behavioral measure that reflects a form of disengagement while playing the game. Rates of gaming the system are compared between female and male students within each of the game’s two core instructional activities (i.e. problem-solving and self-explanation) as well as tested in a game vs. non-game condition. We found that female students game the system significantly less than male students in the self-explanation step in the game condition, in all three studies. This difference in the rates of gaming mediates the relationship between gender and learning outcomes, a pattern in which female students tend to learn more than male students, across all three studies. These results suggest that future design iterations of the game could focus on reducing gaming behaviors for male students, which might improve learning outcomes for female students as well. Understanding gender-based differences in game behaviors can inform future game design to promote better learning outcomes for all students.
... The Decimal Point students also reported enjoying their experience significantly more than the tutored students, according to a post-game questionnaire. An analysis of learning outcomes by binary gender identity revealed that girls learned significantly more from the game than boys [18]. We subsequently performed experiments with different versions of the game and consistently uncovered the same finding: girls learned significantly more from the game than boys regardless of alterations [3,19]. ...
... In other words, if the self-explanation step in the game was creating the gender effect, then we would no longer expect to see a gender difference in the game when self-explanation was removed. We did not expect to see a gender difference in the non-game platform, regardless of the presence or absence of selfexplanation prompts, as prior research revealed no gender differences in the nongame [18]. ...
... Second, we failed to replicate the gender effect on learning outcomes, although we replicated results showing that boys outperformed girls on the pretest. Girls have learned more than boys across many previous studies with Decimal Point, although this was not the case in the non-game tutor [3,18]. We are unsure why this result failed to replicate in the game condition with self-explanation. ...
Conference Paper
Full-text available
We conducted a 2x2 study comparing the digital learning game Decimal Point to a comparable non-game tutor with or without self-explanation prompting. We expected to replicate previous studies showing the game improved learning compared to the non-game tutor, and that self-explanation prompting would enhance learning across platforms. Additionally, prior research with Decimal Point suggested that self-explanation was driving gender differences in which girls learned more than boys. To better understand these effects, we manipulated the presence of self-explanation prompts and incorporated a multidimensional gender measure. We hypothesized that girls and students with stronger feminine-typed characteristics would learn more than boys and students with stronger masculine-typed characteristics in the game with self-explanation condition, but not in the game without self-explanation or in the non-game conditions. Results showed no advantage for the game over the non-game or for including self-explanation, but an analysis of hint usage indicated that students in the game conditions used (and abused) hints more than in the non-game conditions, which in turn was associated with worse learning outcomes. When we controlled for hint use, students in the game conditions learned more than students in the non-game tutor. We replicated a gender effect favoring boys and students with masculine-typed characteristics on the pretest, but there were no gender differences on the posttests. Finally, results indicated that the multidimensional framework explained variance in pretest performance better than a binary gender measure, adding further evidence that this framework may be a more effective, inclusive approach to understanding gender effects in game-based learning.
... While conducting a study exploring the use of different types of prompted self-explanation in the context of a digital learning game, Decimal Point , we also investigated whether the game led to gender differences, as found in prior studies with the game (McLaren, Farzan, et al. 2017;Nguyen et al., 2022). Prompted self-explanation is used to promote learning by encouraging students to self-explain what they are studying or how they have solved a problem (Wylie & Chi, 2014). ...
... This question is motivated by the consistent gender effect that has been uncovered in prior studies with Decimal Point, regardless of how we have altered the game intervention (McLaren, Farzan, et al. 2017;Nguyen et al., 2022). Thus, as in these earlier studies, we hypothesized for the present study that female students would learn more than male students from playing the Decimal Point game. ...
... Why do female students consistently benefit more from the Decimal Point game than male students? In prior work, we suggested that Decimal Point, which doesn't display scores or otherwise stress math achievement or competition, may have engaged female students more than male students (McLaren, Farzan, et al., 2017). Some prior work has suggested that male students may be more engaged with achievement-and competitionoriented games (Dele-Ajayi et al., 2018). ...
Article
Stereotypes about men being better than women at mathematics appear to influence female students’ interest and performance in mathematics. Given the potential motivational benefits of digital learning games, it is possible that games could help to reduce math anxiety, increase self-efficacy, and lead to better learning outcomes for female students. We are exploring this possibility in our work with Decimal Point, a digital learning game that scaffolds practice with decimal operations for 5th and 6th grade students. In several studies with various versions of the game, involving over 800 students across multiple years, we have consistently uncovered a learning advantage for female students with the game. In our most recent investigation of this gender effect, we decided to experiment with a central feature of the game: its use of prompted self-explanation to support student learning. Prior research has suggested that female students might benefit more from self-explanation than male students. In the new study, involving 214 middle school students, we compared three versions of self-explanation in the game – menu-based, scaffolded, and focused – each presenting students with a different type of prompted self-explanation after they solved problems in the game. We found that the focused approach led to more learning across all students than the menu-based approach, a result reported in an earlier paper. In the additional results reported in this paper, we again uncovered the gender effect – female students learned more from the game than male students, regardless of the version of self-explanation – and also found a trend in which female students made fewer self-explanation errors, suggesting they may have been more deliberate and thoughtful in their self-explanations. This self-explanation finding is a possible key to further investigation into how and why we see the gender effect in Decimal Point.
... In this regard, most studies perform media comparisons to investigate if IVR lessons outperform similar lessons provided via less immersive media such as desktop VR (e.g., , video (e.g., Meyer et al., 2019), or PowerPoint slides (e.g., Parong & Mayer, 2018). These studies consistently find that IVR leads to higher levels of enjoyment (e.g., McLaren et al., 2017) and presence (e.g., Makransky & Lilleholt, 2018;, as well as in some instances higher intrinsic motivation to learn a subject (e.g., Parong & Mayer, 2018;Zhao et al., 2020) or higher selfefficacy (e.g., Meyer et al., 2019). Although recent meta-analyses report that IVR lessons lead to advantages on learning outcomes with small effect sizes compared to non-immersive learning approaches (Coban et al., 2022;Wu et al., 2020), there is great variability across studies with media comparison studies reporting positive (e.g., , mixed (e.g., Jensen & Konradsen, 2018), and negative (e.g., Parong & Mayer, 2018) effects. ...
Article
Full-text available
Introduction This study investigates the effectiveness of the segmentation principle from the cognitive theory of multimedia learning as well as the effectiveness of the generative learning strategy of summarization in immersive virtual reality (IVR) within a sample of preadolescents. Although previous research has supported the effectiveness of these instructional methods in multimedia learning, it remains unclear whether segmentation, summarization or the combination of both are superior to a stand‐alone IVR lesson in facilitating learning. Methods To address this gap, 190 sixth to seventh grade students learned about the human body in an IVR lesson. Students were randomly assigned to one of four experimental conditions: (a) an IVR lesson divided into four segments (segmentation condition); (b) an IVR lesson, where students summarized after the entire lesson (summarizing condition); (c) an IVR lesson presented in four segments, where students summarized after each segment (combined condition); and (d) an IVR lesson without any manipulation (control condition). Results Results indicated that, compared to the control condition, adding segmentation or summarization to an IVR lesson leads to better transfer, but not to acquiring more factual knowledge. Combining the two methods did not improve learning. Conclusion The findings support the evidence that choosing appropriate instructional methods for IVR lessons can foster transfer.
... With over 150 students participating across the two conditions, the game group showed significantly more learning and the game was rated by students as significantly more engaging than the traditional computer-based tutor. Subsequent analyses of this data showed that female students benefited more from the game than male students and the game made difficult problems more tractable, as the game group made significantly fewer errors on the difficult problems than the non-game group [21]. More recently, the game was used to explore student agency, as described earlier in this paper [26]. ...
Conference Paper
How do the features of a learning environment's user interface impact learners' agency and, further, their learning? We explored this question in the context of Decimal Point, a digital learning game designed to support middle school students in learning decimals. Previous studies of the game showed that giving students the ability to choose the order and number of mini-games to play did not significantly impact their learning outcomes compared to a condition without choice. In this paper we explore whether some elements of the game's interface may have inadvertently exerted indirect control over students' choice, leading to the previous effects. We conducted a classroom study using a new version of the game that varied whether students saw a visual path connecting mini-games on the game map to modulate the level of indirect control students would experience with an implied ordering. Ultimately, we found that students in the no-line condition exercised significantly more agency but did not learn any less than the line condition. These results suggest that indirect control can be a subtle but powerful way to direct student attention in digital learning games.
... [20] compared 75 game-playing students to 83 non-game-playing students and found that the game-playing students enjoyed their experience more and learned more. In subsequent data analyses, we also found that female students benefited more from Decimal Point than male students, and the game made difficult problems more tractable for all students, as students in the game condition made significantly fewer errors on the difficult problems than students in the non-game condition [19]. In this paper, we describe a study to explore how extending the agency feature of the game might alter student learning. ...
Chapter
Full-text available
A key feature of most computer-based games is agency: the capability for students to make their own decisions in how they play. Agency is assumed to lead to engagement and fun, but may or may not be helpful to learning. While the best learners are often good self-regulated learners, many students are not, only benefiting from instructional choices made for them. In the study presented in this paper, involving a total of 158 fifth and sixth grade students, children played a mathematics learning game called Decimal Point, which helps middle-school students learn decimals. One group of students (79) played and learned with a low-agency version of the game, in which they were guided to play all “mini-games” in a prescribed sequence. The other group of students (79) played and learned with a high-agency version of the game, in which they could choose how many and in what order they would play the mini-games. The results show there were no significant differences in learning or enjoyment across the low and high-agency conditions. A key reason for this may be that students across conditions did not substantially vary in the way they played, perhaps due to the indirect control features present in the game. It may also be the case that the young students who participated in this study did not exercise their agency or self-regulated learning. This work is relevant to the AIED community, as it explores how game-based learning can be adapted. In general, once we know which game and learning features lead to the best learning outcomes, as well as the circumstances that maximize those outcomes, we can better design AI-powered, adaptive games for learning.
... Bruce M. McLaren is the main contact person for this workshop, a Research Associate Professor at CMU and the current president of the International Society of Artificial Intelligence in Education (IAIED). Bruce has broad and deep experience with educational technology and learning science research, particularly with intelligent tutoring systems ( Aleven et al., 2016;McLaren, Adams, & Mayer, 2015), e-Learning principles McLaren, DeLeeuw, & Mayer, 2011), and educational games McLaren, Farzan et al, 2017). Bruce was the PI in the development of and experimentation with an educational game called Decimal Point, focused on helping middle school students learn decimals. ...
Conference Paper
This paper describes a "data-driven educational game design" CHI workshop. The intent of the workshop is to bring together experts from CHI, educational games, learning science and data analytics to discuss how game playing works for learning and how games can be better designed to lead to engagement and learning. The outcome of the workshop will be a journal paper that summarizes the current state-of-the-art in data-driven educational game design and provides recommendations for the way forward for educational game designers and developers.
Chapter
The McLearn Lab at Carnegie Mellon University (CMU) first designed and developed the artificial intelligence (AI) in education learning game, Decimal Point, in 2013 and 2014 to support middle school children learning decimals and decimal operations. Over a period of 10 years, the McLearn Lab has run a series of classroom experiments with the game, involving over 1,500 elementary and middle school students. In these studies, we have explored a variety of game-based learning and learning science principles and issues, such as whether the game leads to better learning—demonstrated learning gains from a pretest to a posttest and/or a delayed posttest—than a more traditional online instructional approach; whether giving students more agency leads to more learning and enjoyment; whether students benefit from hints and error messages provided during game play; and what types of prompted self-explanation lead to the best learning and enjoyment outcomes. A fascinating finding also emerged during the variety of experiments we conducted: the game consistently led to a gender effect in which girls learned more from the game than boys. In this chapter I will discuss the current state of digital learning games, how we designed and developed Decimal Point, the technology it is built upon—including AI techniques—and the key results of the various experiments we’ve conducted over the years. I conclude by discussing the important game-based learning take-aways from our studies, what we have learned about using a digital learning game as a research platform for exploring learning science principles and issues; and exciting future directions for this line of research.