ArticlePDF Available

Effects of a mathematics game-based learning environment on primary school students' adaptive number knowledge


Abstract and Figures

Developing adaptive expertise with arithmetic problem solving is a much desired aim of primary school mathematics education. However, there are very few practical tools for teachers that would aid reaching this complex mathematical learning goal. The aim of the present study was to test the effects of a game-based learning environment in supporting primary school students' adaptive number knowledge and related arithmetic skills. Participants were 1168 students in grades four, five, and six. Classes were randomized in two conditions: in the experimental group regular mathematics teaching was enriched with gameplay using the Number Navigation Game (NNG), and in the control group students continued according to their regular math curriculum. An experimental design with pre- and post-test was used to measure students' adaptive number knowledge, arithmetic fluency, and pre-algebra knowledge. Overall, results showed that the experimental group outperformed the control group on adaptive number knowledge and math fluency. Results showed varying effects of the training in different grade levels, with more pronounced improvement of students' adaptive number knowledge in grade five. Game performance was related to the experimental group's post-test scores even after controlling for pre-test scores and grade. Results suggest that the NNG is effective in enhancing different types of arithmetic skills and knowledge in different grades of primary school education and can provide teachers with a practical and flexible tool to extend their regular classroom practice.
Content may be subject to copyright.
Effects of a Mathematics Game-Based Learning Environment on Primary School Students’ Adaptive
Number Knowledge
Boglárka Brezovszky1, Jake McMullen1, Koen Veermans1, Minna M. Hannula-Sormunen1,2, Gabriela
Rodríguez-Aflecht 1, Nonmanut Pongsakdi1 Eero Laakkonen1 & Erno Lehtinen1
University of Turku
Author Note
1Centre for Research on Learning and Instruction and Department of Teacher Education, University
of Turku, Finland
2 Department of Teacher Education and Turku Institute for Advanced Studies, University of Turku,
Published in Computers & Education 128 (2019) 63-74
Corresponding Author:
Boglárka Brezovszky
Centre for Research on Learning and Instruction, Department of Teacher Education
Turun Yliopisto, 20014, Finland.
This work was supported by the Academy of Finland [grant number 274163; PI: Erno Lehtinen] and
the Academy of Finland DECIN project [grant number 278579; PI: Minna Hannula-Sormunen].
1. Introduction
Flexible mathematical thinking and the adaptive use of arithmetic strategies have been
highlighted by researchers of mathematics education (Baroody, 2003; Nunes, Dorneles, Lin, &
Rathgeb-Schnierer, 2016) and flexibility is an important aim of national mathematics education
standards and curricula in many countries (e.g., Finnish National Agency for Education, 2014; NCTM
2014). Flexibility is necessary to apply problem solving procedures in new contexts and across
different representations, which is crucial for a transferable mathematical proficiency to everyday life
(NCTM 2014; Nunes et al., 2016). However, pedagogical recommendations and models for enhancing
flexibility and adaptivity are rare and mostly focused on teaching the use of a few strategies
(Verschaffel, Luwel, Torbeyns, & Van Dooren, 2009), which are rarely applied in practice in spite of a
strong curricular emphasis (Hickendorff, 2017). One explanation for this disconnect could be that, in
order to flexibly apply arithmetic strategies in varying situations, it is not enough to teach these
strategies, but students need to develop a well-connected mental representation of the natural
number system which makes it possible for them to notice when and which strategies are applicable
(Baroody, 2003; Brezovszky et al., 2015; Lehtinen, et al., 2015; Threlfall, 2002, 2009; Verschaffel et al.,
Indeed, regular classroom instruction may struggle to provide students with the opportunity to
experience the rich relations of numbers and operations (Baroody, 2003; Threlfall, 2009) and the type
of practice needed for the development of flexible and adaptive mathematical skills (Lehtinen,
Hannula-Sormunen, McMullen, & Gruber, 2017). Thus, it is important to study if game-based learning
could be used in enhancing this highly emphasized, but pedagogically under-supported, aim of
mathematics education. The open-ended and flexible nature of game-based learning environments
can be ideal to support this intensive practice in an engaging way (Devlin, 2011; Ke, 2009; Squire,
2008), and the technological affordances allow the training to be scalable (Rutherford et al., 2014).
These features of the game-based learning environment could provide teachers with an ideal
complementary tool for enriching their methods in developing flexibility and adaptivity with
arithmetic problem solving. The present study explores the potential of the Number Navigation
Game (NNG) in strengthening fourth, fifth, and sixth grade students’ flexible and adaptive arithmetic
skills. Gameplay with the NNG is expected to develop a more well connected representation of the
arithmetic relations between natural numbers in students, which should enable them to become
more adaptive in their arithmetic problem solving.
1.1. Developing flexibility and adaptivity with arithmetic problem solving by strengthening
students’ adaptive number knowledge
The definition of flexibility varies greatly, but in general it is described as: (a) having a rich
repertoire of arithmetic problem solving procedures, (b) being able to flexibly switch between these
procedures, and (c) being adaptive with selecting strategies efficiently (Heinze, Star, & Verschaffel,
2009; Star & Rittle-Johnson, 2008; Verschaffel et al., 2009). It is problematic however that the
meaning of flexible and adaptive can be very subjective, as it might depend on the characteristics of
the problem, on personal preferences, or the context (Verschaffel et al., 2009), and efficient solutions
can sometimes be described as elegant and sometimes as quick or fast (Heinze et al., 2009). With so
much variability in what it means to be flexible and adaptive with arithmetic problem solving it is
equally unclear what type of instructional design can support the development of this knowledge and
It is suggested that instead of focusing on building a strategy repertoire, more emphasis should be
given to supporting students in developing an understanding of the qualities of numbers and
numerical relations (Threlfall, 2009). Studies show that both mathematically proficient primary school
students (Blöte, Klein, & Beishuizen, 2000; Blöte, Van der Burg, & Klein, 2001) and expert
mathematicians (Star & Newton, 2009) are likely to take into account the characteristics of numbers
in a problem when deciding on solution strategies. This approach emphasizes the subjective nature of
flexibility and shifts the focus towards the process of problem solving during which efficient strategies
emerge and are applied adaptively as important features of a problem become visible (Baroody, 2003;
Schneider, Rittle-Johnson, & Star, 2011; Threlfall, 2002, 2009; Verschaffel et al., 2009). From a more
practical perspective, it is suggested that for students to recognize important features of a problem
they need to develop a well-connected understanding of numerical relations for which they need
opportunities to practice with various number combinations and numerical relations making their
own discoveries (Baroody, 2003; Verschaffel et al., 2009).
The present study tested the efficiency of a game-based learning environment for strengthening
these numerical relations defined as adaptive number knowledge. Adaptive number knowledge is a
component of adaptivity with whole-number arithmetic and it refers to the well-connected, network-
like knowledge of numerical characteristics and arithmetic relations between numbers (McMullen et
al., 2016, 2017). Adaptive number knowledge enables the recognition and use of key numbers in
adaptive arithmetic problem solving. This can include recognizing numbers with many factors or
multiples, recognizing numbers in equations that are close to other numbers that are easy to work
with, or the use of the base-ten structure and basic arithmetic principles like associativity or
commutativity (McMullen et al., 2017). Adaptive number knowledge is related to arithmetic fluency
and knowledge of arithmetic concepts and is a unique predictor of pre-algebra skills (McMullen et al.,
2017). This suggests that having a well-connected representation of numerical relations is not only
related to faster and more efficient problem solving, but also to a more complex understanding of
arithmetic relations, which can be foundational to algebraic thinking (McMullen et al., 2017; Nunes et
al., 2016).
Designing tools for developing adaptive number knowledge in the context of regular classroom
teaching can be challenging. In order for students to develop their own flexible procedures they
should be provided opportunities for extensive deliberate practice with various combinations of
numbers and operations (Baroody, 2003; Lehtinen et al., 2017; Verschaffel et al., 2009). This requires
extra resources from the teacher in developing and using individualized training methods. Indeed, the
few existing intervention studies which are not limited to teaching a few strategies but aim at
supporting students’ discovery of flexible and adaptive problem solving procedures are case studies
or have a highly controlled experimental design using small samples. For example, case studies by
Heirdsfield and collegues (Heirdsfield & Cooper, 2004; Heirdsfield, 2011) suggest that using external
representations like the number line or the number square can be useful in promoting students’
engagement in discovering their own strategies when performing mental arithmetic. Additionally,
there is evidence that reflecting on ones’ solutions, comparing and contrasting different solution
methods or re-solving the same problem using an alternative procedure can promote students’
flexibility both in mental arithmetic and in equation solving (Blöte et al., 2000; Rittle-Johnson & Star,
2009; Star & Seifert, 2006).
The present study tested the effectiveness of the NNG game-based learning environment in
strengthening primary school students’ adaptive number knowledge and related arithmetic skills.
Using a game-based format allows extensive opportunities for practice in an engaging open-ended
context that does not pre-define ‘optimal’ solutions but prompts students to reflect on the efficiency
of various alternatives. Through careful design, discovery learning and exploration are organic
components of game-based learning (Ke, 2009; Squire, 2008) and including appropriate rules and
challenges can promote reflection (Ke, 2008; Kiili, 2007). Additionally, due to technological
affordances, the training is scalable and can take into account individual differences (Rutherford et al.,
2014). This format made it possible to apply the training in naturalistic classroom settings on a large
scale, which was not possible in previous interventions with similar aims. It is expected that the NNG
would provide teachers with an ideal complementary tool for enriching their methods for developing
flexibility and adaptivity with arithmetic problem solving and provide a novel training that can be
flexibly used over three different grade levels
1.2. Developing adaptive number knowledge using game-based learning
Digital games can be powerful in presenting complex mathematical concepts as they can provide
an alternative media where interaction with and exploration of the content is inherent (Devlin, 2011;
Ke, 2009; Lowrie & Jorgensen, 2015). However, it is clear now that games as such do not represent a
magic bullet and they do not automatically make the learning experience more motivating or
effective (Wouters & Oostendorp, 2013; Wouters, van Nimwegen, van Oostendorp, & van der Spek,
2013; Young et al., 2012). As with most media, it is a question of a meaningful translation and
integration of the learning content into the game context which makes game-based learning effective.
Thus, the added value of game-based learning depends on the design process where the learning
content integrated and not just added on top of the game-based media. This allows students to
interact with the content in a novel and alternative way that would otherwise be unavailable or
difficult to achieve in the traditional classroom practice (Arnab et al., 2015; Devlin, 2011; Habgood &
Ainsworth, 2011).
In spite of these affordances of game-based learning, many interventions aim to simply develop
basic arithmetic skills or to strengthen already acquired skills (Cheung & Slavin, 2013; Li & Ma, 2010).
Empirically tested game-based learning environments which target the development of complex
mathematical skills and knowledge in primary school mathematics education are rare. Additionally,
some exiting examples can be limited by the domain, for example training multiplicative reasoning
(Bakker, van den Heuvel-Panhuizen, & Robitzsch, 2015) or the understanding of multiplicative
relations (Habgood & Ainsworth, 2011) and only few game-based learning environments aim for more
holistic mathematical learning outcomes such as flexibility and adaptivity with arithmetic problem
solving. Such examples include the game-based intervention by Pope and Mangram (2015) which
aimed at training students’ flexibility with numbers and operations and an understanding of
properties of numbers, or the large-scale intervention by van den Heuvel-Panhuizen, Kolovou, and
Robitzsch (2013) which aimed to promote the development of reasoning about relations between
quantities that are foundational to early algebra knowledge.
Although the learning aims of the NNG are quite close to the ones described by Pope and
Mangram (2015) and van den Heuvel-Panhuizen and collegues (2013), that is the flexible use and
recognition of numerical relations in arithmetic problem solving, the format of the NNG allows for a
different experience with number combinations and numerical relations. Additionally, unlike previous
game-based learning environments (e.g. Bakker et al., 2015; Habgood & Ainsworth, 2011) the NNG
aims to develop the understanding of numerical relations, without limiting the scope of the game to
one type of arithmetic operation (e.g. multiplicative or additive relations). Instead, the aim of the
game is to develop the recognition and use of various numerical relations within the system of natural
numbers as a whole. To our knowedge, the NNG represents the first attempt to develop elemantary
school students’ adaptive number knolwedge by strengthening their understanding of numerical
Based on empirical results and theoretical suggestions for developing adaptive number
knowledge, the game design should be able to (a) maintain students’ long-term engagement to
mentally execute and compare a large number of various equations in a meaningful way (Baroody,
2003; Threlfall, 2009; Verschaffel et al., 2009), (b) be flexible enough to promote students’ curiosity in
searching for alternative solutions (Rodríguez-Aflecht et al., 2018), (c) provide students with an
external representation that supports the formation of rich networks of numbers and operations
(Mulligan & Mitchelmore, 2013), and (d) promote reflection on the quality and efficiency of their
solution process (Heirdsfield, 2011; Rittle-Johnson & Star, 2009; Star & Seifert, 2006). As the learning
content is integrated within the core game mechanics (Devlin, 2011; Habgood & Ainsworth, 2011;
Salen & Zimmerman, 2003), progress in the NNG means performing mental calculations and
combining and comparing various alternative number-operation combinations. The hundred-square
was selected as an external representation as it can be flexibly used to work within the system of
natural numbers, decomposing and recombining numbers, and provides a clear visual representation
of numerical relations and the base-ten system (Laski, Ermakova, & Vasilyeva, 2014).
Figure 1. Example of a NNG map in the energy scoring mode (harbour at number 100, first target
material at number 44).
Within the game, different game modes were developed to engage players in searching for
alternative solutions and reflect on the efficiency of these different solutions. For example, on Figure
1 the game mode requires the player to minimize the sum of numbers used in their equations, so the
player has to look for key numbers which are useful to achieve this goal. One useful solution can be to
realize that both 90 and 88 are ideal to move forward using division with small numbers. The game is
open-ended, therefore there are no right or wrong answers but players need to select their
calculations considering the game rules and the position of islands (which define the available
numbers in a map) or the placement of starting and target numbers. Taking into account these
challenges, players need to mentally execute and compare several alternative routes when selecting
their calculations. It is this type of mental practice with various combinations of numbers and
operations which represents the core game action that is expected to develop students’ adaptive
number knowledge by strengthening the recognition and use of numerical characteristics and
relations during arithmetic problem solving (Brezovszky, Lehtinen, McMullen, Rodriguez, & Veermans,
2013; Brezovszky et al., 2015; McMullen et al., 2016, 2017).
Results of a pilot testing of the NNG showed that players executed around 180 calculations in an
hour of gameplay and used a large variety of number combinations (Brezovszky et al., 2013). Players
were likely to compare and contrast their solutions prior to execution and reflect on their solution
during gameplay. Additionally, results of a pilot intervention using a prototype of the NNG showed an
increase in sixth grade students’ adaptive number knowledge and math fluency after a seven-week
long training (Brezovszky et al., 2015). The present study scales up this training and explores the
effectiveness of the NNG in the development of adaptive number knowledge, math fluency, and pre-
algebra knowledge of primary school students in grades four to six.
2. Current study
The aim of the present study was to explore the effects of a 10-week long training of regular
mathematics teaching enriched by the NNG game-based learning environment on the development of
primary school students’ adaptive number knowledge, arithmetic fluency, and pre-algebra
knowledge. The effectiveness of the game-based learning environment was examined in an
ecologically valid setting where, in line with the Finnish educational context, teachers had the
freedom to decide about the practicalities of the gameplay. Furthermore, the study explored the
relationship between game performance and mathematical learning outcomes. Accordingly, the
present study asked the following research questions
2.1. How does training with the NNG affect the development of primary school students’
adaptive number knowledge, arithmetic fluency, and pre-algebra knowledge in different
grade levels?
An important aim of primary school mathematics education is to enhance the creative and flexible
use of mathematical knowledge by teaching alternative methods of problem solving (Nunes et al.,
2016; NCTM 2014). However, recent research shows that solely teaching strategies is insufficient
(Hickendorff, 2017). For students to develop more flexibility and adaptivity with arithmetic problem
solving they need to have a chance to practice with various number-operation combinations
(Baroody, 2003; Verschaffel et al., 2009). Research has shown advantageous numerical connections
can be noticed by those mathematical experts who have a dense and strong network of numerical
relations (Dowker, 1992). A well-connected representation of the natural number system enables
noticing of strategically important numbers and leads to a more frequent use of flexible strategies
(Heirdsfield & Cooper, 2004), which has been shown to be related to better general math abilities
(Star & Seifert, 2006). As well, adaptive number knowledge has been shown to be related to
arithmetic skills and knowledge (McMullen et al., 2016), and later pre-algebra knowledge (McMullen
et al., 2017).
In the NNG, students are encouraged to explore combinations of numbers and operations, look
for key numbers, and discover number patterns. Thus, it is expected that as a result of this repeated
practice students will develop their recognition and use of different numerical characteristics and
arithmetic relations, as indexed by their adaptive number knowledge, and also develop their
arithmetic fluency and pre-algebra knowledge. The NNG has a complex design with different game
features aimed at triggering different types of mathematical thinking. Thus, it can be anticipated that
students in the experimental classes, where regular teaching was enriched with the NNG play,
outperform students in the control classes but that the effects will be varying in different grade levels.
2.2. Does students’ performance on the NNG affect the development of mathematical learning
Studies in the domain of education and game-based learning are often criticized for failing to align
game goals and learning goals (Devlin, 2011; Young et al., 2012). In the NNG the educational content
is integrated within the core game mechanics (Habgood & Ainsworth, 2011; Lehtinen et al., 2015).
Thus, gaming is not delivered as a reward after students have engaged with the necessary
mathematical content, but players make meaningful progress interacting with the educational
content as they combine, compare, and strategically select the different calculations. Accordingly, it is
expected that the more practice players have with the mathematically relevant content of the NNG
the better their performance will be on the mathematical outcome measures.
3. Methods
3.1. Participants
Participants were 1,168 fourth to sixth grade primary school students (546 female) from four
urban and suburban areas in the southwest and middle of Finland. The mean age of the fourth graders
was 10.18, (SD = 0.42), Mage of fifth graders was 11.14, (SD = 0.38), and Mage of sixth graders was 12.20,
(SD = 0.45). Table 1 provides a description of the distribution of participants by grade in the
experimental and control groups.
Table 1
Number of Participants by Grade and Experimental Condition
Participation was voluntary; informed consents from parents and assent of the students were
obtained before data gathering. All students in the experimental classes played the NNG as part of
their mathematical teaching and completed the pre- and post-tests as part of their regular school
work. However, data was only gathered from students who had the consent to participate in the
study. Ethical guidelines of the University of Turku were followed. All teachers in both the
experimental and control classes were qualified primary school teachers with a masters’ degree.
3.2. Procedure
The study was a large-scale cluster randomized trial. Randomization to experimental and control
conditions was done on the classroom level because classrooms are usually considered as ecologically
valid units of measurement in experimental designs in the field of education (Hedges & Rhoads, 2010;
Winn, 2003). The two-level hierarchical design with covariates (pre-test) was used in the power
analysis and design planning (Hedges & Rhoads, 2010). The parameters used in power analysis were
effect size (d = .35), power (> .8) correlations within (Rw2 = 0.5), and among (Rs2 = 0.8) clusters and
intraclass correlations (ICC = 0.1). The correlation estimates were the ones presented by Hedges and
Rhoads (2010) and the intraclass correlation used in the calculation was based on the values
presented in a comprehensive review of differences between schools in Nordic countries by Yang
Hansen, Gustafsson and Rosén (2014). Based on the power analysis, the minimum number of
classrooms needed were 12 for experimental and 12 for control condition. In order to make sub-
group comparison possible and better control the variation in ways how teachers used the NNG in
their classrooms, a substantially larger sample was used in the present study. There were in total 61
teachers that volunteered to participate and were randomly assigned into experimental (31 classes)
or control groups (30 classes). Two control classes were not able to participate because they had to
move to a temporary school building due to renovations.
The intervention started at the beginning of the spring semester when all participants completed
the pre-test measures. After the pre-test, the experimental group participated in the intervention
over a ten week period in which they used the NNG as part of their regular math classes. The game
was distributed to the experimental group on individual pen-drives and was played on PCs. Gameplay
was expected to be integrated into everyday math teaching, thus the intervention group did not
receive more training compared to the controls. For ethical reasons and in order to keep volunteer
teachers’ motivation during the implementation, after the post-test, the conditions were switched,
and the control group played the NNG for the rest of the semester (5 weeks). There was no pseudo-
treatment in the control classes. Control classes were instructed to continue their regular textbook-
based teaching. In all schools, in line with the national core curriculum (Finnish National Agency for
Education 2014), the local curriculum emphasized the creative use of alternative arithmetic problem
solving strategies.
Before data gathering, all teachers from both conditions were invited to participate in an
information session regarding the main features of the NNG and the general outline and aim of the
study. All teachers received written instructions including a link to a video guide on how to use the
game and were offered e-mail support in case of questions or technical difficulties. Teachers only
received general guidelines regarding the gaming sessions, since (a) the general design of the NNG is
open and flexible and was aimed to be suitable for various grade levels and (b) the present study
aimed to create an ecologically valid intervention in the Finnish context. Teachers were free to choose
if they wanted their students to play the game in pairs or individually, and in case of pair play, it was
the teacher’s decision how to select the pairs. For all classes playing in pairs teachers sent back the list
of pairs when the game log data was gathered during the post-test
The guidelines for teachers in the experimental group were to aim for around ten hours playing
time, with at least three playing sessions a week where a session is no shorter than 30 minutes.
Implementation fidelity was checked using the game log data. Based on the log data the average time
on task (effective gameplay) in the 29 experimental classes was 4 hours and 10 minutes, ranging
between average 3 hours and average 5 hours 30 minutes. There were two classes with average time
on task less than one hour. Because the attempt to have an ecologically valid design, these
differences in time on task were interpreted as a natural variation when this type of method is
introduced in the regular education. Therefore, all classes were included in the comparison of the
experimental and control groups.
3.3. Measures
Paper-and-pencil measures of adaptive number knowledge, arithmetic fluency, and pre-algebra
knowledge were administered by thirteen different trained testers (university master’s students or
researchers). Before data gathering, all testers took part in a training regarding test administration.
The testers used an automatic slide show including standardized timer and sound signals. The three
tests, took 45 minutes to complete and were administered during both time points in the following
order: arithmetic fluency, adaptive number knowledge, and pre-algebra knowledge
3.3.1. Adaptive number knowledge
The Arithmetic Production Task was used in order to measure participants’ adaptive number
knowledge. The task is a timed, paper-and-pencil instrument which aims to capture students’ ability
to recognize and use different numerical characteristics and relations during their arithmetic problem
solving (McMullen et al., 2017). Students are presented with four to five numbers and the four basic
arithmetic operations. Using these numbers and operations, the aim is to produce as many arithmetic
In the experimental group, repeated measures ANOVAs showed no significant interaction effect of time and mode of
play (individually or in pairs) for any of the math learning outcomes. Therefore, mode of play was not included in the
reporting of the results.
sentences that equal a target number as they can in 90 seconds. To increase the reliability of the task,
three more items were added to the post-test. Table 2 shows the items for the two time points. At
both times, the same example item was used to demonstrate the task. After explaining the
instructions, the example item had to be solved in the same amount of time as a regular item, but
students were free to ask clarification questions before and after solving it.
Table 2
Items of the Arithmetic Product Task for Pre- and Post-tests
Students’ solutions were transcribed and then automatically scored using Microsoft Excel
Macros written for this purpose. The scoring criteria were developed based on the results of
previously conducted studies using similar tasks (McMullen et al., 2016, 2017). In the present study,
the scoring criteria examined the quantity and complexity of students’ solutions on the Arithmetic
Production Task. For quantity, the total number of mathematically correct solutions matching the
instructions was counted (Correct). For complexity, the total number of multi-operational (Multi-op.)
solutions was counted. A solution was multi-operational if two or more different arithmetic
operations were used in order to reach the target number (i.e., 2 * 4 + 8 = 16 multi-operational, but
12 + 2 + 2 = 16 not). Final scores were made up of the total average of correct solutions and total
average of multi-operational solutions. Cronbach’s α reliability value for correct solutions was .70 for
pre-test and .86 for post-test and for multi-operational solutions .63 for pre-test and .80 for post-test
3.3.2. Arithmetic fluency
Basic arithmetic fluency was measured by the Woodcock-Johnson Math Fluency sub-test (WJ
III® Test of Achievement), which consists of two pages with a total of 160 items. Students have to
complete as many arithmetic problems (simple addition, subtraction and multiplication) as possible
during three minutes (for more details see Schrank, McGrew, & Woodcock, 2001).
3.3.3. Pre-algebra knowledge
The task measuring pre-algebra knowledge consisted of short-answer and multiple choice
questions on equation solving (i.e., 12 + ___ = 11 + 15). The task consisted of six multiple-choice items
during pre-test and six multiple-choice plus six fill-in items during post-test. At both time points
students had eight minutes to solve all items. Each correct item was worth one point, so the
maximum pre-test score could be six, while the maximum post-test score could be twelve. Cronbach’s
α reliability value for the pre-algebra knowledge test was .73 for pre-test and .88 for post-test. Due to
the different number of items during the two time points, the standardized sum scores of the pre-
algebra knowledge task were used for analyses.
3.3.4. Game performance
The number of maps completed was selected as an indicator of game performance in the
present study. As the educational content and game mechanics are integrated in the NNG interaction
with the game means also interaction with the relevant mathematical learning content. Additionally,
progress in the game (unlocking new levels) is only possible if students complete a set of maps within
a certain performance range. Thus, it can be hypothesized that the number of maps completed can be
used as a reliable estimate of students’ practice with different number combinations and numerical
A map is the basic unit of progress in the NNG, there are 64 maps in total, with 4 target
materials needing to be picked up and returned within each map. Each map has a start and an end
and provides a substantial amount and variation of mathematically relevant game activity in-between
depending also of the active scoring modes within a map. The layout of each map is different, and
every target material is placed on different numbers. Together with the two scoring modes (moves
and energy) this context provides a wide range of unique arithmetic combinations throughout
gameplay and ample opportunities for players to explore number patterns and establish arithmetic
connections. As players progress in the NNG maps get progressively harder and there are more and
more maps in the energy mode.
During the intervention, each student or pair of students received the NNG on an individual pen
drive. All game action was saved and stored in time-stamped text files on these pen drives. The pen
drives were collected at the time of the post-test. After the log data was copied, the pen drives were
returned to the students as a token of reward for participation. The game log data was summarized
and analysed using Microsoft Excel Macros written for this purpose.
3.4. Analyses
As the study was conducted in a naturalistic classroom setting and the unit of randomization was
classroom, a variance component analyses was conducted to explore classroom effects and assess the
need for multilevel analyses. While it is generally accepted that small ICC values indicate no need for
multilevel analyses, there is no agreement about exact guidelines regarding specific cut-off values.
Recommendations for considering multilevel methods can range from 0.10 (e.g., Lee, 2000) to 0.25
(e.g., Bowen & Guo, 2011), but the decision is largely dependent on the specific context and study
design. For the present study, the ICC values at the classroom level using post-test scores were 0.11
for arithmetic fluency, 0.11 for the number of correct solutions, 0.09 for multi-operational solutions
and 0.10 for pre-algebra knowledge.
In order to account for the possible effects of the hierarchical study design, intervention effects
were tested using the linear mixed model (LMM) procedure of SPSS version 24 (Heck, Thomas, &
Tabata, 2010; West, 2009). For each dependent variable (correct solutions, multi-operational
solutions, math fluency, or pre-algebra knowledge), two models were run and compared using the
likelihood-ratio test (Field, 2009; West, 2009). The first of the two models included only fixed effects
to explore the interaction of treatment and time. The second model was a random intercept model
including the same fixed effects and additionally random classroom effects. Thus four separate fixed
effects and four separate random effects models were run using the dependent variable
measurement occasion and treatment as factors. For each separate dependent variable, level1
variables included observations from the repeated measure nested within the students (level2), who
in turn were nested within the classes (level3). The time variable that refers to the measurement
occasion (pre- to post-test) was included at level1. The treatment variable that indicates if students
received training was added on level2 as a fixed covariate and ‘time * treatment’ was added as a
cross-level interaction term. In the random models, in addition to this fixed model structure, on level3
random intercepts (classroom membership) were estimated. Covariance structures were:
‘unstructured’ for level1 and ‘variance components’ for level3.
Based on the comparison of the fixed and random effects models, results from the model with
the best fit are reported for all dependent variables. Within the whole sample, compared to the
model including fixed effects only, the random intercept model including the classroom effects had
the best fit (p < .001) for all measured math learning outcomes. Similarly, in grades five and six
compared to the fixed effects only model, the random intercept model had the best fit for most of the
math learning outcomes (p < .001). The two exceptions were in grade six where for math fluency the
random intercept model had the best fit using the .05 cutoff value (p = .04) and for the multi-
operational solutions the random effects model did not have a significantly better fit than the fixed
effects model (p = .11). In grade four, adding random classroom effects to the model did not
significantly affect the model fit.
To answer the second research question, hierarchical linear regression was used to explore the
relationship of game performance and mathematical learning outcomes. Post-test scores of correct
solution, multi-operational solution, arithmetic fluency and pre-algebra knowledge were used
separately as dependent variables. For each analysis grade level was entered as a predictor in the first
step, followed by the matching pre-test score of the mathematical outcome used as a dependent
variable in the second step, and finally by game performance entered in the third step.
As the second research question examined the relationship of game performance (amount of
gameplay) and mathematical learning outcomes, using game log data, only students with sufficient
amount of gameplay were included (cf. Bakker et al., 2015; van den Heuvel-Panhuizen et al., 2013).
Thus from the experimental group, only students with at least one map completed were used in the
regression analyses. Based on this criterion 25 cases were excluded from the analysis, out of a total of
642 participants in the experimental group. On average, students in this group completed 27.02 maps
(SD = 12.65) with a range of 1-88 maps completed. The total number of maps completed does not
mean unique maps completed; students could repeat the same in order to achieve a better score.
4. Results and discussion
The linear mixed model analyses showed a significant interaction effect between experimental
condition and time-point for the number of correct solutions F(1,1053) = 11.52, p <.001, the number
of multi-operation solutions F(1,1073) = 5.93, p =.02, and arithmetic fluency F(1,1028) = 5.57, p = .02,
but not for pre-algebra knowledge, F(1,1065) = 1.90, p = .17. Overall, training with the NNG seemed to
be more effective than traditional instructional methods in developing students’ adaptive number
knowledge and math fluency, but not their pre-algebra knowledge. However, students’ prior
mathematical knowledge in grades four, five and six is very different and this across grade-level
analysis may conceal the more substantial impact of the NNG training within the grade levels. Thus,
the main examination of the intervention effects was made using separate analysis for each of the
three different grade levels.
4.1. Intervention effects by grade level
Tables 3, 4 and 5 show the raw average scores and the results of the linear mixed model analyses
of the experimental and control groups during pre- and post-test for grades four, five, and six,
Table 3
Linear Mixed Model (Fixed Effects): Interaction Effect of Group and Time for Grade Four (n = 133)
Pre-test M (SD)
Post-test M (SD)
Correct solutions
2.08 (.78)
2.69 (.94)
2.29 (.98)
2.48 (1.23)
Multi-op. solutions
.51 (.43)
.81 (.59)
.63 (.46)
.82 (.59)
Arithmetic fluency
55.30 (16.61)
66.15 (14.77)
65.75 (18.27)
70.58 (18.43)
Pre-algebra knowledge
.54 (.32)
.28 (.22)
.66 (.30)
.44 (.30)
Note. Exp. = Experimental group; Cont. = Control group; Multi-op. = Multi-operational
Table 4
Linear Mixed Model (Fixed and Random Effects): Interaction Effect of Group and Time for Grade Five
(n = 599)
Pre-test M (SD)
Post-test M (SD)
Correct solutions
2.60 (1.04)
2.94 (1.20)
2.83 (1.22)
2.91 (1.25)
Multi-op. solutions
.77 (.61)
1.15 (.82)
.86 (.72)
1.11 (.83)
Arithmetic fluency
69.24 (18.38)
77.43 (19.76)
68.14 (17.00)
75.97 (18.57)
Pre-algebra knowledge
.72 (.30)
.53 (.30)
.71 (.30)
.53 (.31)
Note. Exp. = Experimental group; Cont. = Control group; Multi-op. = Multi-operational
Table 5
Linear Mixed Model (Fixed and Random Effects): Interaction Effect of Group and Time for Grade Six (n
= 423)
Pre-test M (SD)
Post-test M (SD)
Correct solutions
3.03 (1.22)
3.33 (1.36)
2.90 (1.11)
3.11 (1.24)
Multi-op. solutions
.94 (.67)
1.37 (.82)
.92 (.65)
1.36 (.80)
Arithmetic fluency
75.66 (16.77)
84.94 (19.11)
74.91 (16.43)
82.43 (20.23)
Pre-algebra knowledge
.77 (.29)
.67 (.29)
.81 (.27)
.62 (.32)
Note. Exp. = Experimental group; Cont. = Control group; Multi-op. = Multi-operational.
As results show, practice with the NNG had varying effects on different arithmetic skills in
different grade levels. In grade four, results showed a significant interaction of time and group for
correct solutions and arithmetic fluency, but no interaction effects were found for multi-operational
solutions or pre-algebra knowledge. In grade five, results show significant interaction effects of time
and group for correct solutions and multi-operational solutions, but no significant interaction effects
for math fluency and pre-algebra knowledge. In grade six, results show a significant interaction effect
for pre-algebra knowledge.
These results suggest that the NNG was able to support the development of different aspects of
arithmetic and mathematical development at different ages. In grade four, the game supported the
development of basic calculation fluency and the more basic aspect of adaptive number knowledge
(finding correct solutions). In grade five, where calculation fluency is already more established, there
was a positive effect of gameplay on both correct solutions and also in the more complex aspect of
adaptive number knowledge (finding multi-operational solutions). Effects for pre-algebra knowledge
were only found among sixth graders who otherwise did not benefit much from the game, the
settings of which might have been not challenging enough for them. Alternatively, it could be that
pre-algebra knowledge is a more complex skill that develops on top of other abilities, and practice
with the NNG is only beneficial for pre-algebra skills if students already have a relatively high level of
prior knowledge in arithmetic. Finding pre-algebra knowledge intervention effects only in grade six is
similar to the results of van den Heuvel-Panhuizen and colleagues (2013) who found the strongest
intervention effects in grade six when compared to fourth and fifth graders as a result of a game-
based training which aimed to train quantitative reasoning that is foundational to early algebra
In order to progress in the NNG, players need to mentally execute and compare many
calculations, look for strategically important numbers (i.e., numbers with many divisors, numbers
close to the target number), and continuously refine their strategies according to the different game
modes and challenges. This requires noticing and using numerical characteristics and relations in
order to arrive at efficient problem solving strategies in mental calculations (Brezovszky et al., 2015;
McMullen et al., 2017; Threlfall, 2009). This practice is in line with the idea that in order to develop
more flexibility and adaptivity with arithmetic problem solving, students need to have a chance to
practice with various number-operation combinations (Baroody, 2003; Verschaffel et al., 2009). It
seems that the playful practice in the NNG indeed enhances students’ awareness of numerical
relations, so they can notice more of these relations (i.e., correct solutions) and by having a richer
repertoire of number relations come up with more complex solutions (i.e., multi-operational
solutions). With more attention on calibrating gameplay for particular grades, it may be possible to
elicit even more pronounced effects. For example, offering younger students opportunities to
strengthen their understanding of the natural number base-ten system, while stimulating older
students to explore their understanding of arithmetic concepts such as the inverse nature of
multiplication and division more deeply.
The lack of methodologically sound interventions and large-scale randomized control studies is a
common finding of many reviews and meta-analyses in game-based learning (Hainey, Connolly, Boyle,
Wilson, & Razak, 2016; Wouters et al., 2013; Young et al., 2012). This is problematic because it can
distort the picture regarding the effectiveness of game-based learning as smaller and more controlled
quasi-experimental designs might inflate intervention effects, while effect sizes are generally very
small for large-scale randomized control trials (Cheung & Slavin, 2013; Wouters et al., 2013). In spite
of the possible small effects and loss of control in implementation, conducting large-scale randomized
interventions is important as they provide an opportunity to gain more ecological validity and realistic
estimate regarding the efficiency of game-based learning environments in the classroom setting
(Winn, 2003; Cook, & Payne, 2001).
4.2. Effects of game performance
The second research question of the present study explored the relationship of game
performance in the NNG and the development of students’ mathematical learning outcomes.
Hierarchical linear regression analyses were conducted in order to investigate the impact of game
performance on the improvement of the experimental group’s mathematical learning outcomes. As
Table 7 shows, after taking into account grade level and pre-test scores, the number of maps
completed still explains part of the variance for all outcome measures. Since the linear regression
cannot take into account the nested data structure, to confirm the significance level of beta values
linear mixed model analyses including pre-test scores, grade level and maps completed as fixed
effects and random classroom effects was performed. Results showed a significant main effect for the
number of maps completed p < .001.
Table 7
Hierarchical Linear Regression Analyses: Impact of Game Performance on the Mathematical Learning
Outcomes in the Experimental Group (n = 617)
Math Fluency
Total R2
Note 1. * p < .05, ** p < .01, *** p < .001. Note 2. Pre-test = Corresponding pre-test variable to the
post-test variable (correct solutions, multi-operational solutions, math fluency or pre-algebra). Note 3.
Multi-op. = Multi-operational
Results show that students who had more practice with the NNG also benefited more from the
gameplay. This suggests that the type of action necessary in order to make meaningful progress in the
NNG could be transferred outside the game environment as well. These results are in line with a
previous small-scale intervention study using the NNG (Brezovszky et al., 2015).
One issue with many game-based learning environments is that players engage with content or
features that are irrelevant from the perspective of the learning aims (Clark et al., 2011; Wouters &
Oostendorp, 2013). Even if students use the environment as intended, connecting the in-game
learning material with educational content outside of the game-based learning environment is often
problematic (Clark et al., 2011; Lajoie, 2005). In the NNG, at least eight paths from a starting point to a
target were taken within a completed map. But, considering that not all numbers are available and
that players need to adapt their strategies according to the active game mode (moves or energy), by
the time a map was completed a player most likely undertook a large number of mental calculations
(Brezovszky et al., 2013). In light of the results, it seems that extended practice with the NNG was able
to develop students’ recognition and use of numerical characteristics and relations.
4.3. Limitations
A few major limitations of the study are connected to the design decision which was to conduct an
ecologically valid large-scale intervention study. First, one of the major issues with this design is the
loss of control over the implementation details, which in the present study were only accounted for
by analyzing the game log data. It was an important aim of the present study to design an
intervention that is as close to everyday classroom practices as possible, especially given the large
autonomy Finnish teachers have. Thus, guidelines regarding the implementation of the NNG did not
specify strict details and teachers were free to decide, for example, the mode of play. This freedom
could result in a substantial amount of variation and can make the interpretation of the results
difficult. However, as students’ average time on task did not show a large variation across the
classrooms, it can be assumed that most students benefited from the training to a similar extent.
Connected to methodological decisions, it is important to mention that randomization on a
classroom level might have affected results. However, a classroom is a natural unit of analysis in
educational research and randomization by individual students is rarely done as it is almost
impossible for this type of study designs. Multilevel statistical analyses were conducted that takes into
account the nested nature of the data. Additionally, ICC scores were low and the amount of
participating classes was much higher than the necessary number suggested by the power analyses
which strengthens the generalizability of results despite the cluster randomized design.
Using regular instruction and no alternative intervention method (game-based or traditional) in
the control group affects the interpretation of the results in the present study. While the time period
for math instruction was the same for both groups (i.e. NNG play replaced a part of normal
mathematics lessons), the novelty of playing the NNG may have had an effect on student
performance on the post-tests. However, in a separate study of situational interest using the NNG,
results showed substantial variation in situational interest between students and across sessions
which was mainly explained by prior personal mathematics interest (Rodríguez-Aflecht et al., 2018).
This suggests that the novelty effect cannot explain group differences in the mathematical learning
outcomes of the current study. With regards to the type and quality of the regular teaching practice it
is important to add here that developing flexibility and adaptivity with arithmetic is an aim of the
Finnish National Core Curriculum and teachers are encouraged to use various representations, games,
discovery learning, and discussion, as well as plenty of group work in their everyday math classroom
practice (Kupari, 2008). For more comprehensive conclusions, future studies should compare
gameplay with the NNG with other game-based learning environments and more elaborated training
methods of flexibility with arithmetic problem solving without digital games.
A number of issues with the instruments used in the present study should also be addressed in
future studies. First, the relatively low pre-test reliability of the Arithmetic Production Task raises
concerns. These low values could be explained by the low number of items used during pre-test
and/or the novelty of the task type. Increasing the number of items in future studies, as was done on
the post-test in the present study, is recommended. With regards to the measure of game
performance, it has to be acknowledged that the number of maps completed is a crude indicator and
there might be more subtle differential patterns underlying this measure (i.e., performance within a
map, replay trials of the same map, more qualitative aspects of the problem solving strategies used
etc.). As the NNG logs different aspects of the game data, future studies could explore this question in
more detail.
5. Conclusions
NNG is, to our knowledge, the first game-based learning environment directly focused on
enhancing adaptive arithmetic knowledge and skills, which have been difficult to support in
traditional classrooms teaching. This suggests that the NNG could be a flexible tool to develop
complex mathematical skills and knowledge in a naturalistic classroom setting. Adaptive number
knowledge has been suggested to underlie proficiency with whole-number arithmetic problem solving
strategies (McMullen et al., 2017), a core feature of arithmetic development (Nunes et al., 2016;
Verschaffel et al., 2009). By promoting students’ adaptive number knowledge, the NNG is a valuable
pedagogical tool for supporting students’ development of flexible and adaptive arithmetic problem
solving. This type of support may have long-lasting value in mathematical development, for example
with learning algebra. Future studies that focus on integrating gameplay and more typical classroom
activities (Clark, Tanner-Smith, & Killingsworth, 2016; Wouters & Oostendorp, 2013) may improve
both the embedding of the NNG in the curriculum and its positive outcomes for students.
Arnab, S., Lim, T., Carvalho, M. B., Bellotti, F., de Freitas, S., Louchart, S., … De Gloria, A. (2015). Mapping learning and
game mechanics for serious games analysis. British Journal of Educational Technology, 46(2), 391411.
Bakker, M., van den Heuvel-Panhuizen, M., & Robitzsch, A. (2015). Effects of playing mathematics computer games on
primary school students’ multiplicative reasoning ability. Contemporary Educational Psychology, 40, 5571.
Baroody, A. J. (2003). The development of adaptive expertise and flexibility: The integration of conceptual and procedural
knowledge. In The Development of Arithmetic Concepts and Skills: Constructive Adaptive Expertise (pp. 133).
Lawrence Erlbaum Associates Publishers.
Blöte, A. W., Klein, A. S., & Beishuizen, M. (2000). Mental computation and conceptual understanding. Learning and
Instruction, 10(3), 221247.
Blöte, A. W., Van der Burg, E., & Klein, A. S. (2001). Students’ flexibility in solving two-digit addition and subtraction
problems: Instruction effects. Journal of Educational Psychology, 93(3), 627638.
Bowen, N. K., & Guo, S. (2011). Structural equation modeling. Oxford University
Brezovszky, B., Lehtinen, E., McMullen, J., Rodriguez, G., & Veermans, K. (2013). Training flexible and adaptive arithmetic
problem solving skills through
Brezovszky, B., Lehtinen, E., McMullen, J., Rodriguez, G., & Veermans, K. (2013). Training flexible and adaptive arithmetic
problem solving skills through exploration with numbers: The development of NumberNavigation game. In 7th
European Conference on Games Based Learning, ECGBL 2013 (Vol. 2, pp. 626634).
Brezovszky, B., Rodríguez-Aflecht, G., McMullen, J., Veermans, K., Pongsakdi, N., Hannula-Sormunen, M. M., & Lehtinen, E.
(2015). Developing adaptive number knowledge with the number navigation game-based learning environment.
In J. Torbeyns, E. Lehtinen, & J. Elen (Eds.), Describing and Studying Domain-Specific Serious Games (pp. 155
170). Cham: Springer International Publishing.
Cook, T. D., & Payne, M. R. (2001). Objecting to the objections to using random assignment in educational research. In F.
Mosteller, & R. F. Boruch (Eds.), Evidence matters (pp. 150-177). Washington, DC: Brookings Institution Press.
Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing
mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88113.
Clark, D. B., Nelson, B. C., Chang, H. Y., Martinez-Garza, M., Slack, K., & D’Angelo, C. M. (2011). Exploring Newtonian
mechanics in a conceptually-integrated digital game: Comparison of learning and affective outcomes for students
in Taiwan and the United States. Computers and Education, 57(3), 21782195.
Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital Games, Design, and Learning: A Systematic Review
and Meta-Analysis. Review of Educational Research, 86(1), 79122.
Devlin, K. (2011). Mathematics education for a new era: Video games as a medium for learning. A K Peters.
Dowker, A. (1992). Estimation Strategies of Computational. Journal for Research in Mathematics Education, 23(1), 4555.
Field, A. (2009). Discovering statistics using SPSS. Sage publications.
Finnish National Agency for Education (2014). Core curriculum for basic Education 2014.
Habgood, M. P. J., & Ainsworth, S. E. (2011). Motivating Children to Learn Effectively: Exploring the Value of Intrinsic
Integration in Educational Games. Journal of the Learning Sciences, 20(2), 169206.
Hainey, T., Connolly, T. M., Boyle, E. A., Wilson, A., & Razak, A. (2016). A systematic literature review of games-based
learning empirical evidence in primary education. Computers & Education, 102, 202223.
Heck, R. H., Thomas, S. L. T., & Tabata, L. N. (2010). Multilevel and Longitudinal Modeling with IBM SPSS. Routledge. New
Hedges, L. V., & Rhoads, C. (2010). Statistical power analysis. In International Encyclopedia of Education (pp. 436443).
Heinze, A., Star, J. R., & Verschaffel, L. (2009). Flexible and adaptive use of strategies and representations in mathematics
education. ZDM - International Journal on Mathematics Education, 41, 535540.
Heirdsfield, A. M., & Cooper, T. J. (2004). Factors affecting the process of proficient mental addition and subtraction: Case
studies of flexible and inflexible computers. Journal of Mathematical Behavior, 23(4), 443463.
Heirdsfield, A. M (2011). Teaching mental computation strategies in early mathematics. Young Children, 66(2), pp. 96-102.
Hickendorff, M. (2017). Dutch sixth graders’ use of shortcut strategies in solving multidigit arithmetic problems. European
Journal of Psychology of Education, 118.
Ke, F. (2008). A case study of computer gaming for math: Engaged learning from gameplay? Computers and Education,
51(4), 16091620.
Ke, F. (2009). A Qualitative Meta-Analysis of Computer Games as Learning Tools. Handbook of Research on Effective
Electronic Gaming in Education, 132.
Kiili, K. (2007). Foundation for problem-based gaming. British Journal of Educational Technology, 38(3), 394404.
Kupari, P. (2008). Mathematics education in Finnish comprehensive school: characteristics contributing to student success.
In Proceedings of the XI International Congress in Mathematics Education.
Lajoie, S. P. (2005). Extending the scaffolding metaphor. Instructional Science, 33(56), 541557.
Laski, E. V., Ermakova, A., & Vasilyeva, M. (2014). Early use of decomposition for addition and its relation to base-10
knowledge. Journal of Applied Developmental Psychology, 35(5), 444454.
Lee, V. E. (2000). Using Hierarchical Linear Modeling to Study Social Contexts: The Case of School Effects, Educational
Psychologist, 35(2), 125-141, doi: 10.1207/s15326985ep3502_6
Lehtinen, E., Brezovszky, B., Rodríguez-Aflecht, G., Lehtinen, H., Hannula-Sormunen, M. M., McMullen, J., … Jaakkola, T.
(2015). Number Navigation Game (NNG): Design Principles and Game Description. In J. Torbeyns, E. Lehtinen, & J.
Elen (Eds.), Describing and Studying Domain-Specific Serious Games (pp. 4561). Cham: Springer International
Lehtinen, E., Hannula-Sormunen, M., McMullen, J., & Gruber, H. (2017). Cultivating mathematical skills: from drill-and-
practice to deliberate practice. ZDM - Mathematics Education, 49(4), 625636.
Li, Q., & Ma, X. (2010). A Meta-analysis of the Effects of Computer Technology on School Students’ Mathematics Learning.
Educational Psychology Review, 22(3), 215243.
Lowrie, T., & Jorgensen, R. (2015). Digital Games and Learning: What’s New Is Already Old? (pp. 1–9). Springer, Dordrecht.
McMullen, J., Brezovszky, B., Rodríguez-Aflecht, G., Pongsakdi, N., Hannula-Sormunen, M. M., & Lehtinen, E. (2016).
Adaptive number knowledge: Exploring the foundations of adaptivity with whole-number arithmetic. Learning
and Individual Differences, 47, 172181.
McMullen, J., Brezovszky, B., Hannula-Sormunen, M. M., Veermans, K., Rodríguez-Aflecht, G., Pongsakdi, N., & Lehtinen, E.
(2017). Adaptive number knowledge and its relation to arithmetic and pre-algebra knowledge. Learning and
Instruction, 49, 178187.
Mulligan, J. T., & Mitchelmore, M. C. (2013). Early awareness of mathematical pattern and structure. In L. D. English & J. T.
Mulligan (Eds.), Reconceptualising early mathematics learning (pp. 29-45). New York: Springer.
National Council of Teachers of Mathematics. (2014). Principles to actions: Ensuring mathematical success for all. Reston.
Nunes, T., Dorneles, B. V., Lin, P.-J., & Rathgeb-Schnierer, E. (2016). Teaching and Learning About Whole Numbers in
Primary School. Cham: Springer International Publishing.
Pope, H., & Mangram, C. (2015). Wuzzit Trouble: The Influence of a Digital Math Game on Student Number Sense.
International Journal of Serious Games, 2(4).
Rittle-Johnson, B., & Star, J. R. (2009). Compared With What? The Effects of Different Comparisons on Conceptual
Knowledge and Procedural Flexibility for Equation Solving. Journal of Educational Psychology, 101(3), 529544.
Rodríguez-Aflecht, G., Jaakkola, T., Pongsakdi, N., Hannula-Sormunen, M., Brezovszky, B., & Lehtinen, E. (2018). The
development of situational interest during a digital mathematics game. Journal of Computer Assisted Learning.
Rutherford, T., Farkas, G., Duncan, G., Burchinal, M., Kibrick, M., Graham, J., … Martinez, M. E. (2014). A Randomized Trial
of an Elementary School Mathematics Software Intervention: Spatial-Temporal Math. Journal of Research on
Educational Effectiveness, 7(4), 358383.
Salen, K., & Zimmerman, E. (2003). Rules of Play: Game Design Fundamentals. The MIT Press.
Schneider, M., Rittle-Johnson, B., & Star, J. R. (2011). Relations among conceptual knowledge, procedural knowledge, and
procedural flexibility in two samples differing in prior knowledge. Developmental Psychology, 47(6), 15251538.
Schrank, F. A., McGrew, K. S., & Woodcock, R. W. (2001). Technical Abstract (Woodcock-Johnson III Assessment Service
Bulletin No. 2). Itasca, IL: Riverside Publishing. Retrieved from
Star, J. R., & Seifert, C. (2006). The development of flexibility in equation solving. Contemporary Educational Psychology,
31(3), 280300.
Star, J. R., & Rittle-Johnson, B. (2008). Flexibility in problem solving: The case of equation solving. Learning and Instruction,
18(6), 565579.
Star, J. R., & Newton, K. J. (2009). The nature and development of experts’ strategy flexibility for solving equations. ZDM -
International Journal on Mathematics Education, 41(5), 557567.
Squire, K. (2008). Open-Ended Video Games: A Model for Developing Learning for the Interactive Age. In Salen K., The
ecology of games: Connecting youth, games, and learning (p. 167-198). Cambridge MA: The MIT Press.
Threlfall, J. (2002). Flexible Mental Calculation. Educational Studies in Mathematics, 50(1), 2947.
Threlfall, J. (2009). Strategies and flexibility in mental calculation. ZDM, 41(5), 541555.
van den Heuvel-Panhuizen, M., Kolovou, A., & Robitzsch, A. (2013). Primary school students’ strategies in early algebra
problem solving supported by an online game. Educational Studies in Mathematics, 84(3), 281307.
Verschaffel, L., Luwel, K., Torbeyns, J., & Van Dooren, W. (2009). Conceptualizing, investigating, and enhancing adaptive
expertise in elementary mathematics education. European Journal of Psychology of Education, 24(3), 335359.
West, B. T. (2009). Analysing longitudinal data with the linear mixed models procedure in SPSS. Eval Health Prof, 32(3),
Winn, W. (2003). Research Methods and Types of Evidence for Research in Educational Technology. Educational
Psychology Review, 15(4), 367-373.
Wouters, P., & Oostendorp, H. Van. (2013). Computers & Education A meta-analytic review of the role of instructional
support in game-based learning. Computers & Education, 60(1), 412425.
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and
motivational effects of serious games. Journal of Educational Psychology, 105(2), 249265.
Yang Hansen, K., Gustafsson, J.-E. , & Rosén, M. (2014). School performance difference and policy variations in Finland,
Norway and Sweden. In K. Yang Hansen, J.-E. Gustafsson, M. Rosén, S. Sulkunen, K. Nissinen, P. Kupari, R.F.
Ólafsson, J.K. Björnsson, L.S. Grønmo, L. Rønberg, J. Mejding, I.C. Borge, & A. Hole. Northern Lights on TIMMS and
PIRLS 2011. TemaNord 2014:528, 25-48.
Young, M. F., Slota, S., Cutter, B., R., Jalette, G., Mullin, G., … Yukhymenko, M. (2012). Our Princess Is in Another Castle: A
Review of Trends in Serious Gaming for Education. Review of Educational Research, 82(1), 6189.
... The game aids the student to become an independent learner, it stimulates intuition and allows for all student levels to participate in the lesson (Hwa, 2018). Games provide teachers with a simple way to adapt the teaching to students at different learning levels by having the students develop different calculation strategies while playing (Brezovszky et al., 2019;Deng et al., 2020). For example, Brezovszky et al. (2019) assessed the effects of a digital game-based learning environment (Number Navigation Game [NNG]), in advancing primary school students' arithmetic skills by enriching regular mathematics teaching with gameplay. ...
... Games provide teachers with a simple way to adapt the teaching to students at different learning levels by having the students develop different calculation strategies while playing (Brezovszky et al., 2019;Deng et al., 2020). For example, Brezovszky et al. (2019) assessed the effects of a digital game-based learning environment (Number Navigation Game [NNG]), in advancing primary school students' arithmetic skills by enriching regular mathematics teaching with gameplay. The results showed the positive effect of the gameplay on students' different types of arithmetic skills and knowledge and provided teachers with a flexible and useful tool to extend their classroom practice. ...
... These should be espoused with game elements that prompt students' sense of playfulness, thereby increasing their motivation to engage in the learning activity. Consequently, these suggested activities could lead to several learning-related outcomes, such as cognitive, affective, and behavioral learning outcomes (Nurtanto et al., 2021;Sailer & Homner, 2020); improvement in knowledge-based learning outcomes (Papp & Theresa, 2017); mathematical knowledge (Hwa, 2018); learning in formal educational settings (Huang et al., 2020); interest in math classes (Stoyanova et al., 2017); and calculation or arithmetic skills (Brezovszky et al., 2019;Deng et al., 2020). Collectively, the researchers underscored the importance of ensuring that the design of the gamified learning activities is closely connected to learning outcomes. ...
... Students are generally more enthusiastic about participating in the learning process that involves games. Students' success be improved and learning made easier through the use of games (Bottino et al., 2007;Brezovszky et al., 2019;Lee et al., 2022). If this engaging activity is carried out in a setting that takes into account the characteristics of the students, it may assist them in achieving success in subjects like physics where students are more likely to be intimidated and suspicious of their complexity. ...
... In these studies, there are many treatments to increase students' creativity. These treatments include reading and writing in cooperative learning classes (Marcos et al., 2020), playful design jams (Tang et al., 2020), English language course design for junior high school students (Bağ & Gürsoy, 2021), through mathematics and music (Azaryahu et al., 2023), STEM (Yalçın & Erden, 2021;Eroğlu & Bektaş, 2022), mathematical games based on the learning environment (Brezovszky et al., 2019), entrepreneurial skills (Durnali et al., 2023), 5-I training program (Gu et al., 2019). Figure 1 shows the percentage of the pre-test, post-test, and N-gain scores of 26 tenth-grade MAN 2 Semarang students. ...
This study aims to investigate how PjBL via traditional games impact students’ critical thinking, creative thinking, and collaborative skills. The research sample is a local school in Semarang. The research design was a quantitative study using the pre-experimental design type one group pretest-posttest method. There is only one predetermined group in this design. The theme of the game that the project will create was revealed following the pre-test. The research results show students’ creative thinking skills in applying the PjBL model via game techniques are 61.53% in high criteria and 38.46% in medium criteria. The study results also show a fair increase in the N-Gain value for the four indicators of creative thinking skills: improvisation, elaboration, creativity, vision, effectiveness, and efficiency. The results show it is effective for six students (25%), fairly effective for 13 students (54.16%), and less effective for five students (20.83%). The study’s results also show that the Mann-Whitney U statistical test is 69. It means that the PjBL model impacts students’ collaborative skills. This research has a big impact on increasing students’ creative thinking skills and collaboration.
... Quality human resources require the role of educational institutes that can provide full service. In terms of forming a quality educational institute as expected by the public, the community is not only consistently playing a role, but it is the responsibility of all, including involving parties from the educational environment (Brezovszky et al., 2019). ...
Full-text available
This study aims to describe the management of excellent and regular classes at SD Hang Tua 10 Juanda. This research is field research conducted using qualitative methods with a case study research design. Data collection techniques include observation, interviews, and documentation. Data analysis involves data condensation, data presentation, and verification of conclusions. This study tested credibility, transferability, dependability, and confirmability to ensure data validity. The research findings reveal the implementation of the excellent class and regular class programs, covering aspects such as planning (objectives, management, curriculum, costs), implementation (administrators, educators, students, materials, scheduling, facilities, costs), evaluation, and comparing the results with existing standards. This evaluation is usually conducted during mid-term or end-of-term assessments, where reports from each excellent class or regular class teacher are submitted to the headmaster. The evaluation examines whether the objectives of the excellent class program, as measured by student learning through exams or assessments, have been achieved. If not, improvements will be made. However, if the learning process and outcomes align with the established standards, enrichment activities will be implemented to maintain or enhance student learning achievements.
... Ricerche precedenti hanno dimostrato l'efficacia di questo approccio nel migliorare i risultati di apprendimento degli studenti (Brezovszky, 2019;Wardoyo et al., 2020;Yang et al., 2022) ed è per questo sempre più adottato in setting didattici innovativi (Lorenzo-Alvarez et al., 2020). ...
Full-text available
L’emergenza sanitaria ha costituito un momento di crisi per l’istruzione, soprattutto nei Paesi in cui le scuole non avevano un’esperienza consolidata di formazione a distanza, tra cui l’Italia. La didattica laboratoriale ha sofferto il distanziamento, ma ha anche (ri)scoperto nuove modalità. Presentiamo un percorso di didattica immersiva condotto prima e durante la pandemia, attraverso il feedback degli studenti che hanno sperimentato l’uso di Minecraft in presenza e a distanza. Al campione (2264) è stato somministrato un questionario volto a indagare vantaggi e limiti dell’uso didattico del gioco e le differenze in termini di gradimento, autoefficacia percepita dagli studenti, collaborazione fra pari, ordine di scuola e genere dei partecipanti. I risultati sono analizzati e discussi.
... In this direction, the importance of creating teaching environments in a way that will highlight these behaviors that are expected from students has been emphasized in the national education program (MoNE, 2018). One of the preferred methods in mathematics education to highlight these skills is the game-based teaching method (Brezovszky et al., 2019). Game-based teaching is defined as the realization of learning through the game method within the teaching process (Demir and Bilgin, 2021). ...
Bu çalışmada akıl yürütme oyunları vasıtası ile işlenen doğrusal denklem konularının öğrenciler üzerindeki etkileri ve öğrencilerin uygulama hakkındaki görüşleri araştırılmıştır. Yapılan çalışma ile öğrencilerin akıl yürütme ve işlem oyunlarını çözme sürecinde oyun içerisinde yer alan doğrusal denklem sorularını çözmeleri beklenmiştir. Bu bağlamda yürütülen uygulama ile öğrencilerin doğrusal denklem konularındaki öğrenme seviyelerinin artırılması hedeflenmiştir. Nitel yöntemin tercih edildiği bu çalışmada, durum çalışması metotlarından bütüncül durum deseni benimsenmiştir. Çalışmaya Ankara ilinde eğitim görmekte olan 34 öğrenci katılmış olup, 8 öğrenci ile de yarı yapılandırılmış görüşmeler gerçekleştirilmiştir. Akıl yürütme ve işlem oyunlarına dayalı etkinlikler, uygulama sırasında çekilen videolar, öğrencilerin çalışma kâğıtları, araştırmacının gözlemleri ve yarı yapılandırılmış görüşmelerle veriler toplanmıştır. Çalışmadan elde edilen veriler betimsel ve içerik analiz yoluyla incelenmiştir. Analiz sonucunda, oyun tabanlı öğrenme yöntemi ve akıl yürütme oyunlarının öğrencilerin doğrusal denklemler konularındaki öğrenme sevilerini olumlu etkilediği görülmüştür. Öğrencilerin akıl yürütme oyunları ile öğretime yönelik pozitif fikirler belirttiği gözlemlenmiştir. Öğrencilerin oyunları sevip etkili buldukları tespit edilmiştir.
Full-text available
Adaptive expertise is a greatly appreciated, yet rarely achieved, goal of Mathematics Curricula because it is considered to typify high-level Mathematical thinking. Adaptive expertise demonstrates knowledge and skills that can be dynamically implemented in uncommon situations, not just within highly defined tasks or sufficiently prepared contexts. To achieve Adaptive expertise, students must be given occasions to practice Solving open-ended Mathematical tasks in unfamiliar circumstances, allowing them to contemplate, analyze, and explore different connections and alternative solutions to develop their Emerging skills and knowledge structures. Traditional math classrooms are often equipped with textbooks and instructional approaches that focus on isolated, routine exercises, or drill-andpractice, which encourage students to master isolated procedural techniques to find the most or only efficient solution. Math teachers, therefore, employ teaching methods that emphasize speed and accuracy using these materials. The idea of Mathematics as a "fixed" subject, which is full of rigid and absolute rules, unintentionally continues to be reinforced. This doctoral dissertation aims to investigate design principles for learning environments that support flexible Mathematical thinking in Mathematics education. This thesis focuses on two objectives: first, it aspires to understand how Adaptive expertise can be promoted with deliberate practice, and whether it can be done by using a Mathematical game-based learning environment called the Number Navigation Game (NNG). The nature of deliberate practice is demanding and occurs just beyond one's abilities. It necessitates deep engagement, continuous efforts to enhance performance, and a positive attitude towards challenges—traits synonymous with a growth mindset. Given the association between a growth Mindset and persistent learning behavior, the second objective explores ways to cultivate a growth Mindset in Mathematics classrooms. This dissertation is divided into two parts, encompassing three sub-studies. Part one, comprising Studies I and II, focuses on the Number Navigation Game (NNG). Study I explores game experiences during the NNG development process and examines how different design choices influence students' Gaming experiences. The results provide Insights into the iterative design process of a research-based serious game, shedding light on students' interactions with both learning and Gaming components and their relation to novel Mathematical learning objectives. Study II delves into various game performance profiles using Gaming analytics and investigates the diverse ways students Engage with the NNG. Utilizing log data from game performances in the energy mode, combined with measured Mathematics learning outcomes, math interest, perceived challenge, and experienced flow during gameplay, Study II offers evidence on promoting Adaptive expertise through deliberate practice, game-based learning environments, and learning outcomes. In essence, Studies I and II highlight how the NNG serves as a supportive platform for presenting students with novel contexts, challenging tasks, and immediate feedback, making it a viable tool for traditional classrooms. Part two (Study III) investigates the current state of growth Mindset interventions in Mathematics education through a systematic review. The results show that when implicit theories of Intelligence interventions were conducted specifically in the math domain, positive results were reported, whereas general implicit theories of Intelligence interventions yielded mixed results. This indicates that to make the necessary behavioral changes based on changed beliefs, participants need to engage with mathematical content at a deeper level than the surface level. Most importantly, the learning environment must be embedded with elements that support struggle and mistakes, encourage effortful practices, and make progress visible to students. In this way, KEYWORDS: Adaptive expertise, game-based learning environment, growth mindset, deliberate practice, flexible Mathematical thinking
Full-text available
En el presente artículo, se describen los hallazgos de la investigación titulada "Disposición emocional hacia la geometría en estudiantes de grado 11°: análisis de una experiencia de aprendizaje gamificada en tres instituciones educativas en el municipio de Chigorodó", llevada a cabo en el marco del doctorado en educación del autor. El estudio evaluó el impacto de la gamificación en el aprendizaje de la geometría en estudiantes de tres instituciones educativas, encontrando un nivel promedio de desempeño en geometría con fortalezas en algunas áreas, pero también identificando aspectos a mejorar; además, se observó ansiedad relacionada con la geometría, la cual se atribuyó a dificultades conceptuales y presión por obtener buenas calificaciones. Se desarrollaron tres actividades gamificadas que resultaron efectivas para mejorar el aprendizaje y se brindaron recomendaciones para reducir la ansiedad y mejorar el rendimiento académico en el estudio de esta disciplina
Designing a digital game using method, activities, contexts and problems which have been proved to be effective in mathematics education for a long time, will increase the effectiveness and success of the game. The purpose of this study was to reflect the digital game design process for the area concept. The design-based research method was used in the study. The digital game was developed by the researchers in a circular design process including the design, analysis, evaluation, and revision. In the design process, preliminary concepts, skills, and the way to teach them were determined with the help of the literature on the concept of area aimed to be taught via the digital game, and then the mechanics and context of the game were designed under the supervision of three instructors of mathematics. After the design stage, fifteen students were asked to play the game in order to evaluate it. The findings showed that it is important to refer to the literature and to involve mathematics instructors in the process of creating a digital game by considering how such a game should be in terms of mathematics. Getting help from students with different socio-economic levels during the game design phase will help the development of the educational and game structure of the game.
Full-text available
This research aims to combine the pursuit of literacy cultivation with a focused investigation into the evidence of game-based teaching (GBT). To achieve this, the study employs a mixed-methods approach including the interview method, Delphi method, and network hierarchical analysis (ANP) to analysis Expert opinions and construct a comprehensive GBT evaluation index system. The results indicated that a comprehensive GBT evaluation index system is comprised of five primary indicators: teaching objectives, game-based teaching methods, teaching content, game-based teaching processes, and game-based teaching characteristics. Additionally, there are 19 secondary indicators, such as objective content, game presentation, context construction, and flow experience. This study expects to effectively capture the unique attributes of game-based classes and to assist teachers in improving the design of GBT activities in practical applications.
Full-text available
The present study focused on 212 fifth graders' situational interest trajectories during an intervention with a digital mathematics game called Number Navigation. Our aims were to explore the development of situational interest whilst playing the game and to investigate the relationship between situational interest and individual math interest. Growth mixture model analyses showed that in the whole sample situational interest was stable within sessions but decreased across sessions. Three different situational interest trajectories were found. Situational interest trajectories were predicted by pre-test individual interest. In turn, situational interest had an effect on post-test individual interest. Students whose situational interest trajectories were stable (either high or low) presented no changes in individual interest, yet the individual interest of students whose situational interest was triggered but not maintained markedly decreased from pre-test to post-test. Results suggest that it is important to use game-based learning not because games are believed to be “motivating”; rather, games with proven learning outcomes should be carefully selected.
Full-text available
Strategy flexibility, adaptivity, and the use of clever shortcut strategies are of major importance in current primary school mathematics education worldwide. However, empirical results show that primary school students use such shortcut strategies rather infrequently. The aims of the present study were to analyze the extent to which Dutch sixth graders (12-year-olds) use shortcut strategies in solving multidigit addition, subtraction, multiplication, and division problems, to what extent student factors and task instructions affected this frequency of shortcut strategy use, and to what extent the strategies differed in performance. A sample of 648 sixth graders from 23 Dutch primary schools completed a paper-and-pencil task of 12 multidigit arithmetic problems, designed to elicit specific shortcut strategies such as compensation. Based on the students’ written work, strategies were classified into whether a shortcut strategy was used or not. Results showed that the frequency of shortcut strategies ranged between 6 and 21% across problem types, and that boys and high mathematics achievers were more inclined to use shortcut strategies. An explicit instruction to look for a shortcut strategy increased the frequency of these strategies in the addition and multiplication problems, but not in the subtraction and division problems. Finally, the use of shortcut strategies did not yield higher performance than using standard strategies. All in all, spontaneous as well as stimulated use of shortcut strategies by Dutch sixth graders was not very common.
Full-text available
Contemporary theories of expertise development highlight the crucial role of deliberate practice in the development of high level performance. Deliberate practice is practice that intentionally aims at improving one’s skills and competencies. It is not a mechanical or repetitive process of making performance more fluid. Instead, it involves a great deal of thinking, problem solving, and reflection for analyzing, conceptualizing, and cultivating developing performance. This includes directing and guiding future training efforts that are then fine-tuned to dynamically evolving levels of performance. Expertise studies, particularly in music and sport, have described early forms of deliberate practice among children. These findings are made use of in our analysis of the various forms of practice in school mathematics. It is widely accepted that mathematics learning requires practice that results in effortless conducting of lower level processes (such as quick and accurate whole number arithmetic with small numbers), which relieve cognitive capacity for more complex tasks. However, the typical training of mathematical skills in educational contexts can be characterized as drill-and-practice that helps automatize basic skills, but often leads to inert routine skills instead of adaptive and flexible number knowledge. In this article we summarize findings of studies which describe students’ self-initiated, deliberate practice in learning number knowledge and intervention studies applying deliberate practice in mathematics teaching, including technology-based learning environments aimed at triggering practice that goes beyond mechanical repeating of number skills.
Full-text available
Traditionally measured skills with arithmetic are not related to later algebra success at levels that would be expected given the close conceptual relation between arithmetic and algebra. However, adaptivity with arithmetic may be one aspect of arithmetic competences that can account for additional variation in algebra attainment. With this in mind, the present study aims to present evidence for the existence and relevance of a newly acknowledged component of adaptivity with arithmetic, namely, adaptive number knowledge. In particular, we aim to examine whether there are substantial individual differences in adaptive number knowledge and to what extent these differences are related to arithmetic and pre-algebra skills and knowledge. Adaptive number knowledge is defined as the well-connected knowledge of numerical characteristics and relations. A large sample of 1065 Finnish late primary school students completed measures of adaptive number knowledge, arithmetic conceptual knowledge, and arithmetic fluency. Three months later they completed a measure of pre-algebra skills. Substantial individual differences in adaptive number knowledge were identified using latent profile analysis. The identified profiles were related to concurrent arithmetic skills and knowledge. As well, adaptive number knowledge was found to predict later pre-algebra skills, even after taking into account arithmetic conceptual knowledge and arithmetic fluency. These results suggest that adaptive number knowledge is a relevant component of mathematical development, and may help account for disparities in algebra development.
Full-text available
Adaptivity with arithmetic problem solving is a key aspect in the long-term development of mathematical skills, and the knowledge of numerical characteristics and relations is a core component of adaptivity with whole-number arithmetic. The present studies represent the first attempt to investigate the nature of the adaptive number knowledge, referring to the well-connected knowledge of numerical characteristics and arithmetic relations between numbers, which underlies adaptivity with arithmetic. A new measurement tool, the Adaptive Number Knowledge Task, was developed to capture individual differences in primary school students' and adults' number knowledge. Three groups of participants – 3rd to 5th graders, university students, and 6th graders – completed the measure, which required participants to produce arithmetic sentences equaling a target number by using any combination of four to five given numbers and arithmetic operations. These studies revealed substantial individual differences in the quantity and quality of participants' adaptive number knowledge. Adaptive number knowledge was found to be related to arithmetic fluency and knowledge of arithmetic concepts in 6th graders. Results suggest that the Adaptive Number Knowledge Task was able to capture individual differences in recognizing and using numerical relations in whole-number arithmetic problem solving, and that these differences may help explain differences in adaptivity with arithmetic.
Through involvement in this project, the teachers came to understand and incorporate into their teaching some principles to advance children's learning (not only in mental computation): 1. Determine the existing knowledge of the children (in this project, the teachers interviewed the children one-on-one). 2. Identify associated concepts that are necessary for con-nected understanding (I gave the teachers a concept map for mental computation, but they formulated their own when introducing other topics). 3. Teach the associ-ated concepts and support the children in "seeing" the connections. © 2011 by the National Association for the Education of Young Children.