Conference PaperPDF Available

Towards Evidence-informed Design Principles for Adaptive Reading Games

  • April 2019
DOI:10.1145/3290607.3313256
  • Conference: Extended Abstracts of the 2019 CHI Conference
Authors:
Manolis Mavrikis
Manolis Mavrikis
Manolis Mavrikis
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Asimina Vasalou at University College London
Asimina Vasalou
Laura Benton at University College London
Laura Benton
Chrysanthi Raftopoulou
Chrysanthi Raftopoulou
Chrysanthi Raftopoulou
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Show all 7 authorsHide
Download full-text PDF
Read full-text
Download citation

Abstract

This demonstration presents the design principles of the Navigo games for reading. By reflecting on our design tools and processes we explore the way theory, empirical evidence and best practice and expertise have informed our design. We look into the reciprocal role of theory and design and provide transferable lessons for design of educational technologies in the context of HCI.
Content uploaded by Kostas Karpouzis
Author content
All content in this area was uploaded by Kostas Karpouzis on May 08, 2019
Content may be subject to copyright.
KEYWORDS
Adaptive games; learning; design; theory
Towards Evidence-informed Design
Principles for Adaptive Reading
Games
Manolis Mavrikis, Asimina Vasalou
and Laurra Benton
m.mavrikis@ucl.ac.uk; a.vasalou@ucl.ac.uk;
l.benton@ucl.ac.uk
UCL Knowledge Lab,
University College London,
WC1N3QS, London, UK
Chrysanthi Raftopoulou,
Antonios Symvonis
crisraft@mail.ntua.gr; symvonis@math.ntua.gr
School of Applied Mathematics and Applied
Physical Sciences,
National Technical University of Athens,
15780 Greece
Drew Wilkins
drew.wilkins@fishinabottle.com
Fish in a bottle
One Chapel Court
Holly Walk, Leamington Spa
CV32 4YS, UK
Kostas Karpouzis
kkarpou@cs.ntua.gr
Artificial Intelligence and Machine Learning
Lab, Electrical & Computer Engineering,
National Technical University of Athens,
15780 Greece
ABSTRACT
1
This demonstration presents the design principles of the Navigo games for young children’s reading.
By reflecting on our design tools and processes we explore the way theory, empirical evidence and
practice have informed our game design. We look into the reciprocal role of theory and design and
provide transferable lessons for design of educational technologies in the context of HCI.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the
full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses,
contact the owner/author(s).
CHI’19 Extended Abstracts, May 4-9, 2019, Glasgow, Scotland, UK.
© 2019 Copyright is held by the author/owner(s).
ACM ISBN 978-1-4503-5971-9/19/05. DOI: https://doi.org/10.1145/3290607.3313256
CHI 2019 Interactivity
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
INT010, Page 1
Figure 1: Navigo Game
1 GAME DESIGN FOR LEARNING
Games for learning offer the potential to personalise and scaffold children’s learning of different
domain topics. Designing learning games, however, is not a straightforward process, and what
instructional dimensions support the learning process has occupied both education and games design
researchers alike. While theory has been long considered to be a critical input to design, in education
technologies evidence-based design is a particularly dominant dimension [e.g. 3].
Over the past two years, the EU-funded iRead consortium has been designing a game that aims to
support primary school children’s reading skills. Our game “Navigo: the Pyramid of the Lost Words”
(see Figure 1) has been motivated by the importance of literacy as a foundational skill with
implications in lifelong learning. In Navigo the player takes on the role of a child whose grandmother
is a world-renowned adventurer investigating a mysterious pyramid. On her request, the player
travels to the desert but is caught in a sandstorm. It transpires that the player’s grandmother, went
into the pyramid to take shelter from the storm and it is down to the player to unlock its secrets
through completing learning activities focused on different reading skills. The game incorporates 16
different mini-game mechanics, which have been designed for six areas: phonology, morphology,
word recognition, orthography, syntax and morpho-syntax. As part of designing Navigo, we have
created 1,000 instantiations of the 16 mini-game mechanics each practising a specific skill.
The goal of this demo is to showcase a number of Navigo game activities and how they are
adaptively sequenced, while at the same time demonstrate our process to reflect on the reciprocal role
of theory, empirical evidence and practice in educational technology design.
2 EVIDENCE INFORMED PEDAGOGICAL DESIGN PRINCIPLES
Our design work has been guided by a combination of learning theory, a critical evaluation of
previous and our own empirical research, and user research through knowledge elicitation in what
can be characterised as evidence-informed design [3]. We use the term ‘informed’ to highlight that
diverse sources and types of evidence are used to support design decisions. This includes ‘theory’ in a
sense similar to [4] i.e. the kind of theories that support us “in generating, selecting and validating
design alternatives at the level at which they are consequential for learning”.
Our initial point of departure was the literature on learning and games, that has been characterized as
‘frameworks of action’ in [4] i.e. theories that provide ‘general prescriptions of pedagogical
strategies’. In a 2018 CHI paper, we identified that instructional elaborative feedback, informing the
child’s understanding and supporting the child in overcoming a breakdown in their learning, is one of
the most powerful interventions raising attainment. We analysed five commercial games for reading
that are currently used in schools, finding that these games embedded principles that supported the
child’s understanding of feedback such as clear learning aims, success criteria and initial instruction of
the learning content. However, we also discovered that these games rarely provided elaborative
feedback, notifying the players on the correctness of their response rather than how to improve it [1].
CHI 2019 Interactivity
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
INT010, Page 2
Design principles for Navigo
DP1 Introduce clear success criteria and
learning aims in each game
DP2 Offer elaborative game feedback when
the child has a breakdown and deter the child
from using trial and error strategies
DP3 Choose content that focuses on a single
learning aim without introducing new, yet
encountered language features
DP4 Introduce interleaved practice of
language features to support better
transferability
DP5 Facilitate a gradual move from
understanding the basic unit of language to
using it in context and then automatizing it
DP6 Discourage the concurrent introduction
of new game mechanics and learning aims
DP7 Strike a balance between maintaining a
good level of challenge and sense of efficacy
DP8 Reinforce previous learning of linguistic
feature after a certain period of time
DP9 Provide data for teacher awareness and
parent engagement and support to
additionally scaffold learning.
Figure 1: Pedagogical design principles
Having developed a critique on the design of existing commercial games we went on to carry out
empirical observational research with these games exploring when and how young children overcome
learning ‘breakdowns’ in the absence of elaborative feedback. In our 2019 CHI paper we showed that
children were able to independently progress in these commercial games only less than half of the
times they encountered a breakdown. When they did progress, they used a trial and error strategy
which was highly mediated by the game mechanic. Trial and error was most prevalent when the
game mechanic allowed children to find the correct answer through a process of elimination [2].
Alongside the broader engagement with games-based learning theories, we found necessary to
employ domain-specific instructional theories of reading. These theories shed light into children’s
reading development allowing us to specify the characteristics of language that children should
encounter at the different stages of their learning. For example, we identified that the words and
sentence constructions encountered by children should focus on a single learning aim, with non-
encountered language constructions taught and practised separately. Engagement with these domain
theories was also critical in making design decisions about which of the 16 mini-game mechanics to
prioritise in gameplay, and which language feature to choose from when sequencing games. For
example, we identified the importance of diversified learning across different learning areas, and the
learning progression from knowing a language rule to applying it and finally automatizing it.
However, as others have recognised [4], even domain-specific theories can often be too vague and
complex to operationalise, or too specific and non-transferable particularly when the design of a
system itself, the tasks that it affords, and the relationship between the environment and the
knowledge domain co-evolve. The latter is particularly the case in the design of adaptivity
components. We thus found it necessary to engage in co-design in the form of ‘knowledge elicitation’
with experts (teachers and linguists) to invent, discuss and iterate over the rules that inform the
selection of the language features, the game content, and the design of the game mechanics.
Figure 1 summarises the design principles that resulted from our literature review, empirical research
and co-design, guiding our overall game design. Tables 1 and 2 show examples of how some of these
design principles were married with the technical language operationalizing the games.
3 DEMONSTRATION
The main aim of our demo is to showcase the design principles and how they are operationalised in
our context as follows.
Demonstrate games for the six reading skills: we will show six mini-game mechanics with
learning aims belonging to the six diverse reading skills in Navigo. Through this, we will evidence the
use of different game mechanics across language areas showing the scaling up of our approach. In
doing so, we will show the connection between our evidence-informed design and the game
outcomes.
CHI 2019 Interactivity
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
INT010, Page 3
Design principle
DP1, DP2, DP3
Language feature
under the phonology
skill
/s/s
Learning aims
Select the word that
starts with s
Success criteria
Four out of five
responses
Elaborative feedback
Words like sad, start
with s
Restrictions on
correct response
content
Maximum word length:
4 characters; Position of
feature: start; Prefix:
none; Suffix: none
Restrictions on
distractor content
Maximum word length:
4 characters; Position of
feature: start; Prefix:
none; Suffix: none
Table 1: Game design specification of a
Navigo game mechanic
Design principle
DP4
If
Features from multiple
language categories are
available
Then
Order the features
selected as follows: (1)
select up to 3 features
from the same category
(2) then move to the
next category (3) repeat
Table 2: Adaptivity rule that directs the
game to choose a learning aim
Demonstrate sequencing of games for the six reading skills: we will play through a small set of
language features to demonstrate how the adaptivity component relies on the language domain and
child’s user model to sequence games by selecting appropriate features, content and game mechanics
providing the children with mastery opportunities.
3 RELEVANCE TO THE CHI COMMUNITY
The iRead project aligns with an aspirational and value-oriented view of design in HCI. We address a
global and societal need to support the development of reading skills to children in primary school for
both developing readers aged 5-8 as well as older children with dyslexia. Of relevance to HCI, we will
show the design representations and tools generated from our aim to design an evidence-informed
game, discuss some of the strengths and challenges in using the tools, and reflect on the facilitative
role of these tool in making some of the, otherwise implicit, decisions and knowledge more explicit
across project members. This has allowed us to work together to connect pedagogical and
technological expertise. Such reflections provide transferable lessons for design of educational
technologies from an HCI perspective. Finally, the interplay of theory and design is of core interest to
CHI researchers and practitioners and the demo of the specific instructional dimensions of the games
and their adaptivity will also enable us to share insights from early trials that further validate,
challenge or advance domain theories. In doing so, we will contribute to the on-going debate in CHI
about the role of theory in design, and the role of design in further developing theory.
ACKNOWLEDGEMENTS
This work forms part of the iRead project which has received funding from the European Union’s
Horizon 2020 research and innovation programme (grant agreement No 731724). We would like to
thank our colleagues across the project and especially Maria Mastropavlou, Michaela Nerantzini
(University of Ioannina, Greece), Kay Berkling (Cooperative State University Karlsruhe, Germany),
and Roger Gilabert Guerrero (University of Barcelona, Spain) for their contributions in the design of
the games and their domain model.
REFERENCES
[1] Laura Benton, Asimina Vasalou, Kay Berkling, Wolmet Barendregt, and Manolis Mavrikis. 2018. A Critical Examination of
Feedback in Early Reading Games. In proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
(p. 373). ACM
[2] Laura Benton, Asimina Vasalou, Wolmet Barendregt, Leona Johansson Bunting, and Andrea Revesz. 2019 What’s Missing:
The Role of Instructional Design in Children’s Games-Based Learning. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems. ACM.
[3] Mutlu Cukurova, Rose Luckin, and Alison Clark‐Wilson. 2019. “Creating the golden triangle of evidence-informed
education technology with EDUCATE,” British Journal of Educational Technology, vol. 50, no. 2.
[4] Andrea diSessa and Paul Cobb. 2004. “Ontological Innovation and the Role of Theory in Design Experiments,” The Journal
of the Learning Sciences, vol. 13, no. 1, pp. 77–103, 2004.
CHI 2019 Interactivity
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
INT010, Page 4
... Alshammari et al. (2016) discussed adaptivity in the context of e-learning and their results indicated that the adaptive version of their learning system had better results in learning effectiveness and perceived usability level, which (according to their analysis) may lead to learners who are more satisfied, engaged, and motivated. The conceptual advantages of adaptivity in a learning system have to do with identifying or selecting the particular areas of interest for each learner and providing them with teaching material and activities suited to those needs; beyond this approach, Mavrikis et al. (2019) and Tsatiris and Karpouzis (2021) designed an adaptive system which responds not only to the particular aspects of language that need to be taught but also to the competence level of each learner: beginning students are presented with introductory material, which they need to master in order to proceed to more advanced aspects, while better performing or more advanced learners skip this part and focus on more sophisticated gameful exercises, avoiding the inevitable boredom of going through a content that is too easy for someone's level, according to Karpouzis and Yannakakis (2016). ...
... In addition, the fact that the gameful application responds to the choices, preferences, and performance of the students was shown to create better rapport and enabled learners to trust the applications and invest time and effort in them, leading to improvements in app retention and conformance with their study programs. In order to improve the learning experience during distance learning, either as part of everyday practice or as a result of special situations, such as the movement restrictions imposed during the pandemic, the ability to personalize the content of each educational intervention and to respond to different levels of engagement (e.g., Legaki et al., 2019) or performance (Mavrikis et al., 2019) points to a new generation of adaptive educational approaches which produce a better experience in real time, as a response to how each learner interacts with them and not merely based on a pre-defined set of materials or a rigid narrative. It has to be noted, though, that, besides engagement, adaptive approaches do not tend to take into account other cognitive or affective states (Pantic et al., 2011), such as frustration, burn-out, or confusion, and only respond to the effect of those states on performance, usually confusing them with the lack of knowledge (Douglas-Cowie et al., 2007). ...
Adaptive game-based learning in education: a systematic review
Article
Full-text available
  • Jun 2023
The advent and abundance of mobile devices and network connectivity have provided learners of all ages with access to potentially unlimited sources of educational material, from pre-school activities to conventional and after-school courses and continuing education. One of the challenges that learners face is how to locate and access interesting contents suited to their preferences and, more importantly, to the level of expertise and individual needs. From the point of view of content creators and educators, adapting the content and the experience to each learner usually leads to better user retention and a more meaningful and deeper learning experience. In this study, we utilized the PRISMA review methodology to examine research on content and experience adaptivity in educational contexts and report on the authors' findings. Consisting of well-defined steps (keyword-based retrieval, study scope definition, result filtering, and grouping and analysis), the systematic nature of this methodology ensures its objectivity and replicability at a future stage or replicated by other researchers.
View
... Besides altering the user interface of the application by choosing specific color schemes or displaying selected, relevant tools in a toolbar, personalization usually refers to the user's avatar, an iconic rendering which represents the user in the app context (Heffernan et al., 2016). Avatar-based representation (Figure 3) is also an artifact found in digital games and appears to increase user empathy to the app, which in turn increases confidence and engagement (Mavrikis et al., 2019;Sebri and Savioni, 2020). In addition, the user's avatar can be used to represent them in the hero's journey mentioned earlier and visualize their current affective or cognitive state: e.g., a smiling face and energetic avatar gestures can be used to represent a user who advances through the app quickly and complies completely with the instructions given to them (Lyles et al., 2017). ...
... In addition, the user's avatar can be used to represent them in the hero's journey mentioned earlier and visualize their current affective or cognitive state: e.g., a smiling face and energetic avatar gestures can be used to represent a user who advances through the app quickly and complies completely with the instructions given to them (Lyles et al., 2017). Avatars are also widely used to implement a rewards-based gamification scheme, where users receive virtual goods (for instance, coins or experience points) as a token of appreciation of their effort and FIGURE 3 | The ability to personalize their avatar has been shown to increase rapport in users (Mavrikis et al., 2019). progress and "spend" them to buy clothes or accessories for their avatar (Schmidt-Kraepelin et al., 2020). ...
Gameful Design of an Application for Patients in Rehabilitation
Article
Full-text available
  • Jul 2022
The design process of any interactive application is an important part of its lifecycle, since it largely defines its structure, means of interaction with the users and its actual content. In the case of applications related to medical uses and self-help, it is even more important, given the aims of the application, the diversity of target users and the urgent need for increased retention. In this article, we present a gameful design process for a mobile application targeted toward patients in rehabilitation, implementing concepts related to increasing user rapport and motivation through gamification, and means to offer guidance and personalized services to improve user experience. Both gamification and personalization build on narrative concepts, by putting patients in the place of a “hero”, offering them the opportunity to overcome “challenges” and receive a clear view of their progress (a.k.a. a “hero's journey”), both in terms of physical and mental condition. Finally, we discuss measurable indicators used to evaluate the application in terms of the progress that patients showed, their motivation and interest, and degree of adherence to the exercise plans.
View
... Educational technologies used in schools have the potential to contribute a number of benefts. These include improved access to digital resources for organising and monitoring learning, fostering student-centred learning, as well as ofering fexible and personalised learning materials [8,17,25]. When scaling up educational technology use beyond a single classroom, previous research has frequently carried out pilots which were supported by the researchers (e.g. ...
Understanding the Situated Practices of School Technology Leaders in the Early Stages of Educational Technology Adoption
Conference Paper
Full-text available
  • Apr 2022
School-driven technological innovation has the potential to positively impact on classroom practice, yet it can also be disrupted by incompatibilities between the existing school ecology and new educational technologies. To help mitigate this disruption a particular staff member often takes on a facilitative leadership role to champion new technology initiatives. However little is known about how this technology leader role impacts on the adoption of new technologies in the classroom. Taking a situated lens, we embarked on a multiple case study of four schools who were aiming to adopt a new literacy game in the classroom. Through interviews with technology leaders and fieldnotes from our site observations, we systematically analysed their actions and concerns over two academic terms. This highlighted an overwhelming concern with managing the material dimension of the technology, teacher agency and division of labour and mechanisms for communication and monitoring. Our findings raise important considerations for HCI researchers seeking to embed their technologies into practice alongside recommendations for supporting leaders tasked with coordinating this process.
View
... Educational technologies used in schools have the potential to contribute a number of benefts. These include improved access to digital resources for organising and monitoring learning, fostering student-centred learning, as well as ofering fexible and personalised learning materials [8,17,25]. When scaling up educational technology use beyond a single classroom, previous research has frequently carried out pilots which were supported by the researchers (e.g. ...
Understanding the Situated Practices of School Technology Leaders in the Early Stages of Educational Technology Adoption
Preprint
Full-text available
  • Apr 2022
School-driven technological innovation has the potential to positively impact on classroom practice, yet it can also be disrupted by incompatibilities between the existing school ecology and new educational technologies. To help mitigate this disruption a particular staf member often takes on a facilitative leadership role to champion new technology initiatives. However little is known about how this technology leader role impacts on the adoption of new technologies in the classroom. Taking a situated lens, we embarked on a multiple case study of four schools who were aiming to adopt a new literacy game in the classroom. Through interviews with technology leaders and feldnotes from our site observations, we systematically analysed their actions and concerns over two academic terms. This highlighted an overwhelming concern with managing the material dimension of the technology, teacher agency and division of labour and mechanisms for communication and monitoring. Our fndings raise important considerations for HCI researchers seeking to embed their technologies into practice alongside recommendations for supporting leaders tasked with coordinating this process.
View
... The core software applications developed in the project are a reader application 7 , which highlights parts of the words contained in the text, given specific criteria, and a serious game 8 , which consists of a series of gamified activities utilising words and sentences. The foundation of these applications and the software infrastructure that provides access to the content consists of language models for each language, including for children with dyslexia; following the definition of extensive phonological and syntactic models for these languages, the linguists in the project worked with teachers to define the learning objectives for each of the target age groups, as well as the sequence in which each language feature should be taught [31]. The sequencing of these features, including which prerequisites should be taught and mastered by the students before moving on to more advanced features, was encoded in a tree-like hierarchical graph; essentially, this graph encapsulates both the language model (i.e. the features that make up each word or sentence, at least at the given language level) and the teaching model, represented by the selection of necessary features for each school year and the succession in which they should be taught. ...
AI in (and for) Games
Chapter
Full-text available
  • Jan 2022
This chapter outlines the relation between artificial intelligence (AI)/machine learning (ML) algorithms and digital games. This relation is two-fold: on one hand, AI/ML researchers can generate large, in-the-wild datasets of human affective activity, player behaviour (i.e. actions within the game world), commercial behaviour, interaction with graphical user interface elements or messaging with other players, while games can utilise intelligent algorithms to automate testing of game levels, generate content, develop intelligent and responsive non-player characters (NPCs) or predict and respond to player behaviour across a wide variety of player cultures. In this work, we discuss some of the most common and widely accepted uses of AI/ML in games and how intelligent systems can benefit from those, elaborating on estimating player experience based on expressivity and performance, and on generating proper and interesting content for a language learning game.
View
... For example, in our community project with reading volunteers, the child mentioned she likes spiders and the volunteer selected a book about spiders for the reading session with the child. In adaptation, performance metrics of individual children are used in the iRead program to select the sequence and difficulty levels of specific reading challenges (Mavrikis et al., 2019). ...
Designing Personalized Instruction: A Research and Design Framework
Article
Full-text available
  • May 2021
Advances in technology have increased the opportunities for designers to personalise instruction based on student actions. We conducted semi‐structured interviews with an international sample of educational professionals including researchers, teachers and designers, and reviewed interdisciplinary literature on personalisation to propose a framework for personalisation research and design. Thematic analysis of the interviews revealed that professionals value each type of personalisation opportunity (eg, customising for age‐appropriate content, supports for student choice, automated guidance based on learner responses) and identify challenges (eg, trade‐offs between adaptive and standardised instruction). Three research/design dilemmas emerged: individualisation and equity; group customisation and individual benefit; and adaptation and validity of measurement. We discuss these dilemmas in relation to three categories of personalisation: customisation by designers or teachers to support a specific audience (grade level, course, community); individualisation to support user choice (of book to read, project topic); and adaptation of instructional activities based on automated analysis of logged user performance (performance metrics, natural language processing, cumulative indicators). We suggest some guiding questions for a generative agenda for future research on personalised instruction. Practitioner notes What is already known about this topic Personalised learning is popular among educational professionals. Personalised design has multiple and inconsistent definitions. A shared framework for personalised instruction would facilitate research and design. What this paper adds A succinct but comprehensive definition of personalised education. Perspectives on personalisation from an international group of practitioners and designers. A framework including three dilemmas to guide future research on the design and practice of personalised instruction. Implications for practice and/or policy A shared definition of personalisation can support communication across diverse stakeholders. The framework can guide future design and instruction with personalised educational technology. The framework identifies dilemmas that illustrate ethical pathways for policy‐makers responsible for personalised education.
View
... The core software applications developed in the project are a reader application 7 , which highlights parts of the words contained in the text, given specific criteria, and a serious game 8 , which consists of a series of gamified activities utilising words and sentences. The foundation of these applications and the software infrastructure that provides access to the content consists of language models for each language, including for children with dyslexia; following the definition of extensive phonological and syntactic models for these languages, the linguists in the project worked with teachers to define the learning objectives for each of the target age groups, as well as the sequence in which each language feature should be taught [31]. The sequencing of these features, including which prerequisites should be taught and mastered by the students before moving on to more advanced features, was encoded in a tree-like hierarchical graph; essentially, this graph encapsulates both the language model (i.e. the features that make up each word or sentence, at least at the given language level) and the teaching model, represented by the selection of necessary features for each school year and the succession in which they should be taught. ...
AI in (and for) Games
Preprint
Full-text available
  • May 2021
This chapter outlines the relation between artificial intelligence (AI) / machine learning (ML) algorithms and digital games. This relation is two-fold: on one hand, AI/ML researchers can generate large, in-the-wild datasets of human affective activity, player behaviour (i.e. actions within the game world), commercial behaviour, interaction with graphical user interface elements or messaging with other players, while games can utilise intelligent algorithms to automate testing of game levels, generate content, develop intelligent and responsive non-player characters (NPCs) or predict and respond player behaviour across a wide variety of player cultures. In this work, we discuss some of the most common and widely accepted uses of AI/ML in games and how intelligent systems can benefit from those, elaborating on estimating player experience based on expressivity and performance, and on generating proper and interesting content for a language learning game.
View
... When a new student registers with the iRead system, this graph is instantiated as a user profile, with different values of mastery for each feature, depending on the students' age. This is where the adaptivity component in iRead kicks in, first by utilising the mastery levels for each feature to select proper content from the project resource engine (dictionaries and texts) and then by updating the student's model based on their performance in each language game they play [2]; when the mastery level for a given feature surpasses a selected threshold (75%), subsequent features in the model hierarchy become available to play with, provided that all prerequisites for them have been met. In the context of iRead, the game content consists of selecting a particular game activity, a language feature to work with, and a set of words or a sentence that corresponds to that feature (e.g. a particular letter, phoneme or a sequence of phonemes). ...
Developing for personalised learning: the long road from educational objectives to development and feedback
Preprint
Full-text available
  • Jan 2021
This paper describes the development needed to support the functional and teaching requirements of iRead, a 4-year EU-funded project which produced an award-winning serious game utilising lexical and syntactical game content. The main functional requirement was that the game should retain different profiles for each student, encapsulating both the respective language model (which language features should be taught/used in the game first, before moving on to more advanced ones) and the user model (mastery level for each feature, as reported by the student's performance in the game). In addition to this, researchers and stakeholders stated additional requirements related to learning objectives and strategies to make the game more interesting and successful; these were implemented as a set of selection rules which take into account not only the mastery level for each feature, but also respect the priorities set by teachers, helping avoid repetition of content and features, and maintaining a balance between new content and revision of already mastered features to give students the sense of progress, while also reinforcing learning.
View
Using Big and Open Data to Generate Content for an Educational Game to Increase Student Performance and Interest
Article
Full-text available
  • Oct 2020
The goal of this paper is to utilize available big and open data sets to create content for a board and a digital game and implement an educational environment to improve students’ familiarity with concepts and relations in the data and, in the process, academic performance and engagement. To this end, we used Wikipedia data to generate content for a Monopoly clone called Geopoly and designed a game-based learning experiment. Our research examines whether this game had any impact on the students’ performance, which is related to identifying implied ranking and grouping mechanisms in the game, whether performance is correlated with interest and whether performance differs across genders. Student performance and knowledge about the relationships contained in the data improved significantly after playing the game, while the positive correlation between student interest and performance illustrated the relationship between them. This was also verified by a digital version of the game, evaluated by the students during the COVID-19 pandemic; initial results revealed that students found the game more attractive and rewarding than a traditional geography lesson.
View
A Critical Examination of Feedback in Early Reading Games
Conference Paper
Full-text available
  • Apr 2018
Learning games now play a role in both formal and informal learning, including foundational skills such as literacy. While feedback is recognised as a key pedagogical dimension of these games, particularly in early learning, there has been no research on how commercial games available to schools and parents reify learning theory into feedback. Using a systematic content analysis, we examine how evidence-based feedback principles manifest in five widely-used learning games designed to foster young children’s reading skills. Our findings highlight strengths in how games deliver feedback when players succeed. Many of the games, however, were inconsistent and not proactive when providing error feedback, often promoting trial and error strategies. Furthermore, there was a lack of support for learning the game mechanics and a preference for task-oriented rewards less deeply embedded in the gameplay. Our research provides a design and research agenda for the inclusion of feedback in early learning games.
View
Ontological Innovation and the Role of Theory in Design
Article
Full-text available
  • Jan 2004
The motivation for this article is our belief that theory is critically important but currently underplayed in design research studies. We seek to characterize and illustrate a genre of theorizing that seems to us strongly synergistic with design-based research. We begin by drawing contrasts with kinds of theory that are relevant but, we contend, by themselves inadequate. A central element of the type of productive design-based theorizing on which we focus is "ontological innovation," hypothesizing and developing explanatory constructs, new categories of things in the world that help explain how it works. A key criterion to which we adhere when discussing ontological innovations is that theory must do real design work in generating, selecting and validating design alternatives at the level at which they are consequential for learning. Developing and refining an ontological innovation is challenging and requires the kind of extensive, iterative work that characterizes design experiments more generally. However, the pay-off in terms of clarity of focus and explanatory power can be great. We present two case studies that illustrate the development, refinement, extension, and instructional application of ontological innovations.
View
What's Missing: The Role of Instructional Design in Children's Games-Based Learning
Conference Paper
  • May 2019
Learning games that address targeted curriculum areas are widely used in schools. Within games, productive learning episodes can result from breakdowns when followed by a breakthrough, yet their role in children's learning has not been investigated. This paper examines the role of game and instructional design during and after breakdowns. We observed 26 young children playing several popular learning games and conducted a moment-by-moment analysis of breakdown episodes. Our findings show children achieve productive breakthroughs independently less than half of the time. In particular, breakdowns caused by game actions are difficult for children to overcome independently and prevent engagement with the domain skills. Importantly, we identify specific instructional game components and their role in fostering strategies that result in successful breakthroughs. We conclude with intrinsic and extrinsic instructional design implications for both game designers and primary teachers to better enable children's games-based learning.
View
Creating the golden triangle of evidence‐informed education technology with EDUCATE
Article
  • Dec 2018
EDUCATE is a London‐based programme that supports the development of research‐informed educational technology (EdTech), allowing entrepreneurs and start‐ups to create their products and services, and simultaneously grow their companies in a more evidence‐informed manner. The programme partners businesses with researchers who mentor, guide and support this research journey, a key aspect of which is the evaluation of the company’s EdTech product or service. However, conducting impact evaluations of technology in education is challenging, particularly for early stage technologies, as rapid cycles of innovation and change are part of their essence. Here, we present the pragmatic approach to evidence‐informed education technology design and impact evaluation, as developed and adopted by the EDUCATE programme. The research process is shaped by the core principles of evidence‐informed decision making detailed in the paper. The contributions of the paper are threefold. First, it defines and details an academia‐industry‐education collaboration model centred on a research training programme. Second, it presents emerging impact results of the programme. Third, it provides clear reflections on the challenges encountered during the implementation of the model in the EdTech ecosystem of London, which should be addressed if we are to move towards evidence‐informed EdTech globally.
View