Kaska Porayska-Pomsta

University College London | UCL · Institute of Education

The Activity Theory-based Model of Serious Games (ATMSG) provides a visual framework through which designers and researchers can explicitly map the gaming, learning, and instructional design of their learning game mechanics and game flow. Here, we use the ATMSG to redesign an existing learning game, Stop & Think (S&T), which was created to train ch...

This paper presents the key conclusions to the forthcoming edited book on The Ethics of Artificial Intelligence in Education: Practices, Challenges and Debates (August 2022, Routlege). As well as highlighting the key contributions to the book, it discusses the key questions and the grand challenges for the field of AI in Education (AIED)in the cont...

The human–computer interaction (HCI) design of educational technologies influences cognitive behaviour, so it is imperative to assess how different HCI strategies support intended behaviour. We developed a neuroscience-inspired game that trains children's use of “stopping-and-thinking” (S&T)—an inhibitory control-related behaviour—in the context of...

While Artificial Intelligence in Education (AIED) research has at its core the desire to support student learning, experience from other AI domains suggest that such ethical intentions are not by themselves sufficient. There is also the need to consider explicitly issues such as fairness, accountability, transparency, bias, autonomy, agency, and in...

Filipino learners’ lack of English language proficiency is a major barrier to higher education opportunities and participation in high-value industries. Computer-based learning systems have the potential to increase educational quality, equity, and efficacy in the Global South. However, a key challenge is to design systems that are developmentally...

Evidence from cognitive neuroscience suggests that learning counterintuitive concepts in mathematics and science requires inhibitory control (IC). This prevents interference from misleading perceptual cues and naïve theories children have built from their experiences of the world. Here, we (1) investigate associations between IC, counterintuitive r...

Computerised educational neuroscience interventions that train within-domain inhibitory control (IC) can improve children’s counterintuitive reasoning. However, the HCI or adaptive design of such environments often receive less attention. Eye-tracking and click data were used to compare four versions of an IC-training game in terms of their HCI des...

Metacognitive experience (ME) plays an important role in self-regulated learning. To date, through mainly self-reporting methodology, metacognition assessment lacks objective evidence and therefore hinder the discussion of its subjective and implicit nature. In exploring ME, eye-tracking and skin conductance response (SCR) offer certain advantages...

Autism Spectrum Condition (ASC) is a life-long diagnosis, which has a subset of features including hyper-, seeking- and/or hypo-reactivity to sensory inputs or unusual interests (APA, 2013). These qualities are evident across environmental (e.g. response to specific sounds, visual fascination with lights or movements) and physiological domains (e.g...

Animals and humans use a midbrain structure to coordinate and process relevant visual and auditory stimuli while suppressing distracting information. In modelling this assembly and managing both environmental and physiological stimuli using engineering principles, my research aspires to deep learning models that sense, categorize and alert autistic...

Autism Spectrum Condition (ASC) is a life-long diagnosis, which has a subset of individualized characteristics consisting of hyper-, seeking- and/or hypo-reactivity to sensory inputs or unusual interests (APA, 2013). These sensitivities are evident in both environmental (e.g. apparent response to specific sounds, visual fascination with lights or m...

Accountability is an important dimension of decision-making in human and artificial intelligence (AI). We argue that it is of fundamental importance to inclusion, diversity and fairness of both the AI-based and human-controlled interactions and any human-facing interventions aiming to change human development, behaviour and learning. Less debated,...

Filipino learners’ lack of English language proficiency is a major barrier to higher education opportunities and participation in high-value industries. Computer-based learning systems have the potential to increase educational quality, equity, and efficacy in the Global South. However, a key challenge is to design systems that are developmentally...

In this paper, we discuss some of the results of a participatory design workshop used to elicit design guidelines for an education game for phonemic awareness intended for use by disadvantaged students. Using a grounded theory approach, we analyze facilitators’ observations from the workshop and related findings to well-established game design guid...

The paradigm of neurodiversity provides a theoretical scaffold to challenge the idea of dyslexia as a deficit, by considering how difficulties related to literacy may reflect possible cognitive strengths and opportunities for learning. In this paper we adopt this perspective which associates dyslexia with strengths in visual, oral and three-dimensi...

Interpretability of the underlying AI representations is a key raison d'\^{e}tre for Open Learner Modelling (OLM) - a branch of Intelligent Tutoring Systems (ITS) research. OLMs provide tools for 'opening' up the AI models of learners' cognition and emotions for the purpose of supporting human learning and teaching. Over thirty years of research in...

Interpretability of the underlying AI representations is a key raison d'\^{e}tre for Open Learner Modelling (OLM) -- a branch of Intelligent Tutoring Systems (ITS) research. OLMs provide tools for 'opening' up the AI models of learners' cognition and emotions for the purpose of supporting human learning and teaching. Over thirty years of research i...

Metacognitive competencies related to cognitive tasks have been shown to be a powerful predictor of learning. However, considerably less is known about the relationship between student’s metacognition related to non-cognitive dimensions, such as their affect or lifestyles, and academic performance. This paper presents a preliminary analysis of data...

The use of Artificial Intelligence in supporting social skills development is an emerging area of interest in education. This paper presents work which evaluated the impact of a situated experience coupled with open learner modelling on 16–18 years old learners’ verbal and non-verbal behaviours during job interviews with AI recruiters. The results...

Metacognitive competencies related to cognitive tasks have been shown to be a powerful predictor of learning. However, considerably less is known about the relationship between student’s metacognition related to non-cognitive dimensions, such as their affect or lifestyles, and academic performance. This paper presents a preliminary analysis of data...

This study examines patterns of rhyme identification among English Language Learners (ELLs) towards the development of an educational game, JOLLY, intended to improve phonemic awareness among school-aged children in the Philippines. Leveraging on students’ intrinsic interest in Western popular music, we ask students to identify rhyming words from a...

This two volume set LNAI 10947 and LNAI 10948 constitutes the proceedings of the 19th International Conference on Artificial Intelligence in Education, AIED 2018, held in London, UK, in June 2018. The 45 full papers presented in this book together with 76 poster papers, 11 young researchers tracks, 14 industry papers and 10 workshop papers were car...

Metacognition is a neglected area of investment in formal education and in teachers’ professional development. This paper presents an approach and tools, created by a London-based company called Performance Learning Education (PL), for supporting front-line teachers and learners in developing metacognitive competencies. An iterative process adopted...

Experimental intervention studies constitute the current dominant research designs in the autism education field. Such designs are based on a ‘knowledge transfer’ model of evidence-based practice in which research is conducted by researchers, and is then ‘transferred’ to practitioners to enable them to implement evidence-based interventions. While...

Evidence-based practice (EBP) is of critical importance in education where emphasis is placed on the need to equip educators with an ability to independently generate and reflect on evidence of their practices in situ – a process also known as praxis. This paper examines existing research related to teachers’ metacognitive skills and, using two exe...

Storytelling is a powerful means of expression especially for voices that may be difficult to hear or represent in typical ways. This paper reports and reflects on our experiences of co-creating digital stories with school practitioners in a project focusing on embedding innovative technologies for children on the autism spectrum in classroom pract...

We define job interviews as a domain of interaction that can be modelled automatically in a serious game for job interview skills train-ing. We present four types of studies: (1) field-based human-to-human job interviews, (2) field-based computer-mediated human-to-human in-terviews, (3) lab-based wizard of oz studies, (4) field-based human-to-agent...

This paper presents ECHOES, a serious game built to help young children with autism spectrum conditions practice social communication skills. We focus on the design and implementation of the interactive learning activities, which take place in a two-dimensional sensory garden, and the autonomous virtual agent, which acts as a credible social partne...

The goal of this paper is to present a scenario-based serious-game simulation platform that supports social training and coaching in the context of job interviews. It relies on the use of virtual agents that are capable to react in real time to a human interlocutor's expressed affects, through the detection of non-verbal cues, their analysis and th...

The TARDIS project aims to build a scenario-based serious-game simulation platform for NEETs and job-inclusion associations that supports so-cial training and coaching in the context of job interviews. This paper presents the general architecture of the TARDIS job interview simulator, and the serious game paradigm that we are developing.

In this review, we focus on research that has used technology to provide cognitive training - i.e. to improve performance on some measurable aspect of behaviour - in individuals with autism spectrum disorders. We review technology-enhanced interventions that target three different cognitive domains: (a) emotion and face recognition, (b) language an...

This paper describes the Intelligent Engine (IE) of ECHOES, a serious game built for helping young children with Autism Spectrum Conditions acquire social communication skills. ECHOES IE’s main component is an autonomous virtual agent that acts as a credible social partner for children with autism by engaging them in interactive learning activities...

ECHOES is a game that supports young children on the autism spectrum in exploring and learning social communication skills. Our research demonstrates the potential of computer games as complimentary educational tools in autism intervention. IOE Research Briefings are short descriptions of significant research findings, based on the wide range of pr...

This paper presents an approach that makes use of a virtual character and social signal processing techniques to create an immersive job interview simulation environment. In this environment, the virtual character plays the role of a recruiter which reacts and adapts to the user's behavior thanks to a component for the automatic recognition of soci...

Research on the relationship between affect and cognition in Artificial Intelligence in Education AIEd brings an important dimension to our understanding of how learning occurs and how it can be facilitated. Emotions are crucial to learning, but their nature, the conditions under which they occur, and their exact impact on learning for different le...

This demo presents an approach to recognising and interpreting social cues-based interactions in computer-enhanced job interview simulations. We show what social cues and complex mental states of the user are relevant in this interaction context, how they can be interpreted using static Bayesian Networks, and how they can be recognised automaticall...

This paper describes the design and implementation of a planning-based socially intelligent agent built to help young children with Autism Spectrum Conditions acquire social communication skills. We explain how planning technology allowed us to satisfy the requirements relating to the agent’s design that we identified through our consultations with...

This paper describes the design and implementation of a planning-based socially intelligent agent built to help young children with Autism Spectrum Conditions acquire social communication skills. We explain how planning technology allowed us to satisfy the requirements relating to the agent's design that we identified through our consultations with...

Modelling human tutors' socio-linguistic abilities computationally is necessary to underpin human–computer educational interactions with best pedagogic practices and to enhance the naturalness of such interactions. In this paper we present a computational model of tutorial feedback selection based on the context of the immediate situation for which...

This paper introduces the SHARE-IT project, which leverages serious games paradigm to motivate and engage children with autism diagnosis in interactive activities, based on the state-of-the-art autism intervention practices. The aim of SHARE-IT is to formulate, in partnership with schools, parents and industry, the requirements for a robust, intell...

This paper describes the Intelligent Engine (IE) of ECHOES, a serious game built for helping young children with Autism Spectrum Conditions acquire social communication skills. ECHOES IE's main component is an autonomous virtual agent that acts as a credible social partner for children with autism by engaging them in interactive learning activities...

This paper introduces the SHARE-IT project, which leverages serious games paradigm to motivate and engage children with autism diagnosis in interactive activities, based on the state-of-the-art autism intervention practices. The aim of SHARE-IT is to formulate, in partnership with schools, parents and industry, the requirements for a robust, intell...

We present the design and implementation of an autonomous virtual agent that acts as a credible social partner for children with Autism Spectrum Conditions and supports them in acquiring social communication skills. The agent's design is based on principles of best autism practice and input from users. Initial experimental results on the efficacy o...

We present an interdisciplinary methodology for designing interactive technology for young children with Autism Spectrum Disorders (ASD). In line with many other researchers in the field, we believe that the key to developing technology in this context is to embrace perspectives from diverse disciplines to arrive at a methodology that delivers sati...

Learner modelled environments Learner modelled environments (LMEs) are digital environments that are capable of automatically detecting learners’ behaviours in relation to a specific knowledge domain, to reason about those behaviours in order to assess learners’ performance, skills, socio-emotional and cognitive needs and to act accordingly in a pe...

Several recent studies have reported that cognitive training in adults does not lead to generalized performance improvements [1, 2], whereas many studies with younger participants (children 4 years and older) have reported distal transfer [3, 4]. This is consistent with convergent evidence [5-8] for greater neural and behavioral plasticity earlier...

Children with ASD have difficulty with social communication, particularly joint attention. Interaction in a virtual environment (VE) may be a means for both understanding these difficulties and addressing them. It is first necessary to discover how this population interacts with virtual characters, and whether they can follow joint attention cues i...

Background: Virtual environments and characters allow individuals with an autism spectrum disorder (ASD) to practice skills repeatedly, in a way that may be less threatening, less socially demanding, and more controllable than is face-to-face interaction with a human partner. To date, their use as an intervention tool has focused on teaching situat...

We present an interdisciplinary methodology for designing interactive multi-modal technology for young children with autism spectrum disorders (ASDs). In line with many other researchers in the field, we believe that the key to developing technology in this context is to embrace perspectives from diverse disciplines to arrive at a methodology that...

The development of social communication skills in children relies on multimodal aspects of communication such as gaze, facial expression, and gesture. We introduce a multimodal learning environment for social skills which uses computer vision to estimate the children's gaze direction, processes gestures from a large multi-touch screen, estimates in...

This paper describes how researchers from diverse research disciplines are working together with design teams of children, carers and practitioners to create an exploratory multimodal environment for children. This learning environment, entitled ECHOES II, aims to be both an educational intervention and an environment through which we research chil...

In this paper, we discuss the notion of embodiment in the context of the ECHOES project, which aims at developing a multi-modal interactive environment for scaffolding young typically developing (TD) children and children with Asperger Syndrome (AS) in acquiring social interaction skills. Whilst, our approach to embodiment is in line with the curre...

In this paper we explore human tutors’ inferences in relation to learners’ affective states and the relationship between those inferences and the actions that tutors take as their consequence. At the core of the investigations presented in this paper lie fundamental questions associated with the role of affective considerations in computer-mediated...

It is well known that human tutors take into account both the student’s knowledge and understanding of what is being taught, in addition to considering the emotional and motivational state of the student. However, there are many gaps in our understanding of the relationship between cognition and affect in tutoring. We have some insight into how hum...

This report provides a description of the integration of the Extended Learner Model (xLM) into LEACTIVEMATH, replacing the earlier ACTIVEMATH learner model component. It includes a brief description of xLM architecture in relation to LEACTIVEMATH and how information is exchanged between them via event messaging and procedure calls, followed by deta...

This report is part of deliverable D30 Report and Prototype of Diagnostic Functionalities and provides a description of the implementation of the diagnostic functionalities of the Extended Learner Model (xLM), the prototype learner modelling subsystem of LEACTIVEMATH that replaces the earlier ACTIVEMATH learner model. It includes also an explanatio...

Providing students with cognitive and affective support is generally recognised as important to their successful learning. There is an intuitive recognition of the two types of support being related, but little research explains how such a relationship may be manifested in teaching strategies, or what conditions tutors strategic choices in relation...

One of the main objectives of research in Natural Language generation (NLG) is to account for linguistic variation in a systematic way. Research on linguistic politeness provides important clues as to the possible causes of linguistic variation and the ways in which it may be modelled formally. In this paper we present a simple language generation...

Natural language is characterised by enormous linguistic variation (e.g., Fetzer (2003)). Such variation is not random, but is determined by a number of contextual factors. These factors encapsulate the socio-cultural conventions of a speech community and dictate the socially acceptable, i.e. polite, use of language. Producing polite language may n...

It has been long recognised in education that teaching and learning is a highly social and emotional activity. Students’ cognitive progress depends on their psychological predispositions such as their interest, confidence, sense of progress and achievement as well as on social interactions with their teachers and peers who provide them (or not) wit...

Studies aimed at understanding what makes human tutoring effective have noted that the type of indirect guidance that characterizes human tutorial dialogue is a key factor. In this paper, we describe an approach that brings together sociolingusitic research on the basis of linguistic choice with natural language generation technology to systematica...

This document provides a specification of the Student Model as required by the workplan for LEACTIVEMATH. The specification relatesWork Package 4 Student Model (WP4) to the other LEACTIVEMATH components and describes its subcomponents. The Work Package main deliverable, the student modelling subsystem of LEACTIVEMATH, is renamed as the “Extended Le...

In this paper we examine a possible method for classifying speech acts produced by human teachers, with a view of informing the designs of intelligent natural language tutors and of providing the basis for a formal analysis of the effects that teachers' language has on students' learning. We argue that traditional means as initiated by the Ordinary...

We describe the architecture and the components of BEETLE, a dialogue- enhanced Basic Electricity and Elec- tronics Tutorial Learning Environment.

This paper provides a foundation for a prag- matically and empirically based model of the mechanisms involved in teachers' speech act generation. Apart from some standard types of speech acts such as questioning or assert- ing, teachers also make use of some complex acts which combine the qualities of other known types of speech acts. These complex...