Fig 2 - uploaded by Rui Prada
Content may be subject to copyright.

Context in source publication

Context 1
... Modules Inc is a single player game where the player takes on the role of a customer service representative for a spaceship part manufacturer "Space Modules Inc". The virtual characters in the game play the role of customers that call the player (see Figure 2) about hardware and software faults they are experiencing. Some characters will be angry, others uncooperative or stressed, and it's up to the player to manage the situation and decide how best to respond. ...

Similar publications

Article
Full-text available
The current Covid-19 pandemic poses an unprecedented global challenge in the field of education and training. As we have seen, the lack of proper information about the virus and its transmission has forced the general population and healthcare workers to rapidly acquire knowledge and learn new practices. Clearly, a well-informed population is more...
Chapter
Full-text available
Blockchain technologies are already being used in many different applications to secure specific data sets. For the same purpose, blockchain is also used in digital games. This technology is not only discussed, but also actively implemented in the educational sector, for example, in such projects as “Blockcerts”. Typical applications are issuing ce...
Article
Full-text available
We explore how an AR simulation created by a multidisciplinary team evolved into a more interactive, student-centered learning game. The CovidCampus experience was designed to help college students understand how their decisions can affect their probability of infection throughout a day on campus. There were eight decision points throughout the day...

Citations

... Human communications through digital platforms and virtual spaces are prevalent in many applications, including online learning [34,36,52], virtual interviewing [7], counseling [16], social robotics [60], automated character designing [40], storyboard visualizing for consumer media [31,58], and creating large-scale metaverse worlds [45]. Simulating immersive experiences in such digital applications necessitates the development of plausible human avatars with expressive faces and body motions. ...
Preprint
Full-text available
We present a multimodal learning-based method to simultaneously synthesize co-speech facial expressions and upper-body gestures for digital characters using RGB video data captured using commodity cameras. Our approach learns from sparse face landmarks and upper-body joints, estimated directly from video data, to generate plausible emotive character motions. Given a speech audio waveform and a token sequence of the speaker's face landmark motion and body-joint motion computed from a video, our method synthesizes the motion sequences for the speaker's face landmarks and body joints to match the content and the affect of the speech. We design a generator consisting of a set of encoders to transform all the inputs into a multimodal embedding space capturing their correlations, followed by a pair of decoders to synthesize the desired face and pose motions. To enhance the plausibility of synthesis, we use an adversarial discriminator that learns to differentiate between the face and pose motions computed from the original videos and our synthesized motions based on their affective expressions. To evaluate our approach, we extend the TED Gesture Dataset to include view-normalized, co-speech face landmarks in addition to body gestures. We demonstrate the performance of our method through thorough quantitative and qualitative experiments on multiple evaluation metrics and via a user study. We observe that our method results in low reconstruction error and produces synthesized samples with diverse facial expressions and body gestures for digital characters.
... The game business has undergone a change due to AI, becoming a dynamic and interactive environment (Mascarenhas et al., 2018). The user experience has been improved and the bounds of what is feasible have been pushed by the integration of AI technology into different parts of gaming in recent years. ...
Chapter
Full-text available
This chapter explores the fascinating nexus of artificial intelligence (AI), gaming, and individualized mental health care, creating a revolutionary environment for the field of mental health in the future. Early detection and specialized treatments for mental health concerns are now possible due to AI's data-processing skills. Gamification, which uses gaming to make therapy exciting and effective, is transforming conventional gaming from a purely recreational activity to a powerful tool for mental health assistance. A more approachable and stigma-free setting for mental health discussions is promoted by the convergence of AI and gaming, which offers enormous potential for tailored mental health therapy. In this era of technology advancement, these developments have the potential to redefine what it means to be well-being, ushering in a new era of intelligent, interactive, and highly individualized mental health care.
... All rights reserved. generating dancing digital characters for applications such as character design (Mascarenhas et al. 2018), storyboard visualization for consumer media (Kucherenko et al. 2020;Watson et al. 2019), building metaverse tools (Omniverse 2021) and even advancing our understanding of the relationships between music and dance (Brown and Parsons 2008). ...
Article
We present DanceAnyWay, a generative learning method to synthesize beat-guided dances of 3D human characters synchronized with music. Our method learns to disentangle the dance movements at the beat frames from the dance movements at all the remaining frames by operating at two hierarchical levels. At the coarser "beat" level, it encodes the rhythm, pitch, and melody information of the input music via dedicated feature representations only at the beat frames. It leverages them to synthesize the beat poses of the target dances using a sequence-to-sequence learning framework. At the finer "repletion" level, our method encodes similar rhythm, pitch, and melody information from all the frames of the input music via dedicated feature representations. It generates the full dance sequences by combining the synthesized beat and repletion poses and enforcing plausibility through an adversarial learning framework. Our training paradigm also enforces fine-grained diversity in the synthesized dances through a randomized temporal contrastive loss, which ensures different segments of the dance sequences have different movements and avoids motion freezing or collapsing to repetitive movements. We evaluate the performance of our approach through extensive experiments on the benchmark AIST++ dataset and observe improvements of about 7%-12% in motion quality metrics and 1.5%-4% in motion diversity metrics over the current baselines, respectively. We also conducted a user study to evaluate the visual quality of our synthesized dances. We noted that, on average, the samples generated by our method were about 9-48% more preferred by the participants and had a 4-27% better five-point Likert-scale score over the best available current baseline in terms of motion quality and synchronization. Our source code and project page are available at https://github.com/aneeshbhattacharya/DanceAnyWay.
... This heterogeneous environment allows for assessing large-scale systems, as it may be difficult to find human participants for the "humans-only" Serious Game. The system can instead be padded with autonomous, socially intelligent characters with whom players can practise Mascarenhas et al. (2018). However, equipping the agents with 'human-like' characteristics to better engage human players in a realistic environment is a challenge Magnenat-Thalmann and Kasap (2009). ...
Chapter
Full-text available
People’s decisions and actions are informed, influenced, and constrained by socially constructed social arrangements. Usually, these social arrangements are pre-determined, and people joining institutions or organizations may have, at least initially, little control or influence over them. Occasionally, however, but increasingly commonly in the transition to the “Digital Society,” people have an opportunity to self-determine their social arrangements “from scratch.” The issues then are: how do people gain experience in such founding processes, experiment safely with alternative social arrangements, and gain expertise so that they can participate meaningfully, for example in consultation events. In this research paper, we address these issues by presenting a framework for “computer-supported social arrangements” tools which would enable people to specify, learn, apply, evaluate, and innovate social arrangements. This would contribute to empowering people and communities with the necessary experience, experimentation, and expertise for effective and sustainable self-determination of their own social arrangements.
... Especially valuable in the context of serious games is not only the aforemen�oned adapta�on of game contents to the player's needs but, for instance, also guidance of the player by means of intelligent virtual agents (K. Anderson et al., 2013;Mascarenhas et al., 2018) or the ability to interact with the game harnessing all human communica�on channels including voice and gestures, i.e. in a mul�-modal fashion (Kocsis, Ganchev, Mporas, Papadopoulos, & Fakotakis, 2009;Goli, Teymournia, Naemabadi, & Garmaroodi, 2022). By applying the technology of natural language processing (NLP), chatbots allow humans to interact with computers through natural language (Winkler & Söllner, 2018). ...
Chapter
More than fifty years have passed since the first serious computer games appeared. Their general goal is to motivate players to learn and to improve their learning outcomes. To this end, the players are given specific tasks and challenges in entertaining and engaging contexts. Serious games have evolved from a traditional video game setup to diverse, playful, real-time interactive systems also harnessing, for instance, novel extended reality interfaces. There are two obvious challenges in the design of serious games: (1) Turning learning objectives into games, and (2) establishing a playful, rewarding game-loop that ensures the learning objectives are met. The paths to address these challenges vary depending on the learning objectives and contexts, the deployment technology and the available resources for development. The purpose of this chapter is to elaborate on the underlying conceptual and technological challenges and to present perspectives on theoretical and practical solutions.
... All rights reserved. generating dancing digital characters for applications such as character design (Mascarenhas et al. 2018), storyboard visualization for consumer media (Kucherenko et al. 2020;Watson et al. 2019), building metaverse tools (Omniverse 2021) and even advancing our understanding of the relationships between music and dance (Brown and Parsons 2008). ...
Preprint
Full-text available
We present DanceAnyWay, a generative learning method to synthesize beat-guided dances of 3D human characters synchronized with music. Our method learns to disentangle the dance movements at the beat frames from the dance movements at all the remaining frames by operating at two hierarchical levels. At the coarser "beat" level, it encodes the rhythm, pitch, and melody information of the input music via dedicated feature representations only at the beat frames. It leverages them to synthesize the beat poses of the target dances using a sequence-to-sequence learning framework. At the finer "repletion" level, our method encodes similar rhythm, pitch, and melody information from all the frames of the input music via dedicated feature representations. It generates the full dance sequences by combining the synthesized beat and repletion poses and enforcing plausibility through an adversarial learning framework. Our training paradigm also enforces fine-grained diversity in the synthesized dances through a randomized temporal contrastive loss, which ensures different segments of the dance sequences have different movements and avoids motion freezing or collapsing to repetitive movements. We evaluate the performance of our approach through extensive experiments on the benchmark AIST++ dataset and observe improvements of about 7%-12% in motion quality metrics and 1.5%-4% in motion diversity metrics over the current baselines, respectively. We also conducted a user study to evaluate the visual quality of our synthesized dances. We note that, on average, the samples generated by our method were about 9-48% more preferred by the participants and had a 4-27% better five-point Likert-scale score over the best available current baseline in terms of motion quality and synchronization. Our source code and project page are available at https://github.com/aneeshbhattacharya/DanceAnyWay.
... In the field of game design, toolkits, namely authoring toolkits, can support students to create artifacts by integrating the necessary tools for them to develop a determine activity [5], [6]. ...
Chapter
Games are engaging activities for students that can be applied in classes worldwide. Simultaneously, toolkit’s potential has been rising over the years due to its ability to encompass relevant tools on specific subjects. Moreover, toolkits can be applied as authoring tools transforming students into game designers, motivating the creation of digital and/or mobile games, and knowledge acquisition and engagement. The Gamers4Nature (G4N) project has been researching on environmental-themed games’ development, involving students in game development sessions. Aiming to support the whole game design process for upper-secondary and undergraduate education, the G4N project conceived a Toolkit to Game Design that has been used in formal and informal contexts of education fostering the creation of digital games on environmental themes. The G4N Toolkit, addressing both experienced and novice students, was design in a User-Centered approach with iterative sessions for validating aesthetics and contents. As it was understood that students sometimes lacked the skills to develop their games, preventing them to accomplish a finished product, a complement to the already produced resources (Game Construction cards, Rapid Game Design Document, environmental-themed cards, and Mobile Game Design Guidelines), was developed: the Development Cards Set. This chapter introduces the Development Cards Set creation process, to be validated through expert review (by experts in the game development field) and by end-users who will test and validate the development cards along dedicated workshops and game creation sessions.KeywordsGame creation processScratch programmingToolkit to game designGamers4Nature
... Moreover, the task is heavily dependent on the designer's ability to imagine a wide array of social situations and use their intuition to specify social behaviours and decision rules for the agent, which may be difficult to articulate [31]. In addition to this, cognitive concepts such as goals and beliefs, which are quite familiar for AI researchers but not necessarily so for developers, which weights in on the decision of adopting such type of frameworks both in industry and academia [32], despite the empirical evidence of the benefits of using SIAs [33]. ...
... As highlighted above agent modelling architectures rely on a type of authoring that is oriented towards cognitive concepts such as goals, beliefs and emotions, among others [32]. An option to translating natural language stories or descriptions into agent-readable concepts is to leverage recent developments in Natural Language Processing (NLP) and Machine Learning (ML) fields to a partial automatization of this task. ...
... While the accessibility of authoring tools can be improved by creating better user interfaces, the issues that prevent the wide adoption of these tools go beyond interface usability. Previous research suggests that understandable conceptual models should guide the procedural principles of authoring tools [11,32]. An explainable tool, scaffolded by theoretical concepts that the author can easily identify, will ease the authoring burden facilitating the adoption by technical and non-technical users. ...
Preprint
Full-text available
The deployment of Socially Intelligent Agents (SIAs) in learning environments has proven to have several advantages in different areas of application. Social Agent Authoring Tools allow scenario designers to create tailored experiences with high control over SIAs behaviour, however, on the flip side, this comes at a cost as the complexity of the scenarios and its authoring can become overbearing. In this paper we introduce the concept of Explainable Social Agent Authoring Tools with the goal of analysing if authoring tools for social agents are understandable and interpretable. To this end we examine whether an authoring tool, FAtiMA-Toolkit, is understandable and its authoring steps interpretable, from the point-of-view of the author. We conducted two user studies to quantitatively assess the Interpretability, Comprehensibility and Transparency of FAtiMA-Toolkit from the perspective of a scenario designer. One of the key findings is the fact that FAtiMA-Toolkit's conceptual model is, in general, understandable, however the emotional-based concepts were not as easily understood and used by the authors. Although there are some positive aspects regarding the explainability of FAtiMA-Toolkit, there is still progress to be made to achieve a fully explainable social agent authoring tool. We provide a set of key concepts and possible solutions that can guide developers to build such tools.
... Additionally, the results of these tests are difficult to interpret, since it is particularly challenging to translate these psychological profiles into behavioural profiles and thus predict the actions and/or daily verbal interactions that are distinctive of each negotiation style. Technology-enhanced platforms for soft skills assessment are usually designed in the form of virtual environments [2][3][4], where simulated agents can help assess the individual's skills in a realistic situation using role-play techniques [5][6][7] (for a review see [8]). However, in most serious games and game-based simulations, data provided by the user is collected in the form of multiple choices [9], non-verbal information such as eye-gaze and gestures [10] or facial expression [11], with little or no attention given to verbal aspects. ...
Article
Full-text available
Negotiation constitutes a fundamental skill that applies to several daily life contexts; however, providing a reliable assessment and definition of it is still an open challenge. The aim of this research is to present an in-depth analysis of the negotiations occurring in a role-play simulation between users and virtual agents using Natural Language Processing. Users were asked to interact with virtual characters in a serious game that helps practice negotiation skills and to complete a psychological test that assesses conflict management skills on five dimensions. The dialogues of 425 participants with virtual agents were recorded, and a dataset comprising 4250 sentences was built. An analysis of the personal pronouns, word context, sentence length and text similarity revealed an overall consistency between the negotiation profiles and the user verbal choices. Integrating and Compromising users displayed a greater tendency to involve the other party in the negotiation using relational pronouns; on the other hand, Dominating individuals tended to use mostly single person pronouns, while Obliging and Avoiding individuals were shown to generally use fewer pronouns. Users with high Integrating and Compromising scores adopted longer sentences and chose words aimed at increasing the other party’s involvement, while more self-concerned profiles showed the opposite pattern.
... Often developed and adopted in the Human-Computer Interaction field, toolkits are used as tools to support and influence interactive systems' design and implementation [12], emerging as a useful resource to be used in: the conceptual and methodological definition of games (e.g. [13,14]); game ideation and/or brainstorming activities (e.g. [15,16]); game narrative creation activities (e.g. ...
... [17][18][19]); and game implementation activities (e.g. [13,20]) either by game design experts or by users with no relevant experience in the game design field. ...
Chapter
Full-text available
Although research indicates that integrating students as active agents in the development of games encourages awareness and learning by doing and can be seen as a basis for the development of critical thinking skills, it also stresses that there are some obstacles in involving students to explore educational contents through digital game design. This paper introduces the Gamers4Nature toolkit to Game Design and its use by upper-secondary (N = 53) and undergraduate (N = 114) students along several game design sessions addressing an environmental preservation theme. As result of these sessions, 66 prototypes of digital games were produced. The toolkit was used through questionnaires applied by the end of the game design sessions. Results indicate that the Toolkit was a very useful resource in scaffolding the narrative construction process and that its resources are adequate to be used by for both upper-secondary and undergraduate students, and that it can be seen as a valuable resource to support educators and trainers in educational and serious game design activities.