Zoraida Callejas

University of Granada | UGR · Departamento de Lenguajes y Sistemas Informáticos

This paper presents the multidisciplinary work carried out in the RTVE-UGR Chair within the IVERES project, whose main objective is the development of a tool for journalists to verify the veracity of the audios that reach the newsrooms. In the current context, voice synthesis has both beneficial and detrimental applications, with audio deepfakes be...

Conversational interfaces are becoming ubiquitous in an increasing number of application domains as Artificial Intelligence, Natural Language Processing and Machine Learning methods associated with the recognition, understanding and generation of natural language advance by leaps and bounds. However, designing the dialog model of these systems is s...

In recent years, transformer-based models have played a significant role in advancing language modeling for natural language processing. However, they require substantial amounts of data and there is a shortage of high-quality non-English corpora. Some recent initiatives have introduced multilingual datasets obtained through web crawling. However,...

The ChatSubs dataset [5] contains dialogue data in Spanish and three of Spain's co-official languages (Catalan, Basque, and Galician). It has been obtained from OpenSubtitles, from which we have gathered the movie subtitles in our languages of interest and processed them to generate clearly segmented dialogues and their turns. The data processing c...

In this paper we present a study that evaluates different machine learning models for fault detection based on the optimal operation of the biological methanation process. The optimal operation has been obtained from a multi-objective dynamic optimization based on an extended model of the anaerobic digestion model (ADM1 ME). Two datasets have been...

Conversational interfaces offer users a natural way to interact with a range of applications and devices. Human-machine interaction using these systems involves different components that mimic the mechanisms used by humans when using language and speech interaction. In this paper, we are interested in automatically developing the dialog state track...

Conversational interfaces make it possible for users to communicate in their own language, thus making systems easier to use. Thanks to these characteristics, language-based interaction brings multiple benefits for digital mental health systems, as they are more accessible to their users, increasing adherence and rapport. This way, digital interven...

Mental health is one of the most significant public health challenges. In recent years, several motivational applications have been proposed to help users maintain good mental health. However, many times these applications focus on uplifting messages or general resources that are not tailored to the specific needs of their users. In contrast, non-p...

Conversational interfaces have recently become ubiquitous in the personal sphere by improving an individual’s quality of life and industrial environments by automating services and their corresponding cost savings. However, designing the dialog model used by these interfaces to decide the following response is a hard-to-accomplish task for complex...

In the recent years, transformer-based models have lead to significant advances in language modelling for natural language processing. However, they require a vast amount of data to be (pre-)trained and there is a lack of corpora in languages other than English. Recently, several initiatives have presented multilingual datasets obtained from automa...

Intent recognition is a key component of any task-oriented conversational system. The intent recognizer can be used first to classify the user’s utterance into one of several predefined classes (intents) that help to understand the user’s current goal. Then, the most adequate response can be provided accordingly. Intent recognizers also often appea...

Dialogue systems have an increasingly higher number of applications and so their development is raising interest both in academic and industrial setting. Dialogue management is a key aspect for the development of these systems, as it is in charge of the decision making processes and the identification of the most appropriate responses to the user i...

Emotion recognition is attracting the attention of the research community due to its multiple applications in different fields, such as medicine or autonomous driving. In this paper, we proposed an automatic emotion recognizer system that consisted of a speech emotion recognizer (SER) and a facial emotion recognizer (FER). For the SER, we evaluated...

Emotion Recognition is attracting the attention of the research community due to the multiple areas where it can be applied, such as in healthcare or in road safety systems. In this paper, we propose a multimodal emotion recognition system that relies on speech and facial information. For the speech-based modality, we evaluated several transfer-lea...

Coffee plays a key role in the generation of rural employment in Colombia. More than 785,000 workers are directly employed in this activity, which represents the 26% of all jobs in the agricultural sector. Colombian coffee growers estimate the production of cherry coffee with the main aim of planning the required activities, and resources (number o...

Contemporary societies are comprised of individuals very diverse in terms of culture, status, gender and age. In this context, there is no single system behaviour that fits all users, not even considering the traditional “personalization” efforts of adaptable systems, in which individual users can explicitly tailor some system features to their nee...

Conversational interfaces have recently become a ubiquitous element in both the personal sphere by easing access to services, and industrial environments by the automation of services, improved customer support and its corresponding cost savings. However, designing the dialog model used by these interfaces to decide system responses is still a hard...

Humans and machines harmoniously collaborating and benefiting from each other is a long lasting dream for researchers in robotics and artificial intelligence. An important feature of efficient and rewarding cooperation is the ability to assume possible problematic situations and act in advance to prevent negative outcomes. This concept of assistanc...

Deep learning is providing very positive results in areas related to conversational interfaces, such as speech recognition, but its potential benefit for dialog management has still not been fully studied. In this paper, we perform an assessment of different configurations for deep-learned dialog management with three dialog corpora from different...

Currently, the diagnosis of major depressive disorder (MDD) and its subtypes is mainly based on subjective assessments and self-reported measures. However, objective criteria as Electroencephalography (EEG) features would be helpful in detecting depressive states at early stages to prevent the worsening of the symptoms. Scientific community has wid...

As the complexity of intelligent environments grows, there is a need for more sophisticated and flexible interfaces. Conversational systems constitute a very interesting alternative to ease the users’ workload when interacting with such environments, as they can operate them in natural language. A number of commercial toolkits for their implementat...

Intent Detection is a key component of any task-oriented conversational system. To understand the user’s current goal and provide the most adequate response, the system must leverage its intent detector to classify the user’s utterance into one of several predefined classes (intents). This objective can also simplify the set of processes that a con...

In this paper, we present a proposal for emotion recognition using audio speech signal features consisting of two functionally independent systems. First, a voice activity detection module (VAD) acts as a filter prior to the emotion classification task. It extracts features from the input audio and uses a SVM classifier to predict the presence of v...

Recent advances in spoken language technology, artificial intelligence, and conversational interface design, coupled with the emergence of smart devices, have increased the possibilities of using conversational interfaces for a growing range of application domains. These interfaces are currently applied in the healthcare domain in a range of innova...

One of the most demanding tasks when developing a dialog system consists of deciding the next system response considering the user’s actions and the dialog history, which is the fundamental responsibility related to dialog management. A statistical dialog management technique is proposed in this work to reduce the effort and time required to design...

It has become pressing to develop objective and automatic measurements integrated in intelligent diagnostic tools for detecting and monitoring depressive states and enabling an increased precision of diagnoses and clinical decision-makings. The challenge is to exploit behavioral and physiological biomarkers and develop Artificial Intelligent (AI) m...

Deep learning is providing very positive results in areas related to conversational interfaces, such as speech recognition, but its potential benefit for dialog management has still not been fully studied. In this paper, we perform an assessment of different configurations for deep-learned dialog management with three dialog corpora from different...

In recent years, sentiment analysis has attracted a lot of research attention due to the explosive growth of online social media usage and the abundant user data they generate. Twitter is one of the most popular online social networks and a microblogging platform where users share their thoughts and opinions on various topics. Twitter enforces a ch...

With the re-emergence of role playing games, interactive adventures, fantasy novels and tabletop games, the storytelling industry has a renewed interest to create engaging stories that require an interactive world-building process, in which the scenario where the story occurs is constructed, establishing the different regions, cultures and people t...

The aim of this paper is to present a preliminary scientometric study of the area of conversational systems in Spain. In order to do so, we have used the Web of Science database to retrieve the papers in the area using a comprehensive list of keywords and considering those papers with at least one author with Spanish affiliation. Our results presen...

This book compiles and presents a synopsis on current global research efforts to push forward the state of the art in dialogue technologies, including advances to the classical problems of dialogue management, language generation, question answering, human–robot interaction, chatbots design and evaluation, as well as topics related to the human nat...

In this paper, we propose a novel solution for popularity recognition. This methodology consists of categorizing and exploiting web image resources of people in terms of relevant identities. To demonstrate its usefulness, we also study the effects of incorporating this procedure into a visual diarization system. In our setting, training data is obt...

This paper reports on the GTH-UPM team experience in the Predicting Media Memorability task at MediaEval 2020. Teams were requested to predict memorability scores at both short-term and long-term, understanding such score as a measure of whether a video was perdurable in a viewer's memory or not. Our proposed system relies on a late fusion of the s...

Mental health and mental wellbeing have become an important factor to many citizens navigating their way through their environment and in the work place. New technology solutions such as chatbots are potential channels for supporting and coaching users to maintain a good state of mental wellbeing. Chatbots have the added value of providing social c...

Conversational systems have become an element of everyday life for billions of users who use speech‐based interfaces to services, engage with personal digital assistants on smartphones, social media chatbots, or smart speakers. One of the most complex tasks in the development of these systems is to design the dialogue model, the logic that provided...

We present an educational data analytics case study aimed at the early detection of potential dropout in Computer Engineering studies in Cuba. We have employed institutional data of 456 students and performed several experiments for predicting their permanency into three (promotion, repetition, and dropout) or two classes (promoting, not promoting)...

This longitudinal study concerns the analysis of 347 doctoral theses on scientific medical information retrieved from the TESEO database and defended in Spanish universities from 1977 to 2018. At the same time, it considers other factors, such as the geographical scope distinguishing between dissertations defended in the Spanish region of Levante a...

El propósito de esta investigación es identificar los factores relevantes que inciden en la deserción de los estudiantes universitarios, en particular en el contexto de carreras del perfil Ingeniería Informática en la Educación Superior cubana. Se analizan investigaciones previas en el área y estudios específicos de deserción en las enseñanzas técn...

Resumen. La deserción en los estudios de Educación Superior es la consecuencia máxima del fracaso de los estudiantes en este nivel de enseñanza; es una problemática compleja, relevante y de alcance internacional por lo que el estudio de los factores que determinan la decisión del estudiante de abandonar se ha incrementado en las últimas décadas. Te...

Devices with oral interfaces are enabling new interesting interaction scenarios and ways of interaction in ambient intelligence settings. The use of several of such devices in the same environment opens up the possibility to compare the inputs gathered from each one of them and perform a more accurate recognition and processing of user speech. Howe...

Currently there exist many tools that support monitoring and encouragement of healthy nutrition habits in the context of wellness promotion. In this domain, interfaces based on natural language provide more flexibility for nutritional self-reporting than traditional form-based applications, allowing the users to provide richer and spontaneous descr...

In this paper, we present a statistical model for spoken dialog segmentation that decides the current phase of the dialog by means of an automatic classification process. We have applied our proposal to three practical conversational systems acting in different domains. The results of the evaluation show that is possible to attain high accuracy rat...

Check full text here: http://hdl.handle.net/10481/58919 https://www.isca-speech.org/archive/Interspeech_2019/abstracts/2230.html

Electrodermal activity (EDA) is a psychophysiological indicator that can be considered a somatic marker of the emotional and attentional reaction of subjects towards stimuli like audiovisual content. EDA measurements are not biased by the cognitive process of giving an opinion or a score to characterize the subjective perception, and group-level ED...

Purpose This paper aims to identify emerging fronts and hot topics in educational research. The identification of such educational trends can help decision-makers in educational policies and research agendas. Design/methodology/approach Through a quantitative and scientometric research approach, the authors analyze a sample of 198 highly cited sci...

In this paper, we present a methodology for the development of embodied conversational agents for social virtual worlds. The agents provide multimodal communication with their users in which speech interaction is included. Our proposal combines different techniques related to Artificial Intelligence, Natural Language Processing, Affective Computing...

Recent advances in Artificial Intelligence, Semantic Web and intelligent interaction devices have made conversational interfaces increasingly popular. These advances in technologies including automatic speech recognition and synthesis, natural language understanding and generation, and dialog management are result of decades of work in these areas...

Social Virtual Worlds are increasingly being used in education, as their flexibility can be exploited in order to create heterogeneous groups from all over the world who can collaborate synchronously in different virtual spaces. In this paper, the authors describe the potential of virtual worlds as an educative tool to teach and learn abstract conc...

Good health is the result of a healthy lifestyle, where caring about physical activity and nutrition are key concerns. However, in today’s society, nutritional disorders are becoming increasingly frequent, affecting children, adults, and elderly people, mainly due to limited nutrition knowledge and the lack of a healthy lifestyle. A commonly adopte...

Nutrition e-coaches have demonstrated to be a successful tool to foster healthy eating habits, most of these systems are based on graphical user interfaces where users select the meals they have ingested from predefined lists and receive feedback on their diet. On one side the use of conversational interfaces based on natural language processing al...

Technological integration is currently a key factor in teaching and learning. New interaction handheld devices (such as smartphones and tablets) are opening new learning scenarios that require more sophisticated applications and learning strategies. This chapter is focused on the high variety of educational applications that multimodal conversation...

As conversational technologies develop, we demand more from them. For instance, we want our conversational assistants to be able to solve our queries in multiple domains, to manage information from different usually unstructured sources, to be able to perform a variety of tasks, and understand open conversational language. However, developing the r...

Counselling dialogue systems are designed to help users to change and monitor their behaviours in order to achieve beneficial goals, such as the acquisition of healthy habits. To be effective, it is important that these systems include a model that accounts for the effort that users are investing to achieve the goals. However, most of the systems a...

In this paper we propose to combine speech-based and linguistic classification in order to obtain better emotion recognition results for user spoken utterances. Usually these approaches are considered in isolation and even developed by different communities working on emotion recognition and sentiment analysis. We propose modeling the users emotion...

Social Virtual Worlds are increasingly being used in education, as their flexibility can be exploited in order to create heterogeneous groups from all over the world who can collaborate synchronously in different virtual spaces. In this paper, the authors describe the potential of virtual worlds as an educative tool to teach and learn abstract conc...

In this chapter, we discuss the wide variety of applications for which multimodal conversational systems are being used in education within the context of gamification. The chapter also describes a modular and scalable framework to develop such systems efficiently for mobile devices and virtual environments. To show its potentiality, we present two...

Counselling systems such as recommendation systems and virtual coaches assist users to gradually achieve their goals. For that purpose, it is usual to devise a progression plan consisting of intermediate, possibly interrelated, tasks or goals to be accomplished in order to guide counselees from their current state to a (desirable) target state, whi...

Smart mobile devices have fostered new learning scenarios that demand sophisticated interfaces. Multimodal conversational agents have became a strong alternative to develop human-machine interfaces that provide a more engaging and human-like relationship between students and the system. The main developers of operating systems for such devices have...

El aprendizaje a través de Internet está revolucionando la forma de concebir los modelos de formación en las instituciones, y la irrupción del fenómeno de los MOOC (cursos masivos, abiertos y en línea) es la punta del iceberg de este proceso de cambio. Este libro recoge las experiencias llevadas a cabo en el marco del proyecto de innovación docent...

Conversational interfaces have a long history, starting in the 1960s with text-based dialog systems for question answering and chatbots that simulated casual conversation. Speech-based dialog systems began to appear in the late 1980s and spoken dialog technology became a key area of research within the speech and language communities. At the same t...

When a user speaks to a conversational interface, the system has to be able to recognize what was said. The automatic speech recognition (ASR) component processes the acoustic signal that represents the spoken utterance and outputs a sequence of word hypotheses, thus transforming the speech into text. The other side of the coin is text-to-speech sy...

Affect is a key factor in human conversation. It allows us to fully understand each other, be socially competent, and show that we care. As such, in order to build conversational interfaces that display credible and expressive behaviors, we should endow them with the capability to recognize, adapt to, and render emotion. In this chapter, we explain...

In order to build artificial conversational interfaces that display behaviors that are credible and expressive, we should endow them with the capability to recognize, adapt to, and render emotion. In this chapter, we explain how the recognition of emotional aspects is managed within conversational interfaces, including modeling and representation,...

There are a number of different open-source tools that allow developers to add speech input and output to their apps. In this chapter, we describe two different technologies that can be used for conversational systems, one for systems running on the Web and the other for systems running on mobile devices. For the Web, we will focus on the HTML5 Web...

There is a wide range of tools that support various tasks in spoken language, some of which are particularly relevant for processing spoken language understanding in conversational interfaces. Here, the main task is to detect the user’s intent and to extract any further information that is required to understand the utterance. This chapter provides...

One of the core aspects in the development of conversational interfaces is to design the dialog management strategy. The dialog management strategy defines the system’s conversational behaviors in response to user utterances and environmental states. The design of this strategy is usually carried out in industry by handcrafting dialog strategies th...

There is a wide range of tools that support the generation of rule-based dialog managers for conversational interfaces. However, it is not as easy to find toolkits to develop statistical dialog managers based on reinforcement learning and/or corpus-based techniques. In this chapter, we have selected the VoiceXML standard to put into practice the ha...

Once the dialog manager has interpreted the user’s input and decided how to respond, the next step for the conversational interface is to determine the content of the response and how best to express it. This stage is known as response generation (RG). The system’s verbal output is generated as a stretch of text and passed to the text-to-speech com...

We are surrounded by a plethora of smart objects such as devices, wearables, virtual agents, and social robots that should help to make our life easier in many different ways by fulfilling various needs and requirements. A conversational interface is the best way to communicate with this wide range of smart objects. In this chapter, we cover the sp...

Conversation is a natural and intuitive mode of interaction. As humans, we engage all the time in conversation without having to think about how conversation actually works. In this chapter, we examine the key features of conversational interaction that will inform us as we develop conversational interfaces for a range of smart devices. In particul...

Conversational interfaces can be built using a variety of technologies. This chapter shows how to create a conversational interface using chatbot technology in which pattern matching is used to interpret the user’s input and templates are used to provide the system’s output. Numerous conversational interfaces have been built in this way, initially...

When they first appeared, conversational systems were developed as speech-only interfaces accessible usually via landline phones. Currently, they are employed in a wide variety of devices such as smartphones and wearables, with different input and output capabilities. Traditional speech-based multimodal interfaces were designed for Web and desktop...

The evaluation of conversational interfaces is a continuously evolving research area that encompasses a rich variety of methodologies, techniques, and tools. As conversational interfaces become more complex, their evaluation has become multifaceted. Furthermore, evaluation involves paying attention not only to the different components in isolation,...

As a result of advances in technology, particularly in areas such as cognitive computing and deep learning, the conversational interface is becoming a reality. Given the vast number of devices that will be connected in the so-called Internet of Things, a uniform interface will be necessary both for users and for developers. We describe current deve...

Conversational interfaces enable people to interact with smart devices using conversational spoken language. This book describes the technologies behind the conversational interface. Following a brief introduction, we describe the intended readership of the book and how the book is organized. The final section lists the apps and code that have been...

With a conversational interface, people can speak to their smartphones and other smart devices in a natural way in order to obtain information, access Web services, issue commands, and engage in general chat. This chapter presents some examples of conversational interfaces and reviews technological advances that have made conversational interfaces...

Spoken language understanding (SLU) involves taking the output of the speech recognition component and producing a representation of its meaning that can be used by the dialog manager (DM) to decide what to do next in the interaction. As systems have become more conversational, allowing the user to express their commands and queries in a more natur...

Spoken dialog systems have demonstrated a high potential for more flexible, usable and natural human-computer interaction. These improvements are highly dependent on the users’ adaptation and dialog management processes, which respectively integrates adaptation capabilities and decides the next system response for the current dialog state. In this...