Michael F. Mctear

University of Ulster · School of Computing and Mathematics

We developed an innovative system that combines Natural Language Understanding (NLU), a curated knowledge base, and the efficient management of a Large Language Model (LLM) to support motivational health coaching. Using Rasa as the core framework, we enhanced it by integrating the GPT-3.5-turbo model. Users opt into reflective dialogues during conv...

Conversation design plays a pivotal role in the development of a successful conversational system. In light of the customer dissatisfaction issues that arose from earlier telephone-based interactive voice response (IVR) systems, companies now recognize the criticality of delivering exceptional user experiences as they embrace the new and rapidly ev...

For many years, the conventional approach to conversation was based on interconnected modules to process user input and generate system output, as depicted in Figure 2-1.

The main objective of a conversational system is to facilitate meaningful and satisfying interactions between the system and human users. Determining the extent to which this has been achieved successfully involves evaluating the system’s performance to verify whether it functions as intended and assessing how it has been perceived by end users in...

After reading previous chapters, we assume you are eager to start working on a project implementing Generative AI or even building your own AI application. In this chapter, we want to discuss the safety and ethics of generative AI applications, especially applications using LLMs.

In Chapter 3, we explored the architecture of neural conversational systems. In this chapter, we explore Large Language Models (LLMs) which are used in this architecture to process the user’s inputs and generate responses by the system.

On November 30, 2022, OpenAI, a prominent US company with headquarters in San Francisco, released a publicly available version of a chatbot called ChatGPT that transformed the world of Conversational Artificial Intelligence (AI) and ignited what has come to be known as “The Conversational AI Arms Race.” Within just five days of its launch, ChatGPT...

In Chapter 5, we introduced a new discipline called prompt engineering, which is rapidly evolving and becoming more defined as time goes by. We have already learned about basic elements of prompts such as various prompt patterns and use cases, as well as useful prompt techniques that might help conversation designers to be more productive in creati...

In one of her interviews, CTO of Open AI, Mira Murati talks about prompt engineering: To the question by Emily Chang (Bloomberg): “What are some tips on being an ace prompt engineer?” Mira replies: “It’s the ability to develop an intuition to get the most out of the model.”

In previous chapters, we provided an overview of how LLMs work, as well as how well-structured prompts can help get the desired results out of the model. All previous material will serve as a solid foundation for the current chapter, in which we will look at various platforms for building conversational applications.

Conversational AI is a dynamic and fast moving field, and a lot has happened in the six months or so since we began writing this book. We have tried to ensure that what we have covered in the preceding chapters provides a sufficiently general and comprehensive foundation that will remain relevant despite the rapid pace of new developments.

Background This research is part of the STOP project, a H2020 RISE project funded by European Commission (GA No 823978) to address the challenge of preventing obesity in Europe. The interdisciplinary European STOP project aims to establish a data and knowledge ecosystem as a basis for the STOP Portal to enable healthcare professionals in decision s...

Chatbot responses can be generated using traditional rule-based conversation design or through the use of large language models (LLMs). In this paper we compare the quality of responses provided by LLM-based chatbots with those provided by traditional conversation design. The results suggest that in some cases the use of LLMs could improve the qual...

A chatbot usability questionnaire (CUQ) was designed to measure the usability of chatbots. Study objectives: 1) to test the construct validity of CUQ (i.e. does it differentiate between chatbots that we rank as having poor, average or good usability), 2) to assess the intra-rater reliability of CUQ (i.e. do participants provide the same answers/sco...

Since life expectancy has increased significantly over the past century, society is being forced to discover innovative ways to support active aging and elderly care. The e-VITA project, which receives funding from both the European Union and Japan, is built on a cutting edge method of virtual coaching that focuses on the key areas of active and he...

In this toolkit, you will find recommended activities and methods which were used to adopt a stakeholder-centred approach, which is important for the responsible design of digital health technologies.

With life expectancy growing rapidly over the past century, societies are being increasingly faced with a need to find smart living solutions for elderly care and active ageing. The e-VITA project, which is a joint European (H2020) and Japanese (MIC) funded project, is based on an innovative approach to virtual coaching that addresses the crucial d...

Digital technologies such as chatbots can be used in the field of mental health. In particular, chatbots can be used to support citizens living in sparsely populated areas who face problems such as poor access to mental health services, lack of 24/7 support, barriers to engagement, lack of age appropriate support and reductions in health budgets. T...

Sociocognitive Language Processing (SCLP) is the idea of coping with everyday language, including slang and multi-lingual phrases and cultural aspects, and in particular, with irony/sarcasm/humour, and paralinguistic information such as the physical and mental state and traits of the dialogue partner (e.g., affect, age groups, personality dimension...

Contemporary societies are comprised of individuals very diverse in terms of culture, status, gender and age. In this context, there is no single system behaviour that fits all users, not even considering the traditional “personalization” efforts of adaptable systems, in which individual users can explicitly tailor some system features to their nee...

The objective of this study was to understand the attitudes of professionals who work in mental health regarding the use of conversational user interfaces, or chatbots, to support people’s mental health and wellbeing. This study involves an online survey to measure the awareness and attitudes of mental healthcare professionals and experts. The find...

Current research in dialogue systems focuses almost entirely on end-to-end approaches in which an input utterance is mapped directly to an output response without requiring any processing by the modules of the traditional modularised architecture. This approach uses DNNs within a Seq2Seq architecture and is often referred to as neural dialogue.

Until around 2000 dialogue systems developed in academic and industrial research laboratories were based on rules that determined the system’s behavior. Consider Example 2.1 in which the system has to choose between three different possible responses to the user’s utterance: Example 2.1 U1: I want to book a flight to Boston. S1.1: Sorry, please rep...

Although dialogue systems go back a long way, it was not until the 1990s that competitions were set up to systematically compare the performance of different systems on specified tasks. An example in the U.S. was the DARPA Communicator Project that addressed the task of complex travel planning and included evaluations of systems developed in a numb...

A dialogue system is a computer program that supports spoken, text-based, or multimodal conversational interactions with humans. Generally, a distinction is made between task-oriented and non-task-oriented dialogue systems. In task-oriented dialogues the human user and the system engage in an interaction to accomplish some task. Example 1.1 is take...

Conversational AI is a fast moving area that has attracted the interest of researchers in natural language processing as well as companies such as Google, Amazon, Facebook, Microsoft, and IBM that have developed speech and language technologies and are now exploring the potential of text-based and spoken dialogue systems. Numerous smaller companies...

Mental health and mental wellbeing have become an important factor to many citizens navigating their way through their environment and in the work place. New technology solutions such as chatbots are potential channels for supporting and coaching users to maintain a good state of mental wellbeing. Chatbots have the added value of providing social c...

Chatbots have become an increasingly popular choice for organisations in delivering services to users. Chatbots are beginning to become popular in mental health applications and are being seen as an accessible strategy in delivering mental health support alongside clinical/therapy treatment. The aim of this exploratory study was to investigate init...

Obesity is a global challenge that affects health and wellbeing worldwide. In this position paper, we review the digital technology used in prevention of obesity and present the proposed STOP project that integrates state-of-the-art wearable technology, chatbot, gamification data fusion and machine learning with the aim to provide personalised supp...

PLEASE NOTE: There is NO "full-text" available for this tool. Please refer to the CUQ tool website for more information (see below): For details of the Chatbot Usability Questionnaire please refer to the Ulster University website: https://www.ulster.ac.uk/research/topic/computer-science/artificial-intelligence/projects/cuq

Chatbots are becoming increasingly popular as a human-computer interface. The traditional best practices normally applied to User Experience (UX) design cannot easily be applied to chatbots, nor can conventional usability testing techniques guarantee accuracy. WeightMentor is a bespoke self-help motivational tool for weight loss maintenance. This s...

The aim of this paper is to assess the usability of a chatbot for mental health care within a social enterprise. Chatbots are becoming more prevalent in our daily lives, as we can now use them to book flights, manage savings, and check the weather. Chatbots are increasingly being used in mental health care, with the emergence of “virtual therapists...

People’s worries about robot and AI software and how it can go wrong have led them to think of it and its associated algorithms and programs as being like Mary Shelley’s Frankenstein monster. The term Franken-algorithms has been used. Furthermore, there are concerns about driverless cars, automated General Practitioner Doctors (GPs) and robotic sur...

This study set out to analyze questions about type 2 diabetes mellitus (T2DM) from patients and the public. The aim was to better understand people’s information needs by starting with what they do not know, discovered through their own questions, rather than starting with what we know about T2DM and subsequently finding ways to communicate that in...

Questions about cure or reversal. (DOCX)

Crowdsourced questions. (DOCX)

Chatbots have become a popular choice for organisations in delivering services and e-commerce to users through using natural language processing (NLP). The popularity of chatbots in mental health is increasing and is seen as an accessible approach to receiving mental health support along with clinical/therapy treatment. The aim of this paper is to...

The conversational interface has become a hot topic in the past year or so, providing the primary means of interaction with chatbots, messaging apps, and virtual personal assistants. Major tech companies have been making huge investments in the supporting technologies of artificial intelligence, such as deep learning and natural language processing...

When patients cannot get answers from health professionals or retain the information given, increasingly they search online for answers, with limited success. Researchers from the United States, Ireland, and the United Kingdom explored this problem for patients with type 2 diabetes mellitus (T2DM). In 2014, patients attending an outpatient clinic (...

Questions engage patients and foster good communication with health professionals. Consumer questions reveal gaps in knowledge that may guide public-health campaigns. But … •What do consumers (patients, family members, care-givers, general public) want to know about an illness or their health? •What do they not know? •How do they express their inf...

Background/Research Question Patient questions play a vital role in healthcare, engaging patients and fostering good communication with health professionals. Consumer questions play a crucial role in public-health prevention efforts, revealing gaps in knowledge that need to be met. For patients, evidence shows that literacy has an effect on the que...

Conversational interfaces have a long history, starting in the 1960s with text-based dialog systems for question answering and chatbots that simulated casual conversation. Speech-based dialog systems began to appear in the late 1980s and spoken dialog technology became a key area of research within the speech and language communities. At the same t...

When a user speaks to a conversational interface, the system has to be able to recognize what was said. The automatic speech recognition (ASR) component processes the acoustic signal that represents the spoken utterance and outputs a sequence of word hypotheses, thus transforming the speech into text. The other side of the coin is text-to-speech sy...

Affect is a key factor in human conversation. It allows us to fully understand each other, be socially competent, and show that we care. As such, in order to build conversational interfaces that display credible and expressive behaviors, we should endow them with the capability to recognize, adapt to, and render emotion. In this chapter, we explain...

In order to build artificial conversational interfaces that display behaviors that are credible and expressive, we should endow them with the capability to recognize, adapt to, and render emotion. In this chapter, we explain how the recognition of emotional aspects is managed within conversational interfaces, including modeling and representation,...

There are a number of different open-source tools that allow developers to add speech input and output to their apps. In this chapter, we describe two different technologies that can be used for conversational systems, one for systems running on the Web and the other for systems running on mobile devices. For the Web, we will focus on the HTML5 Web...

There is a wide range of tools that support various tasks in spoken language, some of which are particularly relevant for processing spoken language understanding in conversational interfaces. Here, the main task is to detect the user’s intent and to extract any further information that is required to understand the utterance. This chapter provides...

One of the core aspects in the development of conversational interfaces is to design the dialog management strategy. The dialog management strategy defines the system’s conversational behaviors in response to user utterances and environmental states. The design of this strategy is usually carried out in industry by handcrafting dialog strategies th...

There is a wide range of tools that support the generation of rule-based dialog managers for conversational interfaces. However, it is not as easy to find toolkits to develop statistical dialog managers based on reinforcement learning and/or corpus-based techniques. In this chapter, we have selected the VoiceXML standard to put into practice the ha...

Once the dialog manager has interpreted the user’s input and decided how to respond, the next step for the conversational interface is to determine the content of the response and how best to express it. This stage is known as response generation (RG). The system’s verbal output is generated as a stretch of text and passed to the text-to-speech com...

We are surrounded by a plethora of smart objects such as devices, wearables, virtual agents, and social robots that should help to make our life easier in many different ways by fulfilling various needs and requirements. A conversational interface is the best way to communicate with this wide range of smart objects. In this chapter, we cover the sp...

Conversation is a natural and intuitive mode of interaction. As humans, we engage all the time in conversation without having to think about how conversation actually works. In this chapter, we examine the key features of conversational interaction that will inform us as we develop conversational interfaces for a range of smart devices. In particul...

Conversational interfaces can be built using a variety of technologies. This chapter shows how to create a conversational interface using chatbot technology in which pattern matching is used to interpret the user’s input and templates are used to provide the system’s output. Numerous conversational interfaces have been built in this way, initially...

When they first appeared, conversational systems were developed as speech-only interfaces accessible usually via landline phones. Currently, they are employed in a wide variety of devices such as smartphones and wearables, with different input and output capabilities. Traditional speech-based multimodal interfaces were designed for Web and desktop...

The evaluation of conversational interfaces is a continuously evolving research area that encompasses a rich variety of methodologies, techniques, and tools. As conversational interfaces become more complex, their evaluation has become multifaceted. Furthermore, evaluation involves paying attention not only to the different components in isolation,...

As a result of advances in technology, particularly in areas such as cognitive computing and deep learning, the conversational interface is becoming a reality. Given the vast number of devices that will be connected in the so-called Internet of Things, a uniform interface will be necessary both for users and for developers. We describe current deve...

Conversational interfaces enable people to interact with smart devices using conversational spoken language. This book describes the technologies behind the conversational interface. Following a brief introduction, we describe the intended readership of the book and how the book is organized. The final section lists the apps and code that have been...

With a conversational interface, people can speak to their smartphones and other smart devices in a natural way in order to obtain information, access Web services, issue commands, and engage in general chat. This chapter presents some examples of conversational interfaces and reviews technological advances that have made conversational interfaces...

Spoken language understanding (SLU) involves taking the output of the speech recognition component and producing a representation of its meaning that can be used by the dialog manager (DM) to decide what to do next in the interaction. As systems have become more conversational, allowing the user to express their commands and queries in a more natur...

This book provides a comprehensive introduction to the conversational interface, which is becoming the main mode of interaction with virtual personal assistants, smart devices, various types of wearables, and social robots. The book consists of four parts: Part I presents the background to conversational interfaces, examining past and present work...

Computer technology has been reported to pose significant usability problems for older users. Further usability problems have been encountered with small, mobile computing devices due to their size as well as age-related declines. This chapter focuses on the usability of mobile computing devices for older people by first employing target users in a...

As life expectancy increases it has become more necessary to find ways to support healthy ageing. A number of active ageing initiatives are being developed nowadays to foster healthy habits in the population. This paper presents our contribution to these initiatives in the form of a conversational agent that acts as a coach for physical activities....

This paper details a knowledge-based Technology Transfer Framework which attempts to ameliorate the transfer of technology between differing user communities through the use of knowledge-based authoring and browsing facilities. In contrast to current techniques, the system incorporates a large corpus of domain knowledge in a central and easily acce...

This paper is concerned with the implications of recent work in cognitive science for the study of human-computer interaction. One feature of the communicative competence of humans is their ability to take the perspective of their listeners. The computer counterpart of this ability — user modelling for intelligent tutoring systems — is examined and...

Spoken dialogue technology has developed considerably over the past thirty years both in terms of research activity as well as in terms of commercially deployed applications. This chapter presents an overview of trends in dialogue research based on an analysis of papers that were presented at Eurospeech- Interspeech conferences in 1989, 1999, and 2...

This chapter reviews the role of spoken dialogue in such environments. It begins by describing and comparing the functions and technical characteristics of different spoken dialogue applications, including voice control, call routing, voice search, and question answering. This is followed by a review of recent research that illustrates the current...

Spoken dialogue systems, Autonomous preference learning, User feedback, Human-computer interaction. We present an enhanced method for user feedback in an autonomous learning system that includes a spoken dialogue system to manage the interactions between the users and the system. By means of a rule-based natural language understanding module and a...

In this paper, we present a technique for learning new dialog strategies by using a statistical dialog manager that is trained from a dialog corpus. A dialog simulation technique has been developed to acquire data required to train the dialog model and then explore new dialog strategies. A set of measures has also been defined to evaluate the dialo...

In this chapter, we take a closer look at some systems and system frameworks that implement and extend the various dialogue control strategies and models introduced in Chapter 2. We begin with Information State Theory, which is an influential method for representing complex information in the Dialogue Context Model. After this, we present an exampl...

There are two main types of spoken dialogue systems: task-oriented and nontask-oriented. Taskoriented systems involve the use of dialogues to accomplish a task, which may be a fairly simple and well-defined task such as making a hotel booking, or a more complex task involving planning and reasoning, such as planning a family holiday, negotiating a...

The Dialogue Manager (DM) is the central component of a spoken dialogue system, controlling the interaction with the user and communicating with external knowledge sources. Dialogue management is often viewed in terms of two subcomponents: dialogue control, which deals with the flow of control in the dialogue, and dialogue context modeling, which i...

Dialogue technology has developed to a level where practical interactive systems can be built and deployed commercially. However, in order to increase the naturalness and flexibility of the interaction, researchers in spoken dialogue technology are addressing the challenges posed by systems that display more intelligent conversational capabilities....

The preceding chapters have presented an overview of spoken dialogue systems and have examined a range of different approaches and methods that are currently in use within the research community. Chapter 1 provided a general introduction to the topic of spoken dialogue technology, looking at the components of a spoken dialogue system, presenting so...

In this chapter, we focus on the evaluation of dialogue systems. We first present a short historical overview of the evaluation task in Section 6.1, and then discuss some concepts and definitions of evaluation approaches and data collection in Section 6.2. We then introduce different evaluation metrics in Section 6.3, and evaluation frameworks in S...

Miscommunication can occur in any communicative interaction, so it is important that a spoken dialogue system should have mechanisms for detecting and dealing with errors given the impact that errors can have on dialogue performance. For example, in one study it was found that when the frequency of error was low, the impact on task success was mino...

An Intelligent Environment is a physical space that becomes augmented with computation, communication and digital content, thus transcending the limits of direct human perception. Spoken dialogue is a key factor for user-friendly human-computer interaction. This article details how to integrate Spoken Dialogue Systems into Intelligent Environments....

Considerable progress has been made in recent years in the development of dialogue systems that support robust and efficient human-machine interaction using spoken language. Spoken dialogue technology allows various interactive applications to be built and used for practical purposes, and research focuses on issues that aim to increase the system's...

Reminiscence plays an important role in the lives of older adults (8). Many perfect the art of storytelling and enjoy its social benefits. The telling of stories of past events and experiences defines family identities and is an integral part of most cultures. Losing the ability to recollect past memories is not only disadvantageous, but can prove...

We present the design of a spoken dialogue system to provide feedback to users of an autonomous system which can learn different patterns associated with user actions. Our speech interface allows users to verbally refine these patterns, giving the system his/her feedback about the accuracy of the actions learnt.We focus on improving the naturalness...

This paper presents an overview of methods that can be used to collect and analyse data on user responses to spoken dialogue system components intended to increase human-likeness, and to evaluate how well the components succeed in reaching that goal. ...

Semantics deals with the organization of meanings and the relations between sensory signs or symbols and what they denote or mean. Computational semantics performs a conceptualization of the world using computational processes for composing a meaning representation structure from available signs and their features present, for example, in words and...

This paper reports on a collaborative project between clinicians at the Ulster Hospital and researchers at the University of Ulster to produce a Web-based system for monitoring patients with Type 2 diabetes. The typical method of recording measurements of weight, blood sugar, and blood pressure allows for minimal intervention for the consultants as...

Miscommunication occurs frequently in any communicative interaction. Despite a long history of studies of miscommunication across a range of different disciplines, the focus in spoken dialogue technology has been almost exclusively on miscommunication caused by speech recognition errors. This chapter reviews approaches to miscommunication in severa...

Mobile technologies have the potential to enhance the lives of older adults, especially those who experience a decline in cognitive abilities. However, diminutive devices often perplex the aged and many HCI problems exist. This paper discusses the use of Artificial Intelligence (AI) techniques to develop intelligent algorithms which will (a) govern...