PosterPDF Available

SIGDIAL Poster (YRRSDS Workshop) CiViL Commonsense Enhanced Vision and Language for Natural Language Generation in HRI

July 2021

DOI:10.13140/RG.2.2.22570.54722

Conference: SIGDIAL

Authors:

Carl Strathearn

Edinburgh Napier University

CiViL (Commonsense-enhanced Vision and Language) poster for SIGDIAL (YRRSDS Workshop)

Content uploaded by Carl Strathearn

Content may be subject to copyright.

A preview of the PDF is not available

ResearchGate has not been able to resolve any citations for this publication.

ResearchGate has not been able to resolve any references for this publication.

Poster

Full-text available

Commonsense-enhanced Vision and Language (CiViL) for Human-Robot Interaction

June 2023

Carl Strathearn

Socially assistive robots typically employ rule-based dialogue managers (DMs) that lack commonsense knowledge as such concepts are usually implied rather than encoded in dialogue, i.e. intents, entities, slots, etc, this often results in communication failure (questions are out of scope) [1]. To manage such communication phenomena during tasks, we created a hybrid rule-based and generative DM ... [Show full abstract] that employs external knowledge databases to generate new dialogue from context and manage unscripted instances [2]. We integrated a vision component into the DM to automatically detect missing and present objects in scenes for more intuitive HRI.

View full-text

Conference Paper

Full-text available

TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction Spo...

March 2023

The most effective way of communication between humans and robots is through natural language communication. However, there are many challenges to overcome before robots can effectively converse in order to collaborate and work together with humans. This paper introduces TaskMaster 1 a novel cross-platform spoken dialogue system (SDS) for human-robot interaction (HRI) which employs a neural ... [Show full abstract] language model to generate responses in the context of task-based situations. In contrast to previous works that have employed templates and canned text for dialogue in HRI, we show that the dialogue output of TaskMaster is more flexible than a template-based variation. In a series of task-orientated case studies and a video demonstration 2 , we show that in real-world settings TaskMaster can generate more relevant responses to questions, identify missing objects and offer alternatives, confirm and clarify aspects of a task, and adapt to unpredictable situations more effectively than traditional template approaches used in HRI.

View full-text

Poster

Full-text available