CARESSES: Culture-Aware Robots and Environmental Sensor Systems for Elderly Support

The article proposes a system for knowledge-based conversation designed for Social Robots and other conversational agents. The proposed system relies on an Ontology for the description of all concepts that may be relevant conversation topics, as well as their mutual relationships. The article focuses on the algorithm for Dialogue Management that selects the most appropriate conversation topic depending on the user input. Moreover, it discusses strategies to ensure a conversation flow that captures, as more coherently as possible , the user intention to drive the conversation in specific directions while avoiding purely reactive responses to what the user says. To measure the quality of the conversation, the article reports the tests performed with 100 recruited participants, comparing five conversational agents: (i) an agent addressing dialogue flow management based only on the detection of keywords in the speech, (ii) an agent based both on the detection of keywords and the Content Classification feature of Google Cloud Natural Language, (iii) an agent that picks conversation topics randomly, (iv) a human pretending to be a chatbot, and (v) one of the most famous chatbots worldwide: Replika. The subjective perception of the participants is measured both with the SASSI (Subjective Assessment of Speech System Interfaces) tool, as well as with a custom survey for measuring the subjective perception of coherence.

This article describes a novel approach to expand in run-time the knowledge base of an Artificial Conversational Agent. A technique for automatic knowledge extraction from the user's sentence and four methods to insert the new acquired concepts in the knowledge base have been developed and integrated into a system that has already been tested for knowledge-based conversation between a social humanoid robot and residents of care homes. The run-time addition of new knowledge allows overcoming some limitations that affect most robots and chatbots: the incapability of engaging the user for a long time due to the restricted number of conversation topics. The insertion in the knowledge base of new concepts recognized in the user's sentence is expected to result in a wider range of topics that can be covered during an interaction, making the conversation less repetitive. Two experiments are presented to assess the performance of the knowledge extraction technique, and the efficiency of the developed insertion methods when adding several concepts in the Ontology.

This trial represents the final stage of the CARESSES project which aimed to develop and evaluate a culturally competent artificial intelligent system embedded into social robots to support older adult wellbeing. A parallel group, single-blind randomised controlled trial was conducted across older adult care homes in England and Japan. Participants randomly allocated to the Experimental Group or Control Group 1 received a Pepper robot for up 18 h across 2 weeks. Two versions of the CARESSES artificial intelligence were tested: a fully culturally competent system (Experimental Group) and a more limited version (Control Group 1). Control Group 2 (Care As Usual) participants did not receive a robot. Quantitative outcomes of interest reported in the current paper were health-related quality of life (SF-36), loneliness (ULS-8), and perceptions of robotic cultural competence (CCATool-Robotics). Thirty-three residents completed all procedures. The difference in SF-36 Emotional Wellbeing scores between Experimental Group and Care As Usual participants over time was significant (F[1] = 6.614, sig = .019, ηp2 = .258), as was the comparison between Any Robot used and Care As Usual (F[1] = 5.128, sig = .031, ηp2 = .146). There were no significant changes in SF-36 physical health subscales. ULS-8 loneliness scores slightly improved among Experimental and Control Group 1 participants compared to Care As Usual participants, but this was not significant. This study brings new evidence which cautiously supports the value of culturally competent socially assistive robots in improving the psychological wellbeing of older adults residing in care settings.

Social robots and artificial agents should be able to interact with the user in the most natural way possible. This work describes the basic principles of a conversation system designed for social robots and artificial agents, which relies on knowledge encoded in the form of an Ontology. Given the knowledge-driven approach, the possibility of expanding the Ontology in run-time, during the verbal interaction with the users is of the utmost importance: this paper also deals with the implementation of a system for the run-time expansion of the knowledge base, thanks to a crowdsourcing approach.

The main objective of this work is to enhance the capabilities of a knowledge-driven conversational system, making them more natural and pleasant. Exploiting several Natural Language Processing (NLP) techniques, a set of algorithms has been developed to improve the quality of the conversation and expand the knowledge base in run-time adding new concepts recognized in the user sentence. Moreover, a mechanism to validate the newly added concepts has been developed.

Cultural competence - i.e., the capability to adapt verbal and non-verbal interaction to the user's cultural background - may be a key element for social robots to increase the user experience. However, designing and implementing culturally competent social robots is a complex task, given that advanced conversational skills are required. In this context, Cloud services may be useful for helping robots in generating appropriate interaction patterns in a culture-aware manner. In this paper, we present the design and the implementation of the CARESSES Cloud, a set of robotic services aimed at endowing robots with cultural competence in verbal interaction. A preliminary evaluation of the Cloud services as a general dialoguing system for culture-aware social robots has been performed, analyzing the feasibility of the architecture in terms of communication and data processing delays.

With rapid advances in Artificial Intelligence (AI) over the last decade, schools have increasingly employed innovative tools, intelligent applications and methods that are changing the education system with the aim of improving both user experience and learning gain in the classrooms. Even though the use of AI to education is not new, it has not unleashed its full potential yet. Much of the available research looks at educational robotics and at non-intelligent robots in education. Only recently, research has sought to assess the potential of Socially Assistive Robots (SARs), including humanoids, within the domain of classroom learning, particularly in relation to learning languages. Yet, the use of this form of AI in the field of mathematics and science constitutes a notable gap in this field. This study aims to critically review the research on the use of SARs in the pre-tertiary classroom teaching of mathematics and science. Further aim is to identify the benefits and disadvantages of such technology. Databases' search conducted between January and April 2018 yielded twenty-one studies meeting the set inclusion criteria for our systematic review. Findings were grouped into four major categories synthesising current evidence of the contribution of SARs in pre-tertiary education: learning gain, user experience, attitude, and usability of SARs within classroom settings. Overall, the use of SARs in pre-tertiary education is promising, but studies focussing on mathematics and science are significantly under-represented. Further evidence is also required around SARs' specific contributions to learning more broadly, as well as enabling/impeding factors, such as SAR's personalisation and appearance, or the role of families and ethical considerations. Finally, SARs potential to enhance accessibility and inclusivity of multi-cultural pre-tertiary classroom is almost unexplored.

Most studies on socially assistive robots (SARs) in elder care are conducted in care homes and recruit participants with some degree of cognitive impairment. The ethical dimension in these studies thus requires careful attention, suggesting that the researchers involved should be offered specific research ethics training. To meet this need in CARESSES—an international multidisciplinary project that aims to design and evaluate the first culturally competent SAR for the care of older adults—a research ethics training module for the project researchers was developed. The training module is largely based on case-based learning (CBL), a widely recognized approach to learning and instruction that is regarded as highly effective across multiple disciplines. In this paper, we argue that research ethics training should be offered to robotics investigators involved in research on SARs in elder care, and we provide an overview of the ethical issues involved in conducting research with SARs and older adults in care homes. Finally, we show how CBL can be used for research ethics training in this context.

Background This article describes the design of an intervention study that focuses on whether and to what degree culturally competent social robots can improve health and well-being related outcomes among older adults residing long-term care homes. The trial forms the final stage of the international, multidisciplinary CARESSES project aimed at designing, developing and evaluating culturally competent robots that can assist older people according to the culture of the individual they are supporting. The importance of cultural competence has been demonstrated in previous nursing literature to be key towards improving health outcomes among patients. Method This study employed a mixed-method, single-blind, parallel-group controlled before-and-after experimental trial design that took place in England and Japan. It aimed to recruit 45 residents of long-term care homes aged ≥65 years, possess sufficient cognitive and physical health and who self-identify with the English, Indian or Japanese culture (n = 15 each). Participants were allocated to either the experimental group, control group 1 or control group 2 (all n = 15). Those allocated to the experimental group or control group 1 received a Pepper robot programmed with the CARESSES culturally competent artificial intelligence (experimental group) or a limited version of this software (control group 1) for 18 h across 2 weeks. Participants in control group 2 did not receive a robot and continued to receive care as usual. Participants could also nominate their informal carer(s) to participate. Quantitative data collection occurred at baseline, after 1 week of use, and after 2 weeks of use with the latter time-point also including qualitative semi-structured interviews that explored their experience and perceptions further. Quantitative outcomes of interest included perceptions of robotic cultural competence, health-related quality of life, loneliness, user satisfaction, attitudes towards robots and caregiver burden. Discussion This trial adds to the current preliminary and limited pool of evidence regarding the benefits of socially assistive robots for older adults which to date indicates considerable potential for improving outcomes. It is the first to assess whether and to what extent cultural competence carries importance in generating improvements to well-being.

ECHONET Lite is a leading protocol for controlling devices in Japan smart homes. However, it lacks interoperability with service platforms that provide ambient assisted living (AAL) services to residents which are actively researched in order to deal with the population aging. This research proposes an adaptation layer for ECHONET Lite protocol which provides the semantic interoperability based on ontology. In order to verify the proposed solution, a service gateway based on the proposed architecture was implemented to integrate ECHONET Lite protocol into the universAAL platform, a leading AAL platform in Europe.

The Culture Aware Robots and Environmental Sensor Systems for Elderly Support (CARESSES) project is introduced in this paper. In the CARESSES project, a set of experiments is designed to systematically test the user experience, in which the experiment involves a user and a caregiver or a CARESSES robot inside the smart home environment, i.e., iHouse facility. The experiment results reveal that the system integration of the CARESSES robot and the smart home environment can improve the overall user experience.

In this work we investigate the problem of multi-robot cooperative localization in dynamic environments. Specifically, we propose an approach where wheeled robots are localized using the monocular camera embedded in the head of a Pepper humanoid robot, to the end of minimizing deviations from their paths and avoiding each other during navigation tasks. Indeed, position estimation requires obtaining a linear relationship between points in the image and points in the world frame: to this end, an Inverse Perspective mapping (IPM) approach has been adopted to transform the acquired image into a bird eye view of the environment. The scenario is made more complex by the fact that Pepper’s head is moving dynamically while tracking the wheeled robots, which requires to consider a different IPM transformation matrix whenever the attitude (Pitch and Yaw) of the camera changes. Finally, the IPM position estimate returned by Pepper is merged with the estimate returned by the odometry of the wheeled robots through an Extened Kalman Filter. Experiments are shown with multiple robots moving along different paths in a shared space, by avoiding each other without onboard sensors, i.e., by relying only on mutual positioning information.

Objectives Socially assistive humanoid robots are considered a promising technology to tackle the challenges in health and social care posed by the growth of the ageing population. The purpose of our study was to explore the current evidence on barriers and enablers for the implementation of humanoid robots in health and social care. Design Systematic review of studies entailing hands-on interactions with a humanoid robot. Setting From April 2018 to June 2018, databases were searched using a combination of the same search terms for articles published during the last decade. Data collection was conducted by using the Rayyan software, a standardised predefined grid, and a risk of bias and a quality assessment tool. Participants Post-experimental data were collected and analysed for a total of 420 participants. Participants comprised: older adults (n=307) aged ≥60 years, with no or some degree of age-related cognitive impairment, residing either in residential care facilities or at their home; care home staff (n=106); and informal caregivers (n=7). Primary outcomes Identification of enablers and barriers to the implementation of socially assistive humanoid robots in health and social care, and consequent insights and impact. Future developments to inform further research. Results Twelve studies met the eligibility criteria and were included. None of the selected studies had an experimental design; hence overall quality was low, with high risks of biases. Several studies had no comparator, no baseline, small samples, and self-reported measures only. Within this limited evidence base, the enablers found were enjoyment, usability, personalisation and familiarisation. Barriers were related to technical problems, to the robots’ limited capabilities and the negative preconceptions towards the use of robots in healthcare. Factors which produced mixed results were the robot’s human-like attributes, previous experience with technology and views of formal and informal carers. Conclusions The available evidence related to implementation factors of socially assistive humanoid robots for older adults is limited, mainly focusing on aspects at individual level, and exploring acceptance of this technology. Investigation of elements linked to the environment, organisation, societal and cultural milieu, policy and legal framework is necessary. PROSPERO registration number CRD42018092866.

The rapidly increasing number of elderly people has led to the development of in-home assistive robots for assisting and monitoring elderly people in their daily life. To these ends, indoor scene and human activity recognition is fundamental. However, image processing is an expensive process, in computational, energy, storage and pricing terms, which can be problematic for consumer robots. For this reason, we propose the use of computer vision cloud services and a Naive Bayes model to perform indoor scene and human daily activity recognition. We implement the developed method on the telepresence robot Double to make it autonomously find and approach the person in the environment as well as detect the performed activity.

In many cases, complex multidisciplinary research projects may show a lack of coordinated development and integration, and a big effort is often required in the final phase of the projects in order to merge software developed by heterogeneous research groups. This is particularly true in advanced robotic projects: the objective here is to deliver a system that integrates all the hardware and software components, is capable of autonomous behaviour, and needs to be deployed in real-world scenarios toward providing an impact on future research and, ultimately, on society. On the other hand, in recent years there has been a growing interest for techniques related to software integration, but these have been mostly applied to the IT commercial domain. This paper presents the work performed in the context of the project CARESSES, a multidisciplinary research project focusing on socially assistive robotics that involves 9 partners from the EU and Japan. Given the complexity of the project, a huge importance has been placed on software integration, task planning and architecture definition since the first stages of the work: to this aim, some of the practices commonly used in the commercial domain for software integration, such as merging software from the early stage, have been applied. As a case study, the document describes the steps which have been followed in the first year of the project discussing strengths and weaknesses of this approach.

Culture, intended as the set of beliefs, values, ideas, language, norms and customs which compose a person’s life, is an essential element to know by any robot for personal assistance. Culture, intended as that person’s background, can be an invaluable source of information to drive and speed up the process of discovering and adapting to the person’s habits, preferences and needs. This article discusses the requirements posed by cultural competence on the knowledge management system of a robot. We propose a framework for cultural knowledge representation that relies on (i) a three-layer ontology for storing concepts of relevance, culture-specific information and statistics, person-specific information and preferences; (ii) an algorithm for the acquisition of person-specific knowledge, which uses culture-specific knowledge to drive the search; (iii) a Bayesian Network for speeding up the adaptation to the person by propagating the effects of acquiring one specific information onto interconnected concepts. We have conducted a preliminary evaluation of the framework involving 159 Italian and German volunteers and considering 122 among habits, attitudes and social norms.

Purpose: The aim of the present review is to explore the influence of culture on attitudes towards humanoid and animal-like robots. Design: An integrative review of current evidence. Methods: Medline, CINHAL, PsycInfo, PubMed and Google Scholar were searched from 2000 to 2017. A total of 22 articles met the inclusion criteria, were retrieved and analysed. Findings: Culture influences attitudes and preferences towards robots but due to the limitations of the reviewed studies concrete conclusions cannot be made. More consistent evidence was found in regard to the influence of culture on non-verbal behaviours and communication styles with people being more accepting of a robot that behaved ‘closer’ to their own culture. Conclusions: The research field of human-robot interaction provides the current evidence on the influence that culture has on attitudes towards humanoid and animal-like robots but more research which is guided by strong theoretical frameworks is needed. Clinical relevance: With the increased use of humanoid robots in the healthcare system it is imperative that nurses and other healthcare professionals explore and understand the different factors that can affect the use of robots with patients. Keywords: Culture, Cultural Background, Robots, Attitudes, Humanoid Robot, Animal Robot

Background: Robots are introduced in many health and social care settings. Objectives: To provide an overview of the existing evidence related to the views of nurses and other health and social care workers about the use of assistive humanoid and animal-like robots. Methods: Using the Joanna Briggs Institute guidelines we searched MEDLINE, PUBMED, CINHAL, EMBASE, PsycInfo, Web of Science, and IEEE Xplore digital library. Nineteen (19) articles met the criteria for inclusion. Results: Health care workers reported mixed views regarding the use of robots. They considered an array of tasks that robots could perform; they addressed the issue of patient safety and raised concerns about privacy. Conclusions: A limited number of studies have explored the views of health care workers about the use of robots. Considering the fast pace with which technology is advancing in the care field, it is critical to conduct more research in this area. Impact Statement: Robots will increasingly have a role to play in nursing, health and social care. The potential impact will be challenging for the healthcare workforce. It is therefore important for nurses and other health and social care workers to engage in discussion regarding the contribution of robots and their impact not only on nursing care but also on future roles of health and social care workers.

Robots, along with sensors and telemedicine, have been identified as technologies that can assist and prolong independent living for older people, with robots especially being used to help prevent social isolation and depression.

The article describes a system for culture-aware human-robot verbal interaction, that constitutes the basis for designing culturally-competent robots for health-care, i.e., robots able to autonomously re–configure their way of acting and speaking, when offering a service, to match the culture, customs, and etiquette of the person they are asstisting. The article shows how culture-aware verbal interaction is tightly related to cultural knowledge representation and acquisition, by describing the methodological and technological solutions adopted, and showing in details one of the preliminary experiments performed to design a culturally-competent robot.

This work presents a system is able to support the users on their daily livings using an activity and intention recognition method. The system is designed to be focused on the applicability, working in real time. The recognition method uses the concept of activity frame, which is defined as a set of sequenced environmental observations containing meaningful information (such as objects' locations, sensors' activation, etc) related to the recognition of activities and tasks accomplished in one location. Analyzing the specific frame, it is possible to relate, through a set of conditions, the observed states to a specific activity or intention. By analyzing the frequency of those activities and intentions occurrences, it is possible to identify unusual behavior and guide an smart interactive device, such as robot, to support the user. The proposed recognition method was tested with the data provided by an smart home project, and the recognition rate for the proposed method has high accuracy, based on other similar ones. The information of activities intentions can provide meaningful guidelines for the robot.

Culture-Aware Elderly Care Robots in a Smart ICT Environment Rapid demographic change constitutes an unprecedented societal challenge for Japan. I will shed light on the issues of Japan’s super-aging society and introduce the Human-Robot Interaction work package of our new EC Horizon 2020 project “CARESSES”, aiming at developing culturally competent elderly care robots, jointly commissioned by the Ministry of Internal Affairs and Communications of Japan. We envision a future in an aging society, where care robots are able to interact with the elderly with different cultural and personality traits through personalized emotion generation and (facial/vocal/body) expression. I will share some of our preliminary results of multi-modal human-robot interaction with the Korean Healthcare Robotics and Medical Informatics Community and explore opportunities for future collaboration. Furthermore, I will introduce a smart ICT environment testbed iHouse, and a user speech activated interface to enable care robots to gain access to the iHouse resources and provide data through verbal interaction with the user. I am hoping to discuss the technical feasibility of a robotic smart care home interface toward supporting independent living of the elderly.

Paving the way to culturally competent robots: the CARESSES project, Webinar organized by the EU-Japan Centre for Industrial Cooperation. The concepts expressed in this presentation have been contributed by all CARESSES partners, and in particular: Prof. Irena Papadopoulos (Middlesex University), Prof. Hiroko Kamide (Nagoya University), Prof. Alessandro Saffiotti (Orebro University), Prof. Jaeryoung Lee (Chubu University), Dr. Amit Kumar Pandey (SoftBank Robotics), Dr. Sanjeev Kanoria (Advinia HealthCare), Dr. Chris Papadopoulos (University of Bedfordshire)

Invited speech to INCAROS (International Care Robot Symposium) 2017 15th November 2017, Seoul, Korea. IMPORTANT: the presentation does not contain any text, since it is meant to be presented by a speaker. Download it for a better experience.

Description of the Special Session ”Cultural factors in human-robot interactions” held at RO-MAN 2017

Cultural competence is a well known requirement for an effective healthcare, widely investigated in the nursing literature. We claim that personal assistive robots should likewise be culturally competent, aware of general cultural characteristics and of the different forms they take in different individuals, and sensitive to cultural differences while perceiving, reasoning, and acting. Drawing inspiration from existing guidelines for culturally competent healthcare and the state-of-the-art in culturally competent robotics, we identify the key robot capabilities which enable culturally competent behaviours and discuss methodologies for their development and evaluation.

The nursing literature shows that cultural competence is an important requirement for effective healthcare. We claim that personal assistive robots should likewise be culturally competent, that is, they should be aware of general cultural characteristics and of the different forms they take in different individuals, and take these into account while perceiving, reasoning, and acting. The CARESSES project is an Europe-Japan collaborative effort that aims at designing, developing and evaluating culturally competent assistive robots. These robots will be able to adapt the way they behave, speak and interact to the cultural identity of the person they assist. This paper describes the approach taken in the CARESSES project, its initial steps, and its future plans.

Socially assistive robots are currently used in many settings and in healthcare. In this article, we introduce the field of Transcultural Robotic Nursing and a new innovative research project aimed to develop culturally-aware robots.

The nursing literature shows that cultural competence is an important requirement for effective healthcare. We claim that personal assistive robots should likewise be culturally competent, that is, they should be aware of general cultural characteristics and of the different forms they take in different individuals, and take these into account while perceiving, reasoning, and acting. The CARESSES project is an Europe-Japan collaborative effort that aims at designing, developing and evaluating culturally competent assistive robots. These robots will be able to adapt the way they behave, speak and interact to the cultural identity of the person they assist. This paper describes the approach taken in the CARESSES project, its initial steps, and its future plans.

Cultural adaptation, i.e., the matching of a robot's behaviours to the cultural norms and preferences of its user, is a well known key requirement for the success of any assistive application. However, culture-dependent robot behaviours are often implicitly set by designers, thus not allowing for an easy and automatic adaptation to different cultures. This paper presents a method for the design of culture-aware robots, that can automatically adapt their behaviour to conform to a given culture. We propose a mapping from cultural factors to related parameters of robot behaviours which relies on linguistic variables to encode heterogeneous cultural factors in a uniform formalism, and on fuzzy rules to encode qualitative relations among multiple variables. We illustrate the approach in two practical case studies.

Daily life activities, such as eating and sleeping, are deeply influenced by a person's culture, hence generating differences in the way a same activity is performed by individuals belonging to different cultures. We argue that taking cultural information into account can improve the performance of systems for the automated recognition of human activities. We propose four different solutions to the problem and present a system which uses a Naive Bayes model to associate cultural information with semantic information extracted from still images. Preliminary experiments with a dataset of images of individuals lying on the floor, sleeping on a futon and sleeping on a bed suggest that: i) solutions explicitly taking cultural information into account are more accurate than culture-unaware solutions; and ii) the proposed system is a promising starting point for the development of culture-aware Human Activity Recognition methods.

Fact Sheet of the EU-JP H2020 Project CARESSES