Conference Paper

Human-Robots Cooperation through Intelligent Garment

Authors:
  • Ecole Nationale Supérieure des Arts et Industries Textiles
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Robots will face a huge deployment in a very near future for people’s everyday lives. Main objective is to reduce or eliminate safety and health risks inherent to physically demanding duty or risky work. The present paper focuses on unsafe and hazardous tasks performed by firefighters and aims at proposing organizational and technical tools to balance risk with the involvement of cooperative robots. A study has been conducted within the framework of the regional project SUCRé, dealing with Human-Robots Cooperation in hostile/severe environment. After a short presentation of the project, a Human-Centered Design approach is proposed, based on the implementation of rules defined by the Human-Machine Cooperation principles. Two human aspects have been addressed to design and control cooperation among several firefighters and robots: the cognitive and physiological aspects. A use case presents the implementation of such aspects to manage global and local cooperation.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Table 1 presents a summary of the current development of smart clothing in various sectors, including smart health, vulnerable groups (monitoring of babies and the elderly), sports and wellness, industry, defence and public safety, and interaction with the environment. physical stress levels, breathing patterns, sweat, temperature, energy, burnt rate [10] internet-of-smart-clothing ECG signals sleep disorders [11] T-shirt ECG signals heart rate [12] a smart clothing prototype ECG monitoring human respiration [13] a smart garment ECGs respiration rate [14] Smart Garment triaxle accelerometer and gyroscope rehabilitation for assisting osteoarthritis (OA) patients [15] a Bluetooth-enabled smart garment acromion sensor rehabilitating shoulders [16] vivo metrics life Shirt accelerometers pulmonary, cardiac, and other vital signs measurement [17] sensatex smartShirt ECG respiration rate and blood pressure [18] Baby and Elderly Monitoring a smart garment ECG signal, textile electrode new-born monitoring [19] baby vest respiratory sensors, electrodes for ECG, moisture and temperature sensors respiration, heart rate, temperature and humidity, excessive sweating [20] wearable textile magnetic reed sensors accident prevention for babies in crawl phase [21] baby night watch ECG, 3D accelerometer, inertial sensor infant's body temperature, heart and breathing rates and body position [22] T-shirt knitted sensor heart attack or a stroke for elderly people [23] garment for older walkers shimmer sensor (ECG and accelerometer data) user's heart rate [24] Sports and Wellness smart shirt body-worn inertial sensors (acceleration sensors) monitoring posture [25] wearable Instrumented vest accelerometer sensor monitoring elderly people's postures [26] [32] wearable computers and smart clothing textrodes, motion sensors thermal work limits, alertness and fitness for duty status, musculoskeletal fatigue limits, neuropsychological and physiological conditions [33,34] a smart slim fit T-shirt accelerometer sensor, ECG heart rate, heart rate variability (HRV) [35] a firefighting garment temperature sensor, heart-beat sensor, accelerometer sensor heart rate, activity, body temperature [36] a smart T-shirt, a jacket and a pair of boots textile motion sensor, CO sensor heart and breathing rates, body temperature, blood oxygen saturation, position, activity, and posture [37] Interaction with Environment a smart wristband a 3-axis accelerometer (ADXL335), Lilypad Arduino based microcontroller gesture processing to control lights [38] interactive clothes NFC tags and IOT to identify cheap replicas, track loyal customers, minimize the wastage of resources track of the goods using [39] intelligent T-shirt accelerometers Lilypad and 3D accelerometer heart rate [40] ...
Article
Full-text available
Wearable technologies are making a significant impact on people’s way of living thanks to the advancements in mobile communication, internet of things (IoT), big data and artificial intelligence. Conventional wearable technologies present many challenges for the continuous monitoring of human health conditions due to their lack of flexibility and bulkiness in size. Recent development in e-textiles and the smart integration of miniature electronic devices into textiles have led to the emergence of smart clothing systems for remote health monitoring. A novel comprehensive framework of smart clothing systems for health monitoring is proposed in this paper. This framework provides design specifications, suitable sensors and textile materials for smart clothing (e.g., leggings) development. In addition, the proposed framework identifies techniques for empowering the seamless integration of sensors into textiles and suggests a development strategy for health diagnosis and prognosis through data collection, data processing and decision making. The conceptual technical specification of smart clothing is also formulated and presented. The detailed development of this framework is presented in this paper with selected examples. The key challenges in popularizing smart clothing and opportunities of future development in diverse application areas such as healthcare, sports and athletics and fashion are discussed.
... Other authors suggested that such an interaction with IoT systems may be automated by incorporating NFC tags into garments [227]. Furthermore, some researchers have also proposed to interact with robots through intelligent garments [228]. ...
Article
Full-text available
Technology has become ubiquitous, it is all around us and is becoming part of us. Together with the rise of the Internet of Things (IoT) paradigm and enabling technologies (e.g., Augmented Reality (AR), Cyber-Physical Systems, Artificial Intelligence (AI), blockchain or edge computing), smart wearables and IoT-based garments can potentially have a lot of influence by harmonizing functionality and the delight created by fashion. Thus, smart clothes look for a balance among fashion, engineering, interaction, user experience, cybersecurity, design and science to reinvent technologies that can anticipate needs and desires. Nowadays, the rapid convergence of textile and electronics is enabling the seamless and massive integration of sensors into textiles and the development of conductive yarn. The potential of smart fabrics, which can communicate with smartphones to process biometric information such as heart rate, temperature, breathing, stress, movement, acceleration, or even hormone levels, promises a new era for retail. This article reviews the main requirements for developing smart IoT-enabled garments and shows smart clothing potential impact on business models in the medium-term. Specifically, a global IoT architecture is proposed, the main types and components of smart IoT wearables and garments are presented, their main requirements are analyzed and some of the most recent smart clothing applications are studied. In this way, this article reviews the past and present of smart garments in order to provide guidelines for the future developers of a network where garments will be connected like other IoT objects: the Internet-of-Smart-Clothing.
... In this study, three types of agents take place in the scenario: a human supervisor (equipped with EEG and a brain computer interface BCI), operators (equipped with smart garment) and robots [29]. At the operational level, robots and operators have to be cooperative with each other and with the human supervisor in order to achieve a given common goal (solving the crisis or limiting its impact). ...
... A great advantage of using this method for the analysis is that a prior alignment of the garment is not necessary, since no matter the location within the image, the garment contributes the same number of pixels. For this to be true care must be taken that the distance from the sensor to the garments is the same, otherwise, the objects in the image appear different sizes, changing the histogram [10]. ...
Article
Full-text available
A computer vision system is implemented to detect errors in the cutting stage within the manufacturing process of garments in the textile industry. It provides solution to errors within the process that cannot be easily detected by any employee, in addition to significantly increase the speed of quality review. In the textile industry as in many others, quality control is required in manufactured products and this has been carried out manually by means of visual inspection by employees over the years. For this reason, the objective of this project is to design a quality control system using computer vision to identify errors in the cutting stage within the garment manufacturing process to increase the productivity of textile processes by reducing costs.
Thesis
Nous avons constaté un intérêt considérable pour la robotique de service au fil des années. Les robots sont de plus en plus utilisés pour rendre service aux humains dans différents domaines d’application. Néanmoins, compte tenu de leur capacité limitée, les robots ne peuvent pas répondre à toutes les attentes des humains et aux exigences de l’environnement dynamique. Les robots et les humains doivent donc travailler en équipe de façon complémentaire pour bénéficier des compétences de chacun tout en palliant leurs faiblesses. Dans cette recherche, nous appliquons les principes de la coopération homme-machine pour définir des niveaux d’automatisation adaptables permettant d’assurer une coopération homme-robots. L’humain et le robot sont représentés sous forme d’un même modèle cognitif caractérisant leurs compétences et capacités respectives à réaliser une tâche mais aussi à coopérer. Les interactions possibles entre l’humain et le robot sont identifiées à plusieurs niveaux d’abstraction de réalisation de la tâche. Les questions de l’allocation de tâches et de fonctions, ainsi que de gestion d’autorité sont identifiées et utilisées pour répondre aux besoins de la coopération. Outre les aspects théoriques, nous avons implémenté des systèmes homme-robots avec différents niveaux d’automatisation dans deux contextes: un système composé d’un humain et de deux robots mobiles destiné à la gestion de crise et un système composé d’un humain, d’un robot mobile et d’un système de contrôle via les « pensées » de l’humain dans le cadre du handicap. Des études expérimentales sont menées avec ces deux systèmes. Ces études ont permis de valider les systèmes proposés, puis d’évaluer l’adaptation des niveaux d’automatisation pour soutenir la coopération.
Chapter
This chapter presents an intelligent wearable system based on the design of an instrumented garment specially designed for firefighters and focuses on the context of crisis management. A powerful application of intelligent garments is related to risk management in hostile environments. By learning from data measured by the garment's sensors, a local decision support system was created in order to relate fatigue and stress indicators to the measured physiological signals. The chapter outlines the design process of the intelligent garment for firefighters, including the general architecture of the wearable system, choice of sensors and microcontroller, textile design and integration of the electronic components. It discusses signal processing methods used for extracting information about fatigue and stress conditions and provides an analysis of the results obtained. The chapter proposes a cooperation plan between a robot and a firefighter wearing the smart garment.
Article
Full-text available
This paper presents experimental results of cooperative guidance laws embedded on Lego Mindstorms NXT mobile robots for two types of missions. The first one is navigation to a waypoint as a fleet with collision and obstacle avoidance, following a model predictive control (MPC) framework. The second one is source localization, i.e., finding the maximum of a potential field, for which a distributed estimation and control strategy is proposed. Experiments show the ability to perform the two missions on these basic mobile robots, in spite of their limited computational resources. In particular, the search for the optimal control sequence through a dedicated discretization of the command space makes it possible to implement real-time MPC.
Article
Full-text available
Context aware or event based system requires location information of the event of occurrence. Sensor positioning and its location information are mandatory if a data sensed by the wireless sensor network (WSN) has to be meaningful. In cases like routing of packets of data in WSN, the dead sensor nodes should be avoided in the route. For this the location of live nodes is needed, so that the shortest and reliable routing is possible. Moreover some data require inherent location detection like forest fire detection or movement of miners in mines. So localization with better performance, with more reliability, with less computation complexity and lesser cost is much needed. This paper shows an integrated Xbee arduino and differential evolution approach for localization in wireless sensor networks, and the design of differential evolution localization algorithm for minimization of localization error. The algorithm is simple to implement and uses less control variables.
Conference Paper
Full-text available
Especially in life-critical systems decision-making entails cognitive functions such as monitoring, as well as fault prevention and recovery. People involved in the control and management of such systems play two kinds of roles: positive thanks to their unique involvement and capacity to deal with the unexpected; and negative with their ability to make errors. But they are also able to detect and correct these mistakes and able to learn from them. Thus human-machine system designer can allow the humans an innovative behavior to be “aware” and to cope with unknown situations by enhancing Situation Awareness (SA). As humans are more and more involved in collective works the constructs of team-SA are important. But the literature shows a great variety and some incoherence in their definitions. That makes difficult to build a design methodology favoring human SA. In parallel, human machine cooperation models have been developed in the last two decades and validated in different dynamic application fields: Air Traffic Control, fighter aircraft cockpit, reconnaissance robot. These studies showed an increase of the problem solving capabilities and a decrease of workload when the tasks are performed by cooperative teams. In this paper we first synthesize main team-SA constructs, we then present principles of humans-machines cooperation and present a Common Work Space as a medium that allow cooperation. We propose to extend it in order to enrich team-SA constructs.
Article
Full-text available
The autonomic nervous system dynamically controls the response of the body to a range of external and internal stimuli, providing physiological stability in the individual. With the progress of information technology, it is now possible to explore the functioning of this system reliably and non-invasively using comprehensive and functional analysis of heart rate variability. This method is already an established tool in cardiology research, and is increasingly being used for a range of clinical applications. This review describes the theoretical basis and practical applications for this emerging technique.
Article
World becomes more and more densely populated, dynamic and interconnected. In order to support the increase of world demand, automation provides systems to improve efficiency, safety and comfort. But systems are often placed at human operators’ disposal or imposed on them without taking enough care. Ergonomic criteria, training and integration of future users into the design loop define some principals to provide a minimum of coherence in the future user activity. But more precautions must be taken to face the increase of the complexity of human organizations, systems and their interconnections, defining Life Critical Systems. Stability of these systems is more and more precarious and human could have to deal with crisis. The objective of the chapter is to partially respond to this problem by anticipating the occurrence of lacks of coherence in the interaction between humans and systems. Main foundations had been laid down by Human-Machine Cooperation principles with a focus on Dynamic Task Allocation issues (Millot, 1997), (Pacaux-Lemoine, 2002). But these principles usually concern task sharing between one or two human operators and one system. Both human operator and system are expressed according to their know-how and their know-how-to-cooperate. The know-how allows human or technical agent to supervise and control process. The know-how-to-cooperate allows agent to take into account other agents’ activities by building model of them and by developing code of communication. These definitions have been extended in order to model more kinds of interaction between more than two agents. New extensions also try to involve more precisely notions such as responsibility, authority, trust and willingness, and to benefit from levels of automation and adaptive automation researches. Some samples will be presented in order to illustrate this approach in Air Traffic Control, Fighter Aircraft Piloting, Robotics and Car driving.
Conference Paper
The objective of the paper is to present an approach used in order to define task sharing between human operator and robot, when new functions have to be defined for human operator and new autonomous functions are available for robot. A study of complementarities between levels of automation and human machine cooperation approaches is firstly proposed, and leads to the definition of three main axes necessary to design and evaluate human-machine systems and cooperation. An example of use of these definitions is then proposed to study of cooperation between a human operator and an unmanned ground vehicle with various levels of autonomy.
Article
An experiment is described that aimed at evaluating a principle of dynamic task allocation (DTA) of conflict resolution between aircraft in air-traffic control on a large scale simulator. It included three cognitive agents: the radar controller (RC), in charge of conflict detection and resolution; the planning controller (PC), in charge of entry-exit coordination and of workload regulation; and a conflict resolution computer system (SAINTEX), able to manage only simple conflicts. Within this three-agent paradigm, three conditions were compared: (a) a control condition (without computer assistance); (b) an explicit condition (PC and RC in charge of the allocation); and (c) an assisted explicit condition (SAINTEX proposed allocations which could be changed by PC). Comparisons were made on the basis of a detailed cognitive analysis of verbal protocols. The more the assistance, the more anticipative the mode of operation in controllers and the easier the human-human cooperation (HHC). These positive effects of the computer support are interpreted in terms of decreased workload and increased shared information space. In addition, the more the controllers felt responsible for task allocation, the more they criticized the machine operation.
Article
The introduction of information technology based on digital computers for the design of man-machine interface systems has led to a requirement for consistent models of human performance in routine task environments and during unfamiliar task conditions. A discussion is presented of the requirement for different types of models for representing performance at the skill-, rule-, and knowledge-based levels, together with a review of the different levels in terms of signals, signs, and symbols. Particular attention is paid to the different possible ways of representing system properties which underlie knowledge-based performance and which can be characterised at several levels of abstraction-from the representation of physical form, through functional representation, to representation in terms of intention or purpose. Furthermore, the role of qualitative and quantitative models in the design and evaluation of interface systems is mentioned, and the need to consider such distinctions carefully is discussed.
Conference Paper
This paper presents the study of a particular form of human-machine cooperation in order to define and evaluate a system which is able to control the intervehicle distance according to the environment, the driver's behaviors and his/her wish to cooperate with the system. This study highlights the ACC's limits to control the distance headway with the front vehicle in some traffic situations on motorway and A-class road and tries to identify the driver's needs in order to take them into account to design an auto-adaptive cruise control prototype.
Article
This paper deals with the research of means to build a human–machine cooperation in the air traffic control. The experiment described aims at evaluating a principle of dynamic allocation of conflict resolution on a large scale simulator of air traffic control. Artificial and human agents have to cooperate to exchange information, to make a decision and to control the air traffic. To analyse and define cooperation between agents, the know-how and the know-how-to-cooperate are defined. These concepts give a structure which allows to build a common work space to support the cooperation more efficiently.
Article
The use of spectral techniques to quantify short term heart rate fluctuations on the order of seconds to minutes has helped define the autonomic contributions to beat-to-beat control of heart rate. We used similar techniques to quantify the entire spectrum (0.00003-1.0 Hz) of heart rate variability during 24 hour ambulatory ECG monitoring. The ECG from standard Holter monitor recordings from normal subjects was sampled with the use of a phase locked loop, and a heart rate time series was constructed at 3 Hz. Frequency analysis of the heart rate signal was performed after a nonlinear filtering algorithm was used to eliminate artifacts. A power spectrum of the entire 24 hour record revealed power that was inversely proportional to frequency, 1/f, over 4 decades from 0.00003 to 0.1 Hz (period approximately 10 hours to 10 seconds). Displaying consecutive spectra calculated at 5 minute intervals revealed marked variability in the peaks at all frequencies throughout the 24 hours, probably accounting for the lack of distinct peaks in the spectra of the entire records.
Article
Certain abnormalities of autonomic function in the setting of structural cardiovascular disease have been associated with an adverse prognosis. Various markers of autonomic activity have received increased attention as methods for identifying patients at risk for sudden death. Both the sympathetic and the parasympathetic limbs can be characterized by tonic levels of activity, which are modulated by, and respond reflexively to, physiological changes. Heart rate provides an index of the net effects of autonomic tone on the sinus node, and carries prognostic significance. Heart rate variability, though related to heart rate, assesses modulation of autonomic control of heart rate and carries additional prognostic information, which in some cases is more powerful than heart rate alone. Heart rate recovery after exercise represents the changes in autonomic tone that occur immediately after cessation of exercise. This index has also been shown to have prognostic significance. Autonomic evaluation during exercise and recovery may be important prognostically, because these are high-risk periods for sudden death, and the autonomic changes that occur with exercise could modulate this high risk. These markers provide related, but not redundant information about different aspects of autonomic effects on the sinus node.
A common work space to support the cooperation into the cockpit of a two-seater fighter aircraft
  • M P Pacaux-Lemoine
  • A Loiselet
M. P. Pacaux-Lemoine and A. Loiselet, -A common work space to support the cooperation into the cockpit of a two-seater fighter aircraft,‖ Coop. Syst. Des. a Chall. Mobil. Age, pp. 157-172, 2002.
Levels of automation and human-machine cooperation: Application to human-robot interaction
  • M P Pacaux
  • S Debernard
  • A Godin
  • B Rajaonah
  • F Anceaux
  • F Vanderhaegen
M. P. Pacaux, S. Debernard, A. Godin, B. Rajaonah, F. Anceaux, and F. Vanderhaegen, -Levels of automation and human-machine cooperation: Application to human-robot interaction,‖ IFAC Proc. Vol., vol. 18, no. PART 1, pp. 6484-6492, 2011.
  • J Rasmussen
J. Rasmussen, -Skills, Rules, and Knowledge; signals, Signs, and Symbols, and Other Distinctions in Human Performance Models,‖ IEEE Trans. Syst. Man. Cybern., vol. 13, no. 3, pp. 257-266, 1983.
Task Force of The European Society of Cardiology and The North American Society of Pacing and Electrophysiology, -Heart rate variability: Standards of measurement, physiological interpretation, and clinical use,‖ Eur
Task Force of The European Society of Cardiology and The North American Society of Pacing and Electrophysiology, -Heart rate variability: Standards of measurement, physiological interpretation, and clinical use,‖ Eur. Heart J., vol. 17, pp. 354-381, 1996.
  • J Marzat
  • H Piet-Lahanier
  • A Kahn
J. Marzat, H. Piet-lahanier, and A. Kahn, -Cooperative guidance of Lego Mindstorms NXT mobile robots,‖ 11th Int. Conf. Informatics Control. Autom. Robot., vol. 2, no. Vienne, Austria, pp. 605-610, 2014.
A common work space for a mutual enrichment of human-machine cooperation and teamsituation awareness
  • P Millot
  • M P Pacaux-Lemoine
P. Millot and M. P. Pacaux-Lemoine, -A common work space for a mutual enrichment of human-machine cooperation and teamsituation awareness,‖ IFAC Proc. Vol., vol. 12, no. PART 1, pp. 387-394, 2013.