No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Team communication processing and process analytics for supporting robot-assisted emergency response
... On the one hand, robotic assistance increases the complexity, on the other hand, robots make more information available in digital Fig. 5: The A-DRZ teamwork support System Architecture form. The A-DRZ teamwork support functionality is a layer on top of the core robotic systems, which helps to handle the mission complexity and abundance of information by collecting information about an ongoing robot-assisted emergency response mission from different sources, including both human and robotic members of the first response team, integrating it and using it to facilitate situation awareness and provide real-time mission process assistance [26]. In this subsection, we explain the overall system architecture of the teamwork support layer, including the providers, types, and flow of data between the individual components. ...
... MPA matches the executed activities to the reference model to identify and visualize the currently executed processes, including both those activities that were already executed and those activities that should be executed next. This way, mission commanders can see the current situation and plan ahead for the near future [26]. ...
To meet the challenges involved in providing adequate robotic support to first responders, a holistic approach is needed. This requires close cooperation of first responders, researchers and companies for scenario-based needs analysis, iterative development of the corresponding system functionality and integrated robotic systems as well as human-robot teamwork support, and experimentation, system testing and evaluation in realistic missions carried out with or by first responders. We describe how such a holistic approach is implemented by the partners in the cooperative project A-DRZ for the establishment of the German Rescue Robotics Center (DRZ). The A-DRZ approach addresses important requirements identified by first responders: adaptation of operational capabilities of robotic platforms; robust network connectivity; autonomous assistance functions facilitating robot control; improving situation awareness for strategic and tactical mission planning; integration of human-robot teams in the first responders' mission command structure. Solutions resulting from these efforts are tested and evaluated in excercises utilizing the advanced capabilities at the DRZ Living Lab and in external deployments.
... Erfolgten Abweichungen aus guten Gründen, kann das Referenzmodell angepasst und verbessert werden. Im Folgenden wird nun ein weiteres Anwendungsszenario des Referenzmodells als Basis eines Prozessassistenzsystems detailliert dargestellt, dessen grundlegende Idee in [25] bereits kurz beschrieben wurde. Während eines großen Feuerwehreinsatzes ist es die Aufgabe der Führungskräfte, zahlreiche parallel ablaufende Aktivitäten zu überwachen und zu koordinieren, was eine große kognitive Belastung darstellen kann. ...
... 5) listet übersichtlich aktuelle Aktivitäten und nächste offene Schritte auf. [25]. So kann in Echtzeit festgestellt werden, welche Einsatzkraft welche Aktivität ausführt. ...
... In diesem Zusammenhang wird auch das Thema Mensch-Roboter-Teamarbeit behandelt. Ziel ist dabei die Entwicklung eines Systems, welches Methoden der natürlichen Sprachverarbeitung, der Referenzmodellierung und des Process Mining verwendet, um aus der Funkkommunikation der Einsatzkräfte und den Sensordaten der Robotersysteme den Ablauf und aktuellen Status eines Einsatzes zu rekonstruieren [8]. Die so gewonnenen Daten werden genutzt, um den Einsatzkräften während und nach dem Einsatz verschiedene Assistenzfunktionen zur Verfügung zu stellen [9]. ...
... Die Autoren des vorliegenden Beitrags haben im Kontext Projektes A-DRZ an einem technischen Konzept mitgearbeitet, um Prozessabläufe aus der Funkkommunikation der Feuerwehr abzuleiten und diese durch Methoden der Prozessdatenanalyse (Process Mining) aufzubereiten [8]. Dabei stand zunächst die Unterstützung während eines Einsatzes im Vordergrund (Stichwort: Prozessassistenz [9]). ...
Der Einsatz von Rettungsrobotern kann die Arbeit von Einsatzkräften der Feuerwehr erheblich erleichtern. Allerdings müssen die Einsatzkräfte dafür entsprechend geschult werden. Im vorliegenden Beitrag wird ein Softwareprototyp vorgestellt, der vorhandene Daten zum Ablauf von Feuerwehreinsätzen für eine prozessorientierte Schulung der Einsatzkräfte nutzt. Ausgehend von konkreten Anwendungskontexten, in denen der Einsatz von Rettungsrobotern besondere Vorteile bringen kann, werden Referenzprozessmodelle für eine nutzenstiftende Zusammenarbeit zwischen Menschen und Robotern entwickelt. Diese dienen als Grundlage für den Schulungsprototyp, welcher eine schrittweise Rekapitulation des Einsatzablaufs sowie einen Vergleich des Einsatzes mit dem Referenzprozessmodell erlaubt. Dieser Vergleich, für den Process-Mining-Ansätze verwendet werden, legt Abweichungen des Einsatzes vom idealtypischen Ablauf offen, welche bei Schulungen besprochen und diskutiert werden können. Der Prototyp wird im Rahmen eines laufenden Forschungsprojekts in Kooperation mit Einsatzkräften entwickelt und wurde bereits positiv bewertet.
... The goal of our research thus is to develop methods for extracting run-time mission knowledge from the verbal communication in the response team. The acquired mission knowledge can also be used to assist the first responders during or after the mission, for example, by supporting the real-time coordination of human and robot actions or by mission documentation generation (Willms et al., 2019). ...
... The models we develop are being integrated as part of the speech processing pipeline in a mission-support system that provides process assistance and facilitates the creation of mission documentation (Willms et al., 2019). It will be evaluated in practice with and by first responders. ...
... As a result, the system determines that there is enough time for a detour to the nearest service station and changes the planned route of the truck. In a similar vein, NLU technology can be used to document and support the execution of fully manual rescue processes during emergency response missions [104]. ...
Augmented Business Process Management Systems (ABPMSs) are an emerging class of process-aware information systems that draws upon trustworthy AI technology. An ABPMS enhances the execution of business processes with the aim of making these processes more adaptable, proactive, explainable, and context-sensitive. This manifesto presents a vision for ABPMSs and discusses research challenges that need to be surmounted to realize this vision. To this end, we define the concept of ABPMS, we outline the lifecycle of processes within an ABPMS, we discuss core characteristics of an ABPMS, and we derive a set of challenges to realize systems with these characteristics.
... Participating in CCST was a way to kick start the development of an AR system of our own. Our long-term aim is to develop a domain-independent AR system to be used in several projects of the Talking Robots Group at DFKI 2 , including for the interpretation of team communication in disaster response (Willms et al., 2019). ...
Unexpected robot failures are inevitable. We propose to leverage socio-technical relations within the human-robot ecosystem to support adaptable strategies for handling unexpected failures. The Theory of Graceful Extensibility is used to understand how the architecture of the ecosystem can influence its ability to respond to unexpected events. By expanding our perspective from Human-Robot Interaction to the Human-Robot Ecosystem, adaptable failure-handling strategies are identified, alongside technical, social and organizational arrangements that are needed to support them. We argue that robotics and HRI communities should pursue more holistic approaches to failure-handling, recognizing the need to embrace the unexpected and consider socio-technical relations within the human robot ecosystem when designing failure-handling strategies.
Mobile robots like drones or ground vehicles can be a valuable addition to emergency response teams, because they reduce the risk and the burden for human team members. However, the need to manage and coordinate human-robot team operations during ongoing missions adds an additional dimension to an already complex and stressful situation. BPM approaches can help to visualize and document the disaster response processes underlying a mission. In this paper, we show how data from a ground robot’s reconnaissance run can be used to provide process assistance to the officers. By automatically recognizing executed activities and structuring them as an ad-hoc process instance, we are able to document the executed process and provide real-time information about the mission status. The resulting mission progress process model can be used for additional services, such as officer training or mission documentation. Our approach is implemented as a prototype and demonstrated using data from an ongoing research project on rescue robotics.
Task allocation and management is crucial for human-robot collaboration in Urban Search And Rescue response efforts. The job of a mission team leader in managing tasks becomes complicated when adding multiple and different types of robots to the team. Therefore, to effectively accomplish mission objectives, shared situation awareness and task management support are essential. In this paper, we design and evaluate an ontology which provides a common vocabulary between team members, both humans and robots. The ontology is used for facilitating data sharing and mission execution, and providing the required automated task management support. Relevant domain entities, tasks, and their relationships are modeled in an ontology based on vocabulary commonly used by firemen, and a user interface is designed to provide task tracking and monitoring. The ontology design and interface are deployed in a search and rescue system and its use is evaluated by firemen in a task allocation and management scenario. Results provide support that the proposed ontology (1) facilitates information sharing during missions; (2) assists the team leader in task allocation and management; and (3) provides automated support for managing an Urban Search and Rescue mission.
Given an event log describing observed behaviour, process discovery aims to find a process model that ‘best’ describes this behaviour. A large variety of process discovery algorithms has been proposed. However, no existing algorithm returns a sound model in all cases (free of deadlocks and other anomalies), handles infrequent behaviour well and finishes quickly. We present a technique able to cope with infrequent behaviour and large event logs, while ensuring soundness. The technique has been implemented in ProM and we compare the technique with existing approaches in terms of quality and performance.
This paper describes our experience in designing, developing and deploying systems for supporting human–robot teams during disaster response. It is based on R&D performed in the EU-funded project NIFTi. NIFTi aimed at building intelligent, collaborative robots that could work together with humans in exploring a disaster site, to make a situational assessment. To achieve this aim, NIFTi addressed key scientific design aspects in building up situation awareness in a human–robot team, developing systems using a user-centric methodology involving end users throughout the entire R&D cycle, and regularly deploying implemented systems under real-life circumstances for experimentation and testing. This has yielded substantial scientific advances in the state-of-the-art in robot mapping, robot autonomy for operating in harsh terrain, collaborative planning, and human–robot interaction. NIFTi deployed its system in actual disaster response activities in Northern Italy, in July 2012, aiding in structure damage assessment.
This paper summarizes the latest, final version of ISO standard 24617-2 "Semantic annotation framework, Part 2: Dialogue acts". Compared to the preliminary version ISO DIS 24617-2:2010, described in Bunt et al. (2010), the final version additionally includes concepts for annotating rhetorical relations between dialogue units, defines a full-blown compositional semantics for the Dialogue Act Markup Language DiAML (resulting, as a side-effect, in a different treatment of functional dependence relations among dialogue acts and feedback dependence relations); and specifies an optimally transparent XML-based reference format for the representation of DiAML annotations, based on the systematic application of the notion of 'ideal concrete syntax'. We describe these differences and briefly discuss the design and implementation of an incremental method for dialogue act recognition, which proves the usability of the ISO standard for automatic dialogue annotation.
We are exploring using techniques of semantic integration to enable the integration of teams of mobile robots into new do- mains.. This paper proposes an approach to semantic integra- tion which relies on the persona structure first presented for allocating resources within distributed, heterogeneous teams of robots. The persona, a reflection of the internal state of the robot, can be mapped to different domains through the use of a domain adapter that performs a semantic mapping between the ontology of the robot and the ontology of the domain. In this context, the multi-robot system and each robot play some role in the external domain; identifying and mapping these roles will be a critical first step in solving the larger problem. This work will ultimately enable more rapid integration of teams of robots into varied domains. We are investigating semantic integration and ontology mapping as applied to distributed robotics in new domains and systems. Multirobot systems are currently used in a wide variety of situations, from military applications to search and rescue to off-world exploration. Unfortunately, there is little semantic interoperability between domains - existing approaches to multirobot system design incorporate domain knowledge directly in the robot software and sys- tem designs (Marmelstein 2002; DiLeo, Jacobs, & DeLoach 2002). Adaptation to the information environment of a new domain will be important for the rapid deployment of robot systems, particularly for organizations such as CRASAR that respond to Urban Search and Rescue events with a va- riety of external organizations. For instance, this could al- low robot systems designed for use with US search and res- cue teams to be integrated into an agent-based multinational rescue operation of the sort described in (Tate et al. 2004; Pechouˇ cek, Mar´ik, & B´
We describe the design of Kaldi, a free, open-source toolkit for speech recognition research. Kaldi provides a speech recognition system based on finite-state transducers (using the freely available OpenFst), together with detailed documentation and scripts for building complete recognition systems. Kaldi is written is C++, and the core library supports modeling of arbitrary phonetic-context sizes, acoustic modeling with subspace Gaussian mixture models (SGMM) as well as standard Gaussian mixture models, together with all commonly used linear and affine transforms. Kaldi is released under the Apache License v2.0, which is highly nonrestrictive, making it suitable for a wide community of users.
The focus of this paper is ontology-based knowledge man- agement in the framework of a mobile communication and information system for rescue operation management. We present a novel ontology data service, combining prior domain knowledge about large-scale rescue operations with dynamic information about a developing operation. We also discuss the integration of such a data service into a service-oriented application framework to reach high performance and accessibility, and offer examples of SHARE applications to demonstrate the practical ben- efits of the approach chosen.
This paper describes an ontology data service (ODS) for supporting Search and Rescue (SaR) operations. The ontological model
represents various aspects of the command, communication, and organisational structure of the SaR forces and the deployment
and progress of a SaR operation. Furthermore, the ontology supports the semantic indexing of multimedia documents in the context
of SaR processes and activities. This ODS supports a semantically-enhanced information and communication system for SaR forces.
Modelling the spatio-temporal aspects of an operation in alignment with possibly-unreliable information automatically extracted
from multimedia objects, introduces a number of challenges for the field of knowledge representation and reasoning.
Over the last two decades a significant number of projects tried to convey the concept of business process man-agement to the domain of emergency management. Most of these approaches sought a partial automation for the execution of standard operating procedures in the sense of workflow support, while others strived for the sup-port and integration of information management and data streams in command centres during a crisis. This pa-per focuses on the planning of disaster response for reasons of better preparedness. It discusses whether emer-gency management organisations can capitalize on off-the-shelf business process modelling tools to prepare for disasters more effectively, and whether the concepts of process modelling can be applied to standard operating procedures and vice versa. Moreover, it investigates whether such tools can efficiently support a collaborative preparation of police, fire departments, and rescue organizations. This paper will demonstrate why conventional business process means are inapplicable as planning tool in this domain. And it will also give an outlook to so called smart checklists that might be better suited both for the planning and response phase of emergency episodes.
We report on a field exercise in which a team of human fire-fighters used robots to enact a realistic disaster response mission in an industrial environment. In this exercise we evaluated the technical working of an integrated robotic system and gained insights concerning the manner in which robots and information streams can be utilized effectively. We have learnt important lessons regarding the employment of human-robot teams in complex, realistic missions.
Many Natural Language Processing applications nowadays rely on pre-trained word representations estimated from large text corpora such as news collections, Wikipedia and Web Crawl. In this paper, we show how to train high-quality word vector representations by using a combination of known tricks that are however rarely used together. The main result of our work is the new set of publicly available pre-trained models that outperform the current state of the art by a large margin on a number of tasks.
This is the second edition of Wil van der Aalst’s seminal book on process mining, which now discusses the field also in the broader context of data science and big data approaches. It includes several additions and updates, e.g. on inductive mining techniques, the notion of alignments, a considerably expanded section on software tools and a completely new chapter of process mining in the large. It is self-contained, while at the same time covering the entire process-mining spectrum from process discovery to predictive analytics. After a general introduction to data science and process mining in Part I, Part II provides the basics of business process modeling and data mining necessary to understand the remainder of the book. Next, Part III focuses on process discovery as the most important process mining task, while Part IV moves beyond discovering the control flow of processes, highlighting conformance checking, and organizational and time perspectives. Part V offers a guide to successfully applying process mining in practice, including an introduction to the widely used open-source tool ProM and several commercial products. Lastly, Part VI takes a step back, reflecting on the material presented and the key open challenges. Overall, this book provides a comprehensive overview of the state of the art in process mining. It is intended for business process analysts, business consultants, process managers, graduate students, and BPM researchers.
Business process models are used within a range of organizational initiatives, where every stakeholder has a unique perspective on a process and demands the respective model. As a consequence, multiple process models capturing the very same business process coexist. Keeping such models in sync is a challenge within an ever changing business environment: once a process is changed, all its models have to be updated. Due to a large number of models and their complex relations, model maintenance becomes error-prone and expensive. Against this background, business process model abstraction emerged as an operation reducing the number of stored process models and facilitating model management. Business process model abstraction is an operation preserving essential process properties and leaving out insignificant details in order to retain information relevant for a particular purpose. Process model abstraction has been addressed by several researchers. The focus of their studies has been on particular use cases and model transformations supporting these use cases. This thesis systematically approaches the problem of business process model abstraction shaping the outcome into a framework. We investigate the current industry demand in abstraction summarizing it in a catalog of business process model abstraction use cases. The thesis focuses on one prominent use case where the user demands a model with coarse-grained activities and overall process ordering constraints. We develop model transformations that support this use case starting with the transformations based on process model structure analysis. Further, abstraction methods considering the semantics of process model elements are investigated. First, we suggest how semantically related activities can be discovered in process models-a barely researched challenge. The thesis validates the designed abstraction methods against sets of industrial process models and discusses the method implementation aspects. Second, we develop a novel model transformation, which combined with the related activity discovery allows flexible non-hierarchical abstraction. In this way this thesis advocates novel model transformations that facilitate business process model management and provides the foundations for innovative tool support.
Conceptual models play an increasingly important role in all phases of the information systems life cycle. For instance, they are used for business engineering, information systems development and customizing of enterprise resource planning (ERP) systems. Despite conceptual modeling being a vital instrument for developing information systems, the modeling process often is resource consuming and faulty. As a way to overcome these failures and to improve the development of enterprise-specific models, the concept of reference modeling has been introduced. A reference model is a conceptual framework and may be used as a blueprint for information systems development. In this chapter, we seek to motivate research on reference modeling and introduce the chapters of this book on using reference models for business systems analysis. Our discussion is based on a framework for research on reference modeling that consists of four elements: reference modeling languages, reference modeling methods, reference models and reference modeling context. Each element of the framework is discussed with respect to prior research, the contributions of chapters in this book and future research opportunities.
Business process management is one of the most widely discussed topics in information systems research. As process models advance in both complexity and maturity, reference models, serving as reusable blueprints for the development of individual models, gain more and more importance. Only a few business domains have access to commonly accepted reference models, so there is a widespread need for the development of new ones. This article describes a new inductive approach for the development of reference models, based on existing individual models from the respective domain. It employs a graph-based paradigm, exploiting the underlying graph structures of process models by identifying frequent common subgraphs of the individual models, analyzing their order relations, and merging them into a new model. This newly developed approach is outlined and evaluated in this contribution. It is applied in three different case studies and compared to other approaches to the inductive development of reference models in order to highlight its characteristics as well as assets and drawbacks.
Traditionally, business processes are designed using a top down approach. While in top down approaches real process experiences can only be considered in an indirect way, process experiences can be the core input for process model designs using a more innovative bottom up approach with inductive methods, e.g. process mining technologies. The paper introduces a comprehensive seven phases method for inductive reference modelling. Some of the relevant particular techniques in this context are presented. Finally, the vision of the IWi Process Model Corpus is presented. This corpus can serve as a basis for developing and evaluating methods and techniques in the area of inductive reference modelling and currently covers 2,290 single models.
Weaving together information spread over several public and private organizations is crucial for managing catastrophic events and for realizing resilient social infrastructures. While beneficial in emergencies, an unlimited access to (sensitive) data is usually defined as the worst case in any privacy or IT security scenario. As a solution to this tradeoff, the transferability of successful methods and tools known from business process and workflow management to rescue processes is discussed. The resulting framework as well as the identified research questions do not aim at generating "pure" technical security but at reducing the probability of misuse and, thus, providing a sound technical basis for a social discussion on resilient infrastructures.
Conceptual models play an increasingly important role in all phases of the information systems life cycle. Despite being vital for developing information systems, the modeling process is often resource consuming and faulty. Reference Modeling for Business Systems Analysis addresses the problems by covering methodological issues and reference models for several industries, as well as concepts and techniques introduced with concrete examples. Reference Modeling for Business Systems Analysis covers all aspects of reference modeling, and provides foundations for model-driven systems development. This book helps to efficiently reuse conceptual models and explains best practice models for manufacturing, retail, and electronic business.
I. Dialogue act classification in team communication for robot assisted disaster response
- T Anakina
- Á Kruijff-Korbayov
ANAKINA, T., AND KRUIJFF-KORBAYOVÁ, I. Dialogue act classification in team communication for robot assisted disaster response. In
Proceedings of the 20th Annual SIGdial Meeting on Discourse and
Dialogue, Stockholm, Sweden, September 11-13, 2019 (to appear).
Extending OWL ontologies by cartesian types to represent n-ary relations in natural language
- H.-U Krieger
- C Willms
KRIEGER, H.-U., AND WILLMS, C. Extending OWL ontologies by
cartesian types to represent n-ary relations in natural language. In
Language and Ontologies 2015 (2015).
Cognition-enabled framework for mixed human-robot rescue teams
- F Yazdani
- G Kazhoyan
- A K Bozcuoglu
- A Haidu
- F Balint-Benczedi
- D Bessler
Open source automatic speech recognition for German
- B Milde
- A Kohn
MILDE, B., AND KÖHN, A. Open source automatic speech recognition for German. In 13th ITG Symposium on Speech Communication
(2018).
Improving adaptive human-robot cooperation through work agreements
- T Mioch
- M Peeters
- M Neerincx
The drones and robots that helped save Notre Dame
- T Nardi
NARDI, T.
The drones and robots that helped save
Notre Dame.
https://hackaday.com/2019/04/17/
the-drones-and-robots-that-helped-save-notredame. Accessed: 2019-03-05.
Deploying process management for emergency services-lessons learnt and research required
- G Peinel
- T Rose
PEINEL, G., AND ROSE, T. Deploying process management for
emergency services-lessons learnt and research required. In Future
Security-6th Security Research Conference, Berlin, Germany (2010).
Improving emergency management by formal dynamic process-modelling
- U Ruppel
- A Wagenknecht
RÜPPEL, U., AND WAGENKNECHT, A. Improving emergency management by formal dynamic process-modelling. In 24th Conference
on Information Technology in Construction, Maribor, Slovenia (2007),
pp. 559-564.
Dialogue act classification in team communication for robot assisted disaster response
- T And
ANAKINA, T., AND KRUIJFF-KORBAYOV´AKORBAYOV´ KORBAYOV´A, I. Dialogue act classification in team communication for robot assisted disaster response. In
Proceedings of the 20th Annual SIGdial Meeting on Discourse and
Dialogue, Stockholm, Sweden, September 11-13, 2019 (to appear).
Improving emergency management by formal dynamic process-modelling
- U And
R ¨
UPPEL, U., AND WAGENKNECHT, A. Improving emergency management by formal dynamic process-modelling. In 24th Conference
on Information Technology in Construction, Maribor, Slovenia (2007),
pp. 559-564.
Business process model abstraction
- A Polyvyanyy
- S Smirnov
- M Weske