Michael Beetz

Universität Bremen | Uni Bremen · Institute for Artificial Intelligence

To facilitate natural and intuitive interactions with diverse user groups in real-world settings, social robots must be capable of addressing the varying requirements and expectations of these groups while adapting their behavior based on user feedback. While previous research often focuses on specific demographics, we present a novel framework for...

This paper presents SPI-DP, a novel first-order optimizer capable of optimizing robot programs with respect to both high-level task objectives and motion-level constraints. To that end, we introduce DGPMP2-ND, a differentiable collision-free motion planner for serial N-DoF kinematics, and integrate it into an iterative, gradient-based optimization...

In this paper, we present the service robot MARLIN and its integration with the K4R platform, a cloud system for complex AI applications in retail. At its core, this platform contains so-called semantic digital twins, a semantically annotated representation of the retail store. MARLIN continuously exchanges data with the K4R platform, improving the...

In the last decade, there have been great advancements in household robotics, enabling robots to autonomously accomplish household tasks. These robots are typically programmed for specific tasks and/ or objects. We hypothesise that the lack of flexibility in fulfilling new ad-hoc task requests can be overcome by a knowledgebased approach, allowing...

One of the visions in AI based robotics are household robots that can autonomously handle a variety of meal preparation tasks. Based on this scenario, we present a best practice tutorial on how to create actionable knowledge graphs that a robot can use for execution of task variations of cutting actions. We implemented a solution for this task that...

In a previous work, we designed a human-like white-box and causal generative model of perception NaivPhys4RP, essentially based on cognitive emulation to understand the past, the present and the future of the state of complex worlds from poor observations. In this paper, as recommended in that previous work, we first refine the theoretical model of...

Surface treatment tasks such as grinding, sanding or polishing are a vital step of the value chain in many industries, but are notoriously challenging to automate. We present RoboGrind, an integrated system for the intuitive, interactive automation of surface treatment tasks with industrial robots. It combines a sophisticated 3D perception pipeline...

While robots are increasingly present in everyday environments as vacuum cleaners, lawnmowers, or voice assistants , cognitive robots that autonomously help the elderly, prepare meals, do the laundry, or clean up are still missing. This is due to various reasons. Besides such robots being less cost-efficient, needing more knowledge, and the ability...

The paper presents a novel cloud-based digital twin learning platform for teaching and training concepts of cognitive robotics. Instead of forcing interested learners or students to install a new operating system and bulky, fragile software onto their personal laptops just to solve tutorials or coding assignments of a single lecture on robotics, it...

Abstract — Safety is essential in mission-critical operations with respect to human life and financial costs. However, despite the progress achieved in robot hardware and software, safety concerns remain a major issue hindering the effective integration of these robots. Verification and audit in mission-critical processes are essential not only to...

This study addresses the predictive limitation of probabilistic circuits and introduces transformations as a remedy to overcome it. We demonstrate this limitation in robotic scenarios. We motivate that independent component analysis is a sound tool to preserve the independence properties of probabilistic circuits. Our approach is an extension of jo...

The Collaborative Research Center for Everyday Activity Science & Engineering (CRC EASE) aims to enable robots to perform environmental interaction tasks with close to human capacity. It therefore employs a shared ontology to model the activity of both kinds of agents, empowering robots to learn from human experiences. To properly describe these hu...

In a previous work, we designed a human-like white-box and causal generative model of perception NaivPhys4RP, essentially based on cognitive emulation to understand the past, the present and the future of the state of complex worlds from poor observations. In this paper, as recommended in that previous work, we first refine the theoretical model of...

Along the process of digital transformation, retailers are increasingly using digital shopping assistants to support their customers with additional services from electronic shopping carts to click & collect services. Such assistants need large amounts of data that needs to be imported and linked from decentralised sources such as from the Web. Unf...

Aging societies, labor shortages and increasing wage costs call for assistance robots capable of autonomously performing a wide array of real-world tasks. Such open-ended robotic manipulation requires not only powerful knowledge representations and reasoning (KR&R) algorithms, but also methods for humans to instruct robots what tasks to perform and...

There is an increasing demand for robots capable of safely interacting with individuals in everyday scenarios. This necessitates their ability to comprehend human behavior, predict future actions, and respond accordingly. This is precisely the focus of the field known as cognitive robotics . Cognitive robots can learn from experience and others,...

The Web offers plenty of product information that is valuable for supporting decision processes. Research on Web knowledge acquisition and the Semantic Web has led to the creation of many domain ontologies and Web applications. What still is lacking is a connection of such knowledge to the real world. If object information is linked to environment...

Aging societies, labor shortages and increasing wage costs call for assistance robots capable of autonomously performing a wide array of real-world tasks. Such open-ended robotic manipulation requires not only powerful knowledge representations and reasoning (KR&R) algorithms, but also methods for humans to instruct robots what tasks to perform and...

Digital twins (DTs) offer high-fidelity, data-enhanced virtual representations of physical entities. Introduced by NASA, DTs have since found popularity in the automotive, healthcare, energy, and, in particular, robotic sectors. In this work, we discuss the potential of semantic DTs (SemDTs) in the harsh, dynamic, and unstructured environments comm...

While the Digital Twin technology can be used by robotic agents to autonomously digitise retail stores, the Semantic Web offers vast machine-understandable product information that can be utilised by both digital and robotic agents. We propose connecting shopping assistants to a semantic Digital Twin for a service-oriented shopping experience. The...

This paper presents a hybrid robot cognitive architecture, CRAM, that enables robot agents to accomplish everyday manipulation tasks. It addresses five key challenges that arise when carrying out everyday activities. These include (i) the underdetermined nature of task specification, (ii) the generation of context-specific behavior, (iii) the abili...

In this paper, we introduce a proven cognitive architecture to the underwater domain, where such architectures are becoming increasingly needed. Components of the proposed framework include a mission planner and executioner, knowledge base and reasoner, real-time photorealistic simulator, web-based service for replaying and explaining past missions...

We introduce Joint Probability Trees (JPT), a novel approach that makes learning of and reasoning about joint probability distributions tractable for practical applications. JPTs support both symbolic and subsymbolic variables in a single hybrid model, and they do not rely on prior knowledge about variable dependencies or families of distributions....

Human demonstrations of everyday activities are an important resource to learn the particularities of the corresponding control strategies that are needed to perform such activities with ease and competence. However, such demonstrations need to be annotated such that time segments get associated to the appropriate actions. Previous research in psyc...

Perception in complex environments especially dynamic and human-centered ones goes beyond classical tasks such as classification usually known as the what- and where-object-questions from sensor data, and poses at least three challenges that are missed by most and not properly addressed by some actual robot perception systems. Note that sensors are...

Optimising object order in stacking problems remains a hard problem for cognitive robotics research. In this paper, we continue our work on using the spatiotemporal relationships called image schemas to represent affordance spaces founded on object properties. Based on object properties, we introduce a stacking-order algorithm and describe the acti...

Perception in complex environments especially dynamic and human-centered ones goes beyond classical tasks such as classification usually known as the what- and where-object-questions from sensor data, and poses at least three challenges that are missed by most and not properly addressed by some actual robot perception systems. Note that sensors are...

The chapter focuses on research on robotic assistants and the involved challenge of their manipulating the physical world. It describes the state of the art in this regard and outlines directions for future research. Furthermore, it reports how the Delphi respondents assess various facets of human–robot communication and how specifically the group...

If a technology lacks social acceptance, it cannot realize dissemination into society. The chapter thus illuminates the ethical, legal, and social implications of robotic assistance in care and daily life. It outlines a conceptual framework and identifies patterns of trust in human–robot interaction. The analysis relates trust in robotic assistance...

Retail stores are a promising application domain for autonomous robotics. Unlike other domains, such as households, the environments are more structured, products are designed to be easily recognizable, and items are consciously placed to facilitate their detection and manipulation. In this chapter we exploit these properties and propose a mobile r...

Nowadays, robots are used in the retail market mostly for warehousing, while they could be of great help in different in-store logistics processes as discussed in previous chapters. The present chapter deals with the shelf replenishment task; its execution by a robot requires overcoming of technological and methodological barriers in the handling o...

In both industrial and service domains, a central benefit of the use of robots is their ability to quickly and reliably execute repetitive tasks. However, even relatively simple peg-in-hole tasks are typically subject to stochastic variations, requiring search motions to find relevant features such as holes. While search improves robustness, it com...

Service robots in the future need to execute abstract instructions such as “fetch the milk from the fridge”. To translate such instructions into actionable plans, robots require in-depth background knowledge. With regards to interactions with doors and drawers, robots require articulation models that they can use for state estimation and motion pla...

In this extended abstract, we present initial work on intelligent object stacking by household robots using a symbolic approach grounded in image schema research. Image schemas represent spatiotemporal relationships that capture objects' affordances and dispositions. Therefore, they offer the first step to ground semantic information in symbolic de...

A key challenge for robotic systems is to figure out the behavior of another agent. The capability to draw correct inferences is crucial to derive human behavior from examples. Processing correct inferences is especially challenging when (confounding) factors are not controlled experimentally (observational evidence). For this reason, robots that r...

Estimating the pose of an object is essential for robot manipulation. In many applications the spatial and geometric relations between the object and the other parts of the world, e.g. the relation between the object and its supporting plane, are a-priori known or can be assumed with a certain accuracy. This information can be leveraged for pose es...

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave m...

Autonomous robots struggle with plan adaption in uncertain and changing environments. Although modern robots can make popcorn and pancakes, they are incapable of performing such tasks in unknown settings and unable to adapt action plans if ingredients or tools are missing. Humans are continuously aware of their surroundings. For robotic agents, rea...

Today's database systems have shown to be capable of supporting AI applications that demand a lot of data processing. To this end, these systems incorporate powerful querying languages that go far beyond the mere retrieval of data, and provide sophisticated facilities for data processing as well. In the case of SQL, the language has been even demon...

Today's database systems have shown to be capable of supporting AI applications that demand a lot of data processing. To this end, these systems incorporate powerful querying languages that go far beyond the mere retrieval of data, and provide sophisticated facilities for data processing as well. In the case of SQL, the language has been even demon...

As digitization advances, stationary retail is increasingly enabled to develop novel retail services aiming at enhancing efficiency of business processes ranging from in-store logistics to customer shopping experiences. In contrast to online stores, stationary retail digitization demands for an integration of various data like location information,...

Dealing with robotic actions in uncertain environments has been demonstrated to be hard. Many classic planning approaches to robotic action make the closed world assumption, rendering them inefficient for everyday household activities, as they function without generalizability to other contexts or the ability to deal with unexpected changes. In con...

This work presents a cloud-to-edge framework capable of collecting and annotating synthetic images from human performances in virtual environments with the purpose of enabling the training and deployment of robot vision models. The virtual environment is capable of providing close-to-reality image data using state of the art rendering capabilities...

In this paper, we present a novel method to learn end-to-end visuomotor policies for robotic manipulators. The method computes state-action mappings in a supervised learning manner from video demonstrations and robot trajectories. We show that the robot learns to perform different tasks by associating image features with the corresponding movement...

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave m...

Challenging manipulation tasks can be solved effectively by combining individual robot skills, which must be parameterized for the concrete physical environment and task at hand. This is time-consuming and difficult for human programmers, particularly for force-controlled skills. To this end, we present Shadow Program Inversion (SPI), a novel appro...

Human developers want to program robots using abstract instructions, such as "fetch the milk from the fridge". To translate such instructions into actionable plans, the robot's software requires in-depth background knowledge. With regards to interactions with articulated objects such as doors and drawers, the robot requires a model that it can use...

Anticipating what might happen as a result of an action is an essential ability humans have in order to perform tasks effectively. On the other hand, robots capabilities in this regard are quite lacking. While machine learning is used to increase the ability of prospection it is still limiting for novel situations. A possibility to improve the pros...

In this paper we present a system capable of collecting and annotating, human performed, robot understandable, everyday activities from virtual environments. The human movements are mapped in the simulated world using off-the-shelf virtual reality devices with full body, and eye tracking capabilities. All the interactions in the virtual world are p...

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave m...

Many of today's robot perception systems aim at accomplishing perception tasks that are too simplistic and too hard. They are too simplistic because they do not require the perception systems to provide all the information needed to accomplish manipulation tasks. Typically the perception results do not include information about the part structure o...

In this paper, we present an experiment, designed to investigate and evaluate the scalability and the robustness aspects of mobile manipulation. The experiment involves performing variations of mobile pick and place actions and opening/closing environment containers in a human household. The robot is expected to act completely autonomously for exte...

Visual robot perception has been challenging to successful robot manipulation in noisy, cluttered and dynamic environments. While some perception systems fail to provide an adequate semantics of the scene, others fail to present appropriate learning models and training data. Another major issue encountered in some robot perception systems is their...

One of the problems that service robotics deals with is to bring mobile manipulators to work in semi-structured human scenarios, which requires an efficient and flexible way to execute everyday tasks, like serve a cup in a cluttered environment. Usually, for those tasks, the combination of symbolic and geometric levels of planning is necessary, as...

One of the key reasoning tasks of robotic agents is inferring possible actions that can be accomplished with a given object at hand. This cognitive task is commonly referred to as inferring the affordances of objects. In this paper, we propose a novel conceptualization of affordances and its realization as a description logic ontology. The key idea...

Robotic systems in production environments have to adapt quickly to new situations and products to enable customization and short product cycles. This is especially true for the robot perception, which is sensitive to changes in environment and task. Therefore, we present an approach to quickly synthesize perception pipelines based on hierarchical...

A review and comparison of ontology-based approaches to robot autonomy – ADDENDUM - Volume 35 - Alberto Olivares-Alarcos, Daniel Beßler, Alaa Khamis, Paulo Goncalves, Maki K. Habib, Julita Bermejo-Alonso, Marcos Barreto, Mohammed Diab, Jan Rosell, João Quintas, Joanna Olszewska, Hirenkumar Nakawala, Edison Pignaton, Amelie Gyrard, Stefano Borgo, Gu...

Manipulation planning and control are relevant building blocks of a robotic system and their tight integration is a key factor to improve robot autonomy and allows robots to perform manipulation tasks of increasing complexity, such as those needed in the in-store logistics domain. Supermarkets contain a large variety of objects to be placed on the...

To enable cooperative task planning and coordination between the human operator and robot teams, new types of interfaces are needed. We present an interactive strategic mission management system (ISMMS) for underwater explorations performed by mixed teams of robots and human investigators that enables cooperative task planning and coordination betw...

Embodied intelligent agents that are equipped with sensors and actuators have unique characteristics and requirements regarding the storage, management, and usage of information. The goal is to perform intentional activities, within the perception-action loop of the agent, based on the information acquired from its senses, background knowledge, nai...