Lorenzo Natale

Lorenzo Natale
Istituto Italiano di Tecnologia | IIT · Humanoid Sensing and Perception

PhD

About

267
Publications
61,231
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
7,405
Citations
Citations since 2017
110 Research Items
4253 Citations
20172018201920202021202220230200400600
20172018201920202021202220230200400600
20172018201920202021202220230200400600
20172018201920202021202220230200400600

Publications

Publications (267)
Preprint
Full-text available
Many robotic tasks involving some form of 3D visual perception greatly benefit from a complete knowledge of the working environment. However, robots often have to tackle unstructured environments and their onboard visual sensors can only provide incomplete information due to limited workspaces, clutter or object self-occlusion. In recent years, dee...
Preprint
Full-text available
Action recognition is a fundamental capability for humanoid robots to interact and cooperate with humans. This application requires the action recognition system to be designed so that new actions can be easily added, while unknown actions are identified and ignored. In recent years, deep-learning approaches represented the principal solution to th...
Preprint
Performing joint interaction requires constant mutual monitoring of own actions and their effects on the other's behaviour. Such an action-effect monitoring is boosted by social cues and might result in an increasing sense of agency. Joint actions and joint attention are strictly correlated and both of them contribute to the formation of a precise...
Preprint
Full-text available
The visual system of a robot has different requirements depending on the application: it may require high accuracy or reliability, be constrained by limited resources or need fast adaptation to dynamically changing environments. In this work, we focus on the instance segmentation task and provide a comprehensive study of different techniques that a...
Chapter
Full-text available
The current state of the art in cognitive robotics, covering the challenges of building AI-powered intelligent robots inspired by natural cognitive systems. A novel approach to building AI-powered intelligent robots takes inspiration from the way natural cognitive systems—in humans, animals, and biological systems—develop intelligence by exploiting...
Preprint
Full-text available
We consider the task of object grasping with a prosthetic hand capable of multiple grasp types. In this setting, communicating the intended grasp type often requires a high user cognitive load which can be reduced adopting shared autonomy frameworks. Among these, so-called eye-in-hand systems automatically control the hand aperture and pre-shaping...
Article
Full-text available
Social robotics is an emerging field that is expected to grow rapidly in the near future. In fact, it is increasingly more frequent to have robots that operate in close proximity with humans or even collaborate with them in joint tasks. In this context, the investigation of how to endow a humanoid robot with social behavioral skills typical of huma...
Article
Full-text available
According to the World Health Organization 1, ² the percentage of healthcare dependent population, such as elderly and people with disabilities, among others, will increase over the next years. This trend will put a strain on the health and social systems of most countries. The adoption of robots could assist these health systems in responding to t...
Article
This article addresses the concurrency issues affecting behavior trees (BTs), a popular tool to model the behaviors of autonomous agents in the video game and the robotics industry. BT designers can easily build complex behaviors composing simpler ones, which represents a key advantage of BTs. The parallel composition of BTs expresses a way to comb...
Preprint
6D object pose tracking has been extensively studied in the robotics and computer vision communities. The most promising solutions, leveraging on deep neural networks and/or filtering and optimization, exhibit notable performance on standard benchmarks. However, to our best knowledge, these have not been tested thoroughly against fast object motion...
Preprint
Full-text available
This paper addresses the concurrency issues affecting Behavior Trees (BTs), a popular tool to model the behaviors of autonomous agents in the video game and the robotics industry. BT designers can easily build complex behaviors composing simpler ones, which represents a key advantage of BTs. The parallel composition of BTs expresses a way to combin...
Article
Full-text available
Human-object interaction is of great relevance for robots to operate in human environments. However, state-of-the-art robotic hands are far from replicating humans skills. It is, therefore, essential to study how humans use their hands to develop similar robotic capabilities. This article presents a deep dive into hand-object interaction and human...
Chapter
Accurately detecting objects in unconstrained settings is crucial for robotic agents, such as humanoids, that function in ever-changing environments. Current deep learning based methods achieve remarkable performance on this task on general purpose benchmarks and they are therefore appealing for robotics. However, their high accuracy comes at the p...
Article
Full-text available
Conventional approaches to robot navigation in unstructured environments rely on information acquired from the LiDAR mounted on the robot base to detect and avoid obstacles. This approach fails to detect obstacles that are too small, or that are invisible because they are outside the LiDAR’s field of view. A possible strategy is to integrate inform...
Preprint
Recent visual pose estimation and tracking solutions provide notable results on popular datasets such as T-LESS and YCB. However, in the real world, we can find ambiguous objects that do not allow exact classification and detection from a single view. In this work, we propose a framework that, given a single view of an object, provides the coordina...
Article
Full-text available
Tactile sensing represents a valuable source of information in robotics for perception of the state of objects and their properties. Modern soft tactile sensors allow perceiving orthogonal forces and, in some cases, relative motions along the surface of the object. Detecting and measuring this kind of lateral motion is fundamental to react to possi...
Preprint
Reliable perception and efficient adaptation to novel conditions are priority skills for humanoids that function in dynamic environments. The vast advancements in latest computer vision research, brought by deep learning methods, are appealing for the robotics community. However, their adoption in applied domains is not straightforward since adapti...
Article
There is a growing interest in Behavior Trees (BTs) as a tool to describe and implement robot behaviors. BTs were devised in the video game industry and their adoption in robotics resulted in the development of ad-hoc libraries to design and execute BTs that fit complex robotics software architectures. While there is broad consensus on how BTs work...
Preprint
In this paper, we present a toolchain to design, execute, and verify robot behaviors. The toolchain follows the guidelines defined by the EU H2020 project RobMoSys and encodes the robot deliberation as a Behavior Tree (BT), a directed tree where the internal nodes model behavior composition and leaf nodes model action or measurement operations. Suc...
Preprint
Full-text available
There is a growing interest in Behavior Trees (BTs) as a tool to describe and implement robot behaviors. BTs were devised in the video game industry and their adoption in robotics resulted in the development of ad-hoc libraries to design and execute BTs that fit complex robotics software architectures. While there is broad consensus on how BTs work...
Preprint
Full-text available
Our research aims to enable automated property verification of deliberative components in robot control architectures. We focus on a formalization of the execution context of Behavior Trees (BTs) to provide a scalable, yet formally grounded, methodology to enable runtime verification and prevent unexpected robot behaviors to hamper deployment. To t...
Preprint
Research on tactile sensing has been progressing at constant pace. In robotics, tactile sensing is typically studied in the context of object grasping and manipulation. In this domain, the development of robust, multi-modal, tactile sensors for robotic hands has supported the study of novel algorithms for in-hand object manipulation, material class...
Preprint
Full-text available
In this chapter we describe the history and evolution of the iCub humanoid platform. We start by describing the first version as it was designed during the RobotCub EU project and illustrate how it evolved to become the platform that is adopted by more than 30 laboratories world wide. We complete the chapter by illustrating some of the research act...
Preprint
Full-text available
Hand-eye calibration of laser profile sensors is the process of extracting the homogeneous transformation between the laser profile sensor frame and the end-effector frame of a robot in order to express the data extracted by the sensor in the robot's global coordinate system. For laser profile scanners this is a challenging procedure, as they provi...
Article
Full-text available
Tracking the 6D pose and velocity of objects represents a fundamental requirement for modern robotics manipulation tasks. This paper proposes a 6D object pose tracking algorithm, called MaskUKF, that combines deep object segmentation networks and depth information with a serial Unscented Kalman Filter to track the pose and the velocity of an object...
Article
6D object pose tracking has been extensively studied in the robotics and computer vision communities. The most promising solutions, leveraging on deep neural networks and/or filtering and optimization, exhibit notable performance on standard benchmarks. However, to our best knowledge, these have not been tested thoroughly against fast object motion...
Preprint
Several object detection methods have recently been proposed in the literature, the vast majority based on Deep Convolutional Neural Networks (DCNNs). Such architectures have been shown to achieve remarkable performance, at the cost of computationally expensive batch training and extensive labeling. These methods have important limitations for robo...
Conference Paper
Full-text available
5G mobile networks are designed to fulfill very stringent requirements and support new vertical use cases. This transition to a vertical oriented delivery model will have a strong impact in the touristic sector. In this context, the "touristic city node" of 5G-TOURS, built in the city of Turin, aims at exploiting the potential of the media vertical...
Preprint
Full-text available
Object detection is a fundamental task for robots to operate in unstructured environments. Today, there are several deep learning algorithms that solve this task with remarkable performance. Unfortunately, training such systems requires several hours of GPU time. For robots, to successfully adapt to changes in the environment or learning new object...
Preprint
Object segmentation is a key component in the visual system of a robot that performs tasks like grasping and object manipulation, especially in presence of occlusions. Like many other Computer Vision tasks, the adoption of deep architectures has made available algorithms that perform this task with remarkable performance. However, adoption of such...
Conference Paper
Full-text available
The paper introduce a robotics software control architecture suitable for the development of complete robotic industrial applications. The architecture fuse the state-of-the-art software technologies in a single standalone platform to provide an easy integration between all the software components necessary to control a robotic application, i.e. PL...
Preprint
Full-text available
In this paper, we propose Belief Behavior Trees (BBTs), an extension to Behavior Trees (BTs) that allows to automatically create a policy that controls a robot in partially observable environments. We extend the semantic of BTs to account for the uncertainty that affects both the conditions and action nodes of the BT. The tree gets synthesized foll...
Preprint
Full-text available
Task planning in a probabilistic belief state domains allows generating complex and robust execution policies in those domains affected by state uncertainty. The performance of a task planner relies on the belief state representation. However, current belief state representation becomes easily intractable as the number of variables and execution ti...
Conference Paper
In this paper, we outline an interleaved acting and planning technique to rapidly reduce the uncertainty of the estimated robot's pose by perceiving relevant information from the environment, as recognizing an object or asking someone for a direction. Generally, existing localization approaches rely on low-level geometric features such as points, l...
Article
Full-text available
Object detection is a fundamental ability for robots interacting within an environment. While stunningly effective, state-of-the-art deep learning methods require huge amounts of labeled images and hours of training which does not favour such scenarios. This work presents a novel pipeline resulting from integrating (Maiettini et al. in 2017 IEEE-RA...
Preprint
Full-text available
In this paper, we outline an interleaved acting and planning technique to rapidly reduce the uncertainty of the estimated robot's pose by perceiving relevant information from the environment, as recognizing an object or asking someone for a direction. Generally, existing localization approaches rely on low-level geometric features such as points, l...
Preprint
The use of benchmarks is a widespread and scientifically meaningful practice to validate performance of different approaches to the same task. In the context of robot grasping the use of common object sets has emerged in recent years, however no dominant protocols and metrics to test grasping pipelines have taken root yet. In this paper, we present...
Article
The use of benchmarks is a widespread and scientifically meaningful practice to validate performance of different approaches to the same task. In the context of robot grasping the use of common object sets has emerged in latest years, however no dominant protocols and metrics to test grasping pipelines have taken root yet. In this paper, we present...
Chapter
A fundamental ingredient in the success of deep learning for computer and robot vision is the availability of very large-scale annotated databases. ImageNet, with its 1000 object classes and 1.2 million images, tends to be the dominant data collection for creating pre-trained deep architectures. A less investigated avenue is how the possibility to...
Preprint
Full-text available
Behavior Trees (BTs) are becoming a popular tool to model the behaviors of autonomous agents in the computer game and the robotics industry. One of the key advantages of BTs lies in their composability, where complex behaviors can be built by composing simpler ones. The parallel composition is the one with the highest potential since the complexity...
Article
The software development cycle in the robotic research environment is hectic and heavily driven by project or paper deadlines. Developers have only little time available for packaging the C/C++ code they write, develop and maintain the build system and continuous integration tools. Research projects are joint efforts of different groups working rem...
Preprint
Exploration is an extremely challenging problem in reinforcement learning, especially in high dimensional state and action spaces and when only sparse rewards are available. Effective representations can indicate which components of the state are task relevant and thus reduce the dimensionality of the space to explore. In this work, we take a repre...
Article
Full-text available
Musculoskeletal disorders, the single largest category of workrelated injuries in many industrial countries, are associated with very high costs in terms of lost productivity. In highvolume production facilities, large parts of the workstation should ideally be adapted to individual workers in real time to prevent such injuries. However, in smaller...
Preprint
Full-text available
For robots to share the environment and cooperate with humans without barriers, we need to guarantee safety to the operator and, simultaneously, to maximize the robot's usability. Safety is typically guaranteed by controlling the robot movements while, possibly, taking into account physical contacts with the operator, objects or tools. If possible,...
Poster
Full-text available
A full-day workshop on October 1 st , 2018 at IROS2018 in Madrid, Spain. Important Dates Paper submission guidelines We welcome submissions regarding any robotics application where tactile sensing modalities are used. As we aim to encourage meaningful discussion in the tactile perception and learning domain, work that is unpublished, recently publi...
Preprint
Full-text available
Behavior Trees (BTs) have become a popular framework for designing controllers of autonomous agents in the computer game and in the robotics industry. One of the key advantages of BTs lies in their modularity, where independent modules can be composed to create more complex ones. In the classical formulation of BTs, modules can be composed using on...
Article
Full-text available
One of the main advantages of building robots with size and motor capabilities close to those of humans, such as iCub, lies in the fact that they can potentially take advantage of a world populated with tools and devices designed by and for humans. However, in order to be able to do proper use of the tools around them, robots need to be able to inc...
Article
Full-text available
Background: In recent years, biomedical devices have proven to be able to target also different neurological disorders. Given the rapid ageing of the population and the increase of invalidating diseases affecting the central nervous system, there is a growing demand for biomedical devices of immediate clinical use. However, to reach useful therape...
Chapter
Full-text available
In this chapter we present WALK-MAN, a humanoid platform that has been developed to operate in realistic unstructured environments and demonstrate new skills including powerful manipulation, robust balanced locomotion, high strength capabilities and physical sturdiness. To enable these capabilities, WALK-MAN design and actuation are based on the mo...
Article
This work illustrates the design phases leading to the development of a new YARP device interface along with its client/server implementation. In order to obtain a smoother integration and a more reliable software usability, while avoiding common errors during the design phases, a new interface is created in the YARP network when a new family of de...
Article
Full-text available
Latest deep learning methods for object detection provided remarkable performance boost, but have limits when used in robotic applications. One of the most relevant issues is the long training time, which is due to the large size and unbalance of the associated training sets, characterized by few positive and tons of negative (i.e. background) exam...
Article
Tactile sensing is one important element that can enable robots to interact with an unstructured world. By having tactile perception, a robot can explore its environment by touching objects. Like human skin, a tactile sensor that can provide rich information such as distributed normal and shear forces with high density can help the robot to recogni...
Cover Page
Full-text available
Enabling cost-effective technologies for human–robot interactions are crucial for humanoid robotics. A major problem in robotic touch sensing performance is often cited as the lack of sensitivity at the robotic joints (e.g., fingers and elbows). In article number 1700587, Pietro Cataldi, Ilker S. Bayer, and co-workers demonstrate durable and stretc...
Article
Object grasping and manipulation in robotics has been largely approached using visual feedback. Human studies on the other hand have demonstrated the importance of tactile and force feedback to guide the interaction between the fingers and the objects. Inspired by these observations, we propose an approach that consists in guiding a robot’s actions...
Article
Full-text available
This paper presents some recent developments in YARP middleware, aimed to improve its integration with ROS. They include a new mechanism to read/write ROS transform frames and a new set of standard interfaces to intercommunicate with the ROS navigation stack. A novel set of YARP companion modules, which provide basic navigation functionalities for...