Content uploaded by Jonas Stenzel
Author content
All content in this area was uploaded by Jonas Stenzel on Aug 31, 2016
Content may be subject to copyright.
A preview of the PDF is not available
New Approaches for Singularization in Logistic Applications Using Low Cost 3D Sensors
Abstract and Figures
In this chapter, novel approaches for the detection of logistical objects (loading units) in the field of material flow applications are comparative presented, focusing on solutions using low cost 3D sensors. These approaches realize substantial changes in comparison to traditional system design of logistic processes. Complex 3D-vision systems, costly laser scanners or throughput decreasing local sensor solutions integrated in grippers are substituted by low cost Photonic Mixing Device (PMD) cameras or structured light sensors (like Asus Xtion or Microsoft Kinect). By using low cost sensors and modern point cloud processing algorithms for detection and classification in logistic applications like de-palletizing, automation of usually manual processes will be economically feasible. Besides the description of different basic solution concepts for 2.5D and 3D, two practical applications are presented.
Figures - uploaded by Jonas Stenzel
Author content
All figure content in this area was uploaded by Jonas Stenzel
Content may be subject to copyright.
... In 2015, Prasse et al. [Pra+15] used a robot arm and low-cost 3D sensors to perform the task of depalletization. They use two approaches. ...
... The atomic loading units are assumed to have a cuboid shape and are determined by employing a Random Sampling Consensus (RANSAC) [FB87] algorithm. Prasse et al. [Pra+15] evaluate the PMD approach on real data, and report the deviation of package dimensions for 9 parcels. Since the deviation in height is less than their threshold of 10 mm, they argue that approach is suitable for application in logistics. ...
Computer vision applications in transportation logistics and warehousing have a huge potential for process automation. We present a structured literature review on research in the field to help leverage this potential. All literature is categorized w.r.t. the application, i.e. the task it tackles and w.r.t. the computer vision techniques that are used. Regarding applications, we subdivide the literature in two areas: Monitoring, i.e. observing and retrieving relevant information from the environment, and manipulation, where approaches are used to analyze and interact with the environment. In addition to that, we point out directions for future research and link to recent developments in computer vision that are suitable for application in logistics. Finally, we present an overview of existing datasets and industrial solutions. We conclude that while already many research areas have been investigated, there is still huge potential for future research. The results of our analysis are also available online at https://a-nau.github.io/cv-in-logistics.
... The research domain of object activity recognition is significantly smaller than HAR. For pallets, research tends to focus on tracking in terms of temperature deviations or location [15,16,17,18]. In [19] the authors work on sensor-based pallets and develop a system to monitor humidity and temperature in the surrounding environment to make assumptions about the state of the transported food. ...
Pallets are one of the most important load carriers for international supply chains. Yet, continuously tracking activities such as Driving, Lifting or Standing along their life cycle is hardly possible. This contribution is the first to propose a taxonomy for sensor-based activity recognition of pallets. Different types of acceleration sensors are deployed in three logistical scenarios for creating a benchmark dataset. A random forest classifier is deployed for supervised learning. The results demonstrate that automated, sensor-based life cycle assessment based on the proposed taxonomy is feasible. All data and corresponding videos are published in the SPARL dataset [1].
We focus on enabling damage and tampering detection in logistics and tackle the problem of 3D shape reconstruction of potentially damaged parcels. As input we utilize single RGB images, which corresponds to use-cases where only simple handheld devices are available, e.g. for postmen during delivery or clients on delivery. We present a novel synthetic dataset, named Parcel3D, that is based on the Google Scanned Objects (GSO) dataset and consists of more than 13,000 images of parcels with full 3D annotations. The dataset contains intact, i.e. cuboid-shaped, parcels and damaged parcels, which were generated in simulations. We work towards detecting mishandling of parcels by presenting a novel architecture called CubeRefine R-CNN, which combines estimating a 3D bounding box with an iterative mesh refinement. We benchmark our approach on Parcel3D and an existing dataset of cuboid-shaped parcels in real-world scenarios. Our results show, that while training on Parcel3D enables transfer to the real world, enabling reliable deployment in real-world scenarios is still challenging. CubeRefine R-CNN yields competitive performance in terms of Mesh AP and is the only model that directly enables deformation assessment by 3D mesh comparison and tampering detection by comparing viewpoint invariant parcel side surface representations. Dataset and code are available at https://a-nau.github.io/parcel3d.
This paper presents a mobile manipulation platform designed for autonomous depalletizing tasks. The proposed solution integrates machine vision, control and mechanical components to increase flexibility and ease of deployment in industrial environments such as warehouses. A collaborative robot mounted on a mobile base is proposed, equipped with a simple manipulation tool and a 3D in-hand vision system that detects parcel boxes on a pallet, and that pulls them one by one on the mobile base for transportation. The robot setup allows to avoid the cumbersome implementation of pick-and-place operations, since it does not require lifting the boxes. The 3D vision system is used to provide an initial estimation of the pose of the boxes on the top layer of the pallet, and to accurately detect the separation between the boxes for manipulation. Force measurement provided by the robot together with admittance control are exploited to verify the correct execution of the manipulation task. The proposed system was implemented and tested in a simplified laboratory scenario and the results of experimental trials are reported.
Wenn bereits die Sensorik durch komplexe Datenverarbeitung und detailliertes Applikationswissen im Sensor zahlreiche Funktionen zur Lösung der Kundenanforderungen bereitstellt sowie weitere integrierte Funktionen eine automatische Adaption und Optimierung der Funktion ermöglichen, spricht man von Intelligenter Sensorik. Intelligente Sensorik ist notwendig, um die erforderliche Autonomie von Maschinen und Anlagen im Wandel zu cyber-physischen Systemen (CPS) im Kontext von Industrie 4.0 zu erreichen und gleichzeitig das Engineering von CPS zu vereinfachen. Im Fokus dieses Beitrags ist die Diskussion über die Eigenschaften eines Intelligenten Sensors und dessen Nutzung in exemplarischen logistischen Anwendungen, die durch den Wandel hin zu CPS profitieren können. Intelligente Sensorik lässt Zukunftsszenarien wie die Smart Factory erst realisierbar erscheinen.
This work addresses the task of robot depalletizing by means of a mobile manipulator, taking into account the problem of localizing the boxes to be removed from the pallet and a manipulation strategy that allows to pull the boxes without lifting them with the robot arm. The depalletizing task is of particular interest in the industrial scenario in order to increase efficiency, flexibility and economic affordability of automatic warehouses. The proposed solution makes use of a multi-sensor vision system and a force-controlled collaborative robot in order to detect the boxes on the pallet and to control the robot interaction with the boxes to be removed. The vision system comprises a fixed 3D Time-of-flight camera and an eye-in-hand 2D camera. Preliminary experimental results performed on a laboratory setup with a fixed-based robotic manipulator are reported to show the effectiveness of the perception and control system.
The need for automated distribution systems has recently grown due to labor shortages and an increasing preference among consumers to shop online. The process of depalletizing roll box pallets (RBPs) requires skillful work in a limited space, but an automated RBP depalletizing system with a speed equal to or exceeding that of human workers has yet to be developed. We solved this issue using two technologies. One is a system design and the other is an algorithm for prioritizing the packages to be removed from the stack. We designed a highspeed depalletizing system that can be installed in a narrow space at the distribution site. Equipped with a compact linear actuator mechanism, the system utilizes an end effector that can handle multiple packages simultaneously. Moreover, by optimizing the process of transferring packages on the system, we increased the speed. Our algorithm for prioritizing packages allows the proper selection of packages held simultaneously for faster depalletizing. However, if the height of complexly stacked packages is not known, depalletizing is difficult. To solve this issue, we developed an algorithm with special rules. Finally, we conducted depalletizing trials with the latest prototype system using different arrangements of stacked packages, and achieved an average speed of 683 packages per hour.
Wenn bereits die Sensorik durch komplexe Datenverarbeitung und detailliertes Applikationswissen im Sensor zahlreiche Funktionen zur Lösung der Kundenanforderungen bereitstellt sowie weitere integrierte Funktionen eine automatische Adaption und Optimierung der Funktion ermöglichen, spricht man von Intelligenter Sensorik. Intelligente Sensorik ist notwendig, um die erforderliche Autonomie von Maschinen und Anlagen im Wandel zu cyber-physischen Systemen (CPS) im Kontext von Industrie 4.0 zu erreichen und gleichzeitig das Engineering von CPS zu vereinfachen. Im Fokus dieses Beitrags ist die Diskussion über die Eigenschaften eines Intelligenten Sensors und dessen Nutzung in exemplarischen logistischen Anwendungen, die durch den Wandel hin zu CPS profitieren können. Intelligente Sensorik lässt Zukunftsszenarien wie die Smart Factory erst realisierbar erscheinen.
In this paper, a novel approach for the detection of parcel loading positions on a pallet is presented. This approach realizes a substantial change in comparison to traditional system design of contour detection in de-palletizing processes. Complex 3D-vision systems, costly laser scanners or throughput decreasing local sensor solutions integrated in grippers are substituted by a low-cost Photonic Mixing Device (PMD) camera. By combining PMD technology and a predetermined model of loading situations, stored during the assembly of the pallet, this approach can compensate for the drawbacks of each respective system. An essential part of the approach are computer-graphics methods specific to the given problem to both detect the deviation between the nominal and the actual loading position and if necessary an automated correction of the packaging scheme. From an economic point of view this approach can decrease the costs of mandatory contour checking in automated de-palletizing processes.
In this study, a novel approach for the detection of parcel loading positions on a pallet is presented. This approach was realized as a substantial change in comparison with traditional system design of contour detection in de-palletizing processes. Complex 3D-vision systems, costly laser scanners or throughput decreasing local sensor solutions integrated in grippers are substituted by a low-cost photonic mixing device (PMD) camera. By combining PMD technology and a predetermined model of loading situations, stored during assembling the pallet, this approach can compensate for the drawbacks of each respective system. An essential part of the approach are computer-graphics methods specific to the given problem to both detect the deviation between the nominal and the actual loading position and if necessary an automated correction of the packaging scheme. From an economic point of view, this approach can decrease the costs of mandatory contour checking in automated de-palletizing processes.
This paper presents a novel approach to camera calibration that improves final accuracy with respect to standard methods using precision planar targets, even if now inaccurate, unmeasured, roughly planar targets can be used. The work builds on a recent trend in camera calibration, namely concurrent optimization of scene structure together with the intrinsic camera parameters. A novel formulation is presented that allows maximum likelihood estimation in the case of inaccurate targets, as it extends the camera extrinsic parameters into a tight parametrization of the whole scene structure. It furthermore observes the special characteristics of multi-view perspective projection of planar targets. Its natural extensions to stereo camera calibration and hand-eye calibration are also presented. Experiments demonstrate improvements in the parametrization of the camera model as well as in eventual stereo reconstruction.
This paper presents a calibration method for eye-in-hand systems in order to estimate the hand-eye and the robot-world transformations. The estimation takes place in terms of a parametrization of a stochastic model. In order to perform optimally, a metric on the group of the rigid transformations SE(3) and the corresponding error model are proposed for nonlinear optimization. This novel metric works well with both common formulations AX=XB and AX=ZB, and makes use of them in accordance with the nature of the problem. The metric also adapts itself to the system precision characteristics. The method is compared in performance to earlier approaches
In this paper, we present a novel concept for the detection of loading positions of parcels or bins on a pallet to enable automated order picking using knowledge about the packing pattern model. The approach comprises (1) a new combination of pattern model data and PMD-camera-generated point clouds and (2) a novel concept of RFID data management using a Binary data on Tag / Schema on Net and semantic coding approach. The latter enables the use of additional services like storing of loading positions on auto-id devices (RFID-tags) in a wider concept of the Internet of Things, while the former presents an alternative approach in the context of contour check and position detection of unit loads for automated de-palletizing.
Robotics-logistics can be understood as the field of activities in which applications of industrial robot-technologies are offered and demanded in order to ensure the optimization of internal material flows. According to a survey conducted by BIBA in 2007, the area of robotics-logistics has great need for modernization. Robotics-Logistics activities can be classified in several scenarios as unloading / loading of goods and palletizing / depalletizing of goods. Possible scenarios for research and development activities within theses fields are given.
In the paper-container industry, bag stacking and un-stacking are very labor-intensive work. It is hard for companies to find enough people to fill these positions. Also the repetitive stack and un-stack work can easily cause back and waist injury. Therefore robot de-palletizing system is highly desirable. Guiding a robot tool reliably and robustly to insert into the gap on bag stack to pick up a layer of bags without disturbing the stack is highly challenging due to the variation of the gap-center position and gap size under different pressure depending upon the number of layers above it, the so-called ¿variable crunch¿ factor. In this paper, the method combining an uncalibrated vision and 3D laser-assisted image analysis based on camera-space manipulation (CSM) is developed. The developed prototype system demonstrates the reliable gap insertion in de-palletizing process. It is ready to be installed to a factory floor at the Smurfit-Stone Container Corporation.