Conference PaperPDF Available

Method for the Evaluation of An Autonomous Handling System For Improving the Process Efficiency of Container Unloading

Authors:

Abstract and Figures

Rising trade volume creates an increasing need for automatic unloading solutions for containers. Some systems are already on the market but not widely used due to lack of robustness and difficult-to-predict performance. We present the first approach towards a universal estimation of unloading performance and apply it to a new system. We divide the unloading process into five steps, made up of six individual tasks, and present the ten parameters affecting these tasks. We show how the total unloading time and performance can be calculated based on the task times, reducing the number of necessary tests. Using this method, we calculate the unloading performance of a system gripping multiple cartons. The estimated performance ranges from 341 to 3,252 cartons per hour. This shows that for many systems, the unloading performance depends on multiple parameters. We anticipate this contribution to serve as the first step towards a standardized calculation of unloading performance for containers.
Content may be subject to copyright.
The 2021 International Scientific Symposium on Logistics
is a joint event of Bundesvereinigung Logistik
and Fraunhofer Institute for Material Flow and Logistics.
Method for the Evaluation of
An Autonomous Handling System
For Improving the Process Efficiency of Container Unloading
Jasper Wilhelm, Research Associate,
BIBA - Bremer Institut für Produktion und Logistik GmbH, University of Bremen, Germany
Nils Hendrik Hoppe, Research Associate,
Faculty of Production Engineering, University of Bremen, Germany
Paul Kreuzer, Senior Developer,
Framos GmbH, Taufkirchen, Germany
Christoph Petzoldt, Department Head,
BIBA - Bremer Institut für Produktion und Logistik GmbH, University of Bremen, Germany
Lennart Rolfs, Research Associate,
BIBA - Bremer Institut für Produktion und Logistik GmbH, University of Bremen, Germany
Michael Freitag, Director,
BIBA - Bremer Institut für Produktion und Logistik GmbH, University of Bremen, Germany,
and Professor, Faculty of Production Engineering, University of Bremen, Germany
Summary. Rising trade volume creates an increasing need for automatic unloading solutions
for containers. Some systems are already on the market but not widely used due to lack of
robustness and difficult-to-predict performance. We present the first approach towards a uni-
versal estimation of unloading performance and apply it to a new system. We divide the un-
loading process into five steps, made up of six individual tasks, and present the ten parameters
affecting these tasks. We show how the total unloading time and performance can be calculated
based on the task times, reducing the number of necessary tests. Using this method, we
calculate the unloading performance of a system gripping multiple cartons. The estimated
performance ranges from 341 to 3,252 cartons per hour. This shows that for many systems, the
unloading performance depends on multiple parameters. We anticipate this contribution to serve
as the first step towards a standardized calculation of unloading performance for containers.
1. Introduction
International trade volume is steadily rising (World Trade Organization 2020, 12). The majority
of this cargo is transported in containers, mainly by ship, and packed and emptied in the hinter-
land (United Nations 2020, 9). To increase transport efficiency, both on ship and truck, cargo in
Wilhelm, Hoppe et al.
14 2021 International Scientific Symposium on Logistics
containers are loaded with individual cartons instead of pallets, complicating the emptying of
containers (Bortfeldt and Wäscher 2013; Zhao et al. 2016).
Several automatic handling solutions for unloading containers are available on the market but
are not economical, as fully autonomous solutions often fail due to the varying loading patterns
(Wilhelm, Beinke, and Freitag 2020). Existing solutions either pick items individually, limiting
the potential throughput, or handle cartons in bulk, potentially damaging fragile goods. In this
contribution, we present an overview of available systems for container unloading and provide
a first evaluate of the unloading performance of a newly developed solution for the container
unloading. We propose a process segmentation of unloading-tasks for an objective calculation
of unloading performance and robustness in different scenarios. We evaluate this method on a
newly developed system for the autonomous unloading of containers.
The remainder of this work is organized as follows. Section 2 reviews currently available con-
tainer-unloading systems and describes a semi-autonomous system recently developed by the
authors. The method for evaluating the unloading system and its performance is presented in
Section 3. In Section 4 we describe the test-bed and experiments and present the resulting task
time and unloading performance. Section 5 concludes this article and presents both future work
and further perspectives.
2. State of the Art and System Description
2.1 State of the Art
Recent overviews of autonomous unloading systems for stacked cartons in containers are pre-
sented in Wilhelm, Beinke, and Freitag (2020) and Freitag et al. (2020). Despite the large
number of unloading systems for containers or trucks, none of these solutions have achieved
widespread use yet (Wilhelm, Beinke, and Freitag 2020). Reasons are the high variability of
packing patterns and short process times, which is currently hard to achieve for fully autonomous
systems (Petzoldt et al. 2020). Especially in complex scenarios this leads to system downtimes
and costly manual interventions (Freitag et al. 2020).
The available solutions can be classified by various characteristics (Petzoldt et al. 2020; Freitag
et al. 2020). One feature that directly affects the process time is the type of unloading. Tech-
niques are the individual picking of items (Boston Dynamics 2021; Bastian Solutions 2018;
Stoyanov et al. 2016), gripping of multiple cartons stored in a row (Honeywell Intelligrated
2019), and bulk-unloading of the entire content of the container on conveyor belts (Siemens
Logistics GmbH 2019; Honeywell Intelligrated 2019). In the bulk-unloading scenario, the system
does not pick up individual items, but unloads the entire cargo of the container via conveyor
technology in the floor, potentially damaging the cartons due to falls. The unloading speed
ranges from 500 (Echelmeyer, Bonini, and Rohde 2014) to 1,000 cartons per hour (Bastian
Solutions 2018) for individually picked items to 25,000 cartons per hour in the bulk unloading
scenario (Siemens Logistics GmbH 2019). The unloading performance of human operators
ranges from 420 to 840 cartons per hour (Petzoldt et al. 2020). The unloading speed for all
systems presented are extracted from commercial publications and therefore not reviewed. An
independent evaluation or methods for calculating the unloading speeds are not available.
Method for the Evaluation of an Autonomous Handling System
Conference Volume 15
Unloading Type
Unloading speed in cartons/h
Manual
420 .. 840
Petzoldt et al. 2020
Individual picking
500
1,000
Echelmeyer, Bonini, and Rohde 2014;
Bastian Solutions 2018
Multi-grip
1,500
* Krantz 2021
Bulk
1,500
25,000
* Krantz 2021;
Siemens Logistics GmbH 2019
Table 1. Performance of different unloading methods
* values given as the upper bound for a system
that can perform both multi-grip and bulk-unloading
2.2 System for Multi-Grip Unloading
The method for the calculation of unloading times will be evaluated on a newly developed system
for unloading loose-loaded containers in which the cargo is stacked in multiple layers (Petzoldt
et al. 2020). To improve the throughput over unloading systems that pick items one at a time,
the authors proposed a solution the unloading multiple items packed in a row (Petzoldt et al.
2020). This increases the unloading performance without high impact-loads on the items as in
the bulk-unloading scenario.
The systems consists of a omnidirectional mobile chassis, a vertically moveable platform with
tilt-adjustment, and three individually movable gripping-modules with vacuum suction cups to
grip and pull cartons. The platform and center of the robot are equipped with conveyors to move
the unloaded items to external material-handling technology at the back of the system. Figure 1
highlights the controllable parts of the system.
Figure 1. Unloading system with its degrees of freedom modules
(in accordance with Petzoldt et al. 2020)
The system is equipped with an array of four RGB-D cameras, placed at the top and bottom, left
and right corners of the vehicle. To increase robustness, the detection process runs in-
dependently on the four individual camera frames. After the detection of the box corners in the
3D color-frames, the positions of the corners are transformed into the common coordinate
system of the robot. For carton identification we adapted a methodology based on deep neural
networks (DNNs) which has been very successful in 2D Multi-Person Skeleton Estimation (Gong
et al. 2016; Chen, Tian, and He 2020; Xiao, Wu, and Wei 2018; Cao et al. 2021). In this scenario,
Wilhelm, Hoppe et al.
16 2021 International Scientific Symposium on Logistics
the four front-facing corners of the cartons present the skeleton to be detected. The key
advantage of this concept over traditional computer vision methods is the long-term sustain-
ability. This approach allows for a quick re-training in case the type of cargo changes, compared
to traditional methods, in which an expert would have to adjust or even re-develop the
algorithms.
2.3 Unloading Process
The unloading process performed by the robot consists of multiple steps, each built from unique
tasks (Hoppe et al. 2020). Table 2 presents the list of process steps and their corresponding
tasks. First, an array of four depth-cameras scans the area in front of the robot. A carton de-
tection algorithm identifies the individual cartons in each of these four images and creates a
skeleton representation of all identified objects. After merging this array of four skeletal images,
the cartons for the next grip are chosen based on their reachability and the optimal unloading
pose of the robot is calculated. Second, the robot approaches the cartons by moving its chassis
and platform concurrently. To grab the cartons, the robot moves the gripping-modules to the
front of the platform and starts the vacuum once the carton-front is in proximity of the suction
cups. The vacuum is individually monitored and controlled so that the robot can distinguish
between failed and successful grips. Once either all packages have been successfully grabbed or
non-gripping suction cups are deactivated, the gripping-modules move to the rear. The robot
switches off the vacuum and moves the gripper modules to their rest position, with the center
module sinking below the conveyor belts. Finally, the conveyor modules unload all cartons pulled
onto the platform.
Step
Tasks ()
Conc.+
1
Identify
Carton detection (1), Carton selection*, Pose calculation*
2
Approach
Chassis motion (2), Platform motion (3)
yes
3
Grab
Gripper motion (4), Vacuum effect (5)
4
Pull
Gripper motion (4), Vacuum effect (5)
5
Convey
Conveyor motion (6)
Table 2. Individual tasks of the system to be tested
based on the unloading process
* task time and robustness independent of parameters,
negligible impact on unloading time
+ Tasks in this step are performed concurrently
3. Method
3.1 Experimental Design
Before the performance of the system is determined under real conditions in a field-test, inte-
gration and laboratory tests are performed to provide an initial estimate. We place particular
emphasis on the coverage of as many potential scenarios as possible and the determination of
the robustness of the solution.
Method for the Evaluation of an Autonomous Handling System
Conference Volume 17
To minimize testing and create universally valid results, we propose a modular design of
experiments. The separation of the overall process into individual steps and their basic tasks
allows to specifically test each task with a reduced number of parameters. For each step only
the relevant parameters of the tasks need to be adapted. The unloading time for one row of
cartons is the sum of the times of the different steps based on the individual parameters of
their underlying tasks . In first approximation, each step is performed once for each row, since
we assume that all cartons of a row are unloaded at once.
        
 (1)
with
        

     (2a)
    

       (2b)
 is the set of all steps with sequential subtasks (see Table 2).
Due to the multi-grip performed by the system, the total unloading time for a given container is
unloading time of an individual row times the total number of rows in a container. Assuming
homogeneously stacked cartons, the number of rows in a container is the number of layers
in length times the number of layers in height. Therefore, the total unloading time is
   (3)
3.2 Parameter Identification
In a first step, we identified the external parameters of influence for each task by interviewing
system and process experts. For each task, we deducted their elementary parameters from first-
order principles and the expert evaluations. Thus, we can vary only task-relevant parameters,
drastically reducing the number of tests necessary for each task. By combining the individual
times with parameters of the unloading process (e.g., carton size), we can estimate the overall
unloading time for varying conditions. Table 3 shows the relevant parameters affecting unloading
performance and robustness for each task.
Task
Performance factors
Carton Detection
object size (carton width, carton height), lightning (brightness,
contrast), refractions, reflection (carton surface)
Chassis motion
distance (carton depth), resistance (floor inclination)
Platform Motion
distance (carton height), resistance (carton mass)
Gripper Motion
resistance (carton mass, platform inclination)
Control Vacuum
resistance (carton surface: porosity, carton mass, platform
inclination)
Control Conveyor
distance, resistance (carton mass)
Table 3. Parameters affecting unloading performance and robustness of tasks
In the second step, we designed individual tests to analyze the effect of the corresponding
parameters. For each parameter, we created discrete variations described in Table 4.
Wilhelm, Hoppe et al.
18 2021 International Scientific Symposium on Logistics
Parameter
Parameter value
1
Carton surface
matte; laminated
2
Brightness
no ambient lighting; bright ambient lightning
3
Contrast
equally distributed light; mixed lighting (spots)
4
Atmosphere
no fog/dust (clean laboratory); Reduced visibility (fog)
5
Floor inclination
-4%; 4%
6
Platform inclination
0%; 40%
7
Carton width
200 mm; 800 mm
8
Carton depth
200 mm; 800 mm
9
Carton height
200 mm; 800 mm
10
Carton mass
0 kg; 35 kg
Table 4. Range of values for the parameters
4 Experiments and results
For this contribution, we identified the times of the tasks that significantly affect the unloading
time (tasks 2, 3, 4, 6). Each task was performed ten times for each combination of parameters.
Since all controllers are velocity controller, we only changed the parameters affecting the dis-
tance. The task time for the conveyor motion was determined on theoretical grounds. The total
distance of  can be covered in  assuming a conveyor speed of 
. We performed all
tests in a laboratory test-bed with a container of cartons of different sizes.
4.1 Preliminary Task Times
Table 5 lists the results of the experiments performed. It presents the mean time and its devi-
ation for the slowest and fastest combination of parameters for each task. The total unloading
time is given by the sum of all steps of the unloading process (Eq. 1). The time of each step
is defined by the total time or maximum time of all tasks    as given in Table 2 (Eq. 2). Table
5 gives the minimal and maximal time of step task.
Task
 in s
 in s
2
Chassis motion
4.0
12.0
3
Platform Motion
4.9
13.3
4
Gripper Motion
6.4
6.4
6
Control Conveyor
5.6
5.6
Table 5. Preliminary results of the unloading task times. The time for the
conveyor motion was determined theoretically
Method for the Evaluation of an Autonomous Handling System
Conference Volume 19
4.2 Performance Evaluation
The unloading performance is defined as the number of cartons per time (Table 1). The number
of cartons per container depends on the size of the cartons. The size of conventional cartons are
between 300×200×100 mm (small) and 800×640×600 mm (large). Therefore, the maximum
number of cartons in a 1AA 40-feet container
1
is 9,009 with the long side oriented to the back.
The maximum number of large cartons is 126 also with the long side oriented to the back.
Table 6 presents the different scenarios and the stacking pattern for these scenarios and the
estimated unloading performance. The total unloading time is calculated with Eq. (3). It should
be noted that chassis and platform motion are performed concurrently and the gripper motion
is performed twice, both when gripping and unloading.
Carton
size
Number of
cartons in
length
Number of
cartons in
width
Number of
cartons in
height
Total
number of
cartons
Unloading
time 
per row in s
Unloading
performance
in carton/h
Small
39
21
11
9,009
23.2
3,252
Large
14
3
3
126
31.7
341
Table 6. Carton pattern for standard sized containers and the
preliminary unloading performance
4.3 Limitations
The aforementioned test-setup allows for a flexible test of multiple criteria with a reduced over-
head due to the evaluation of individual tasks. With equations (13), the final unloading time
can be evaluated for a wide range of scenarios. This flexibility comes at the cost of lacking full-
service evaluations. The estimated times are the result of distinct tests and present only an
expected value for the total unloading time under various conditions. Since we did not perform
full factorial tests and so far only tested for parcel size and mass, potential correlation between
the parameters might affect the unloading performance. Additionally, the conveyor time was
estimated based on conveyor velocity.
Additional effects will be evaluated in a field-test. There, actual 40ft-containers in the receiving
area of a large logistics service provider will serve as the testbed for the system. With this test,
we will evaluate the robustness over longer periods of time as well as the systems approach to
unforeseen situations.
5. Summary
This paper presents a list of available solutions and their performance as well as a first approach
towards a standardized evaluation of unloading throughput for automatic unloading solutions.
In the presented method for throughput estimation, the process is divided into multiple steps
and the tasks in each steps are evaluated in terms of performance and robustness. We propose
1
internal dimensions of 11.998 × 2.330 × 2.350 m (International Organization for Standardization 2020)
Wilhelm, Hoppe et al.
20 2021 International Scientific Symposium on Logistics
distinct tests for each task under variation of all parameters affecting performance and robust-
ness and evaluate this division on a new unloading system, resulting in a setup with five different
tasks and ten parameters.
In a first test, we estimate the unloading performance of the system to range from 341 cartons
per hour for very large items to over 3,200 cartons per hour for small items. Next, we will
evaluate all parameters affecting robustness in a laboratory environment. In field-tests we will
evaluate the robustness and performance of the system under varying conditions.
Acknowledgments
This work was supported by the German Federal Ministry of Transport and Digital Infrastructure
(BMVI) under Grant 19H17016C.
References
Bastian Solutions. 2018. “Robotic Truck Trailer Case Loader/Unloader.” Indianapolis.
https://www.bastiansolutions.com/solutions/service/industrial-robotics/industrial-robotic-
solutions/mobile-robotics/robotic-truck-unloading-loading/.
Bortfeldt, Andreas, and Gerhard Wäscher. 2013. “Constraints in Container Loading – A State-of-
the-Art Review.” European Journal of Operational Research 229 (1): 120. https://doi.org/
10.1016/j.ejor.2012.12.006.
Boston Dynamics. 2021. “Boston dynamics unveils new robot for warehouse automation.” 2021.
https://www.bostondynamics.com/new-robot-for-warehouse-automation.
Cao, Zhe, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2021. “OpenPose: Real-
time Multi-Person 2D Pose Estimation Using Part Affinity Fields.” IEEE Transactions on
Pattern Analysis and Machine Intelligence 43 (1): 17286. https://doi.org/10.1109/
TPAMI.2019.2929257.
Chen, Yucheng, Yingli Tian, and Mingyi He. 2020. “Monocular Human Pose Estimation: A Survey
of Deep Learning-Based Methods.” Computer Vision and Image Understanding 192
(March): 102897. https://doi.org/10.1016/j.cviu.2019.102897.
Echelmeyer, Wolfgang, Marco Bonini, and Moritz Rohde. 2014. “From Manufacturing to Logistics:
Development of a Kinematic for Autonomous Unloading of Containers.” Advanced Materials
Research 903: 24551. https://doi.org/10.4028/www.scientific.net/AMR.903.245.
Freitag, Michael, Nils Hoppe, Christoph Petzoldt, Jasper Wilhelm, Lennart Rolfs, Rafael Mortensen
Ernits, and Thies Beinke. 2020. “Digitaler Zwilling Zur Mensch-Technik-Interaktion.” In
Mensch-Technik-Interaktion in Der Digitalisierten Arbeitswelt, 16582. GITO Verlag.
https://doi.org/10.30844/wgab_2020_8.
Garrido-Jurado, S., R. Muñoz-Salinas, F.J. Madrid-Cuevas, and M.J. Marín-Jiménez. 2014.
“Automatic Generation and Detection of Highly Reliable Fiducial Markers under Occlusion.”
Pattern Recognition 47 (6): 228092. https://doi.org/10.1016/j.patcog.2014.01.005.
Method for the Evaluation of an Autonomous Handling System
Conference Volume 21
Gong, Wenjuan, Xuena Zhang, Jordi Gonzàlez, Andrews Sobral, Thierry Bouwmans, Changhe
Tu, and El-hadi Zahzah. 2016. “Human Pose Estimation from Monocular Images: A
Comprehensive Survey.” Sensors 16 (12): 1966. https://doi.org/10.3390/s16121966.
Honeywell Intelligrated. 2019. “Robotic Unloader.” Mason. https://www.intelligrated.com/
en/solutions/technology/robotics/robotic-unloader.
Hoppe, Nils, Jasper Wilhelm, Christoph Petzoldt, Lennart Rolfs, Thies Beinke, and Michael Freitag.
2020. “Design Eines Robotiksystems Zur Entleerung VonSeecontainern.” Logistics Journal
Proceedings. Wissenschaftliche Gesellschaft Für Technische Logistik (WGTL), 112.
International Organization for Standardization. 2020. “Series 1 Freight Containers Classi-
fication, Dimensions and Ratings (ISO Standard No. 668:2020).” https://www.iso.org/
standard/76912.html.
Krantz, Eric. 2021. “Honeywell Announces Fully Automated Robotic Unloader for Distribution
Centers”. https://www.honeywell.com/us/en/press/2019/04/honeywell-announces-fully-
automated-robotic-unloader-for-distribution-centers.
Lin, Tsung-yi, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr
Dollár, and C Lawrence Zitnick. 2014. “Microsoft COCO: Common Objects in Context.” In
Computer Vision ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, Vol 8693.,
edited by D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, 74055. Springer, Cham.
https://doi.org/10.1007/978-3-319-10602-1_48.
Petzoldt, Christoph, Jasper Wilhelm, Nils Hendrik Hoppe, Lennart Rolfs, Thies Beinke, and
Michael Freitag. 2020. “Control Architecture for Digital Twin-Based Human-Machine
Interaction in a Novel Container Unloading System.” Procedia Manufacturing 52: 21520.
https://doi.org/10.1016/j.promfg.2020.11.037.
Plackett, R. L., and J. P. Burman. 1946. “The Design of Optimum Multifactorial Experiments.”
Biometrika 33 (4): 30525. https://doi.org/10.1093/biomet/33.4.305.
Siemens Logistics GmbH. 2019. “Siemens Revolutionizes Parcel Unloading.” 2019. https://www.
siemens-logistics.com/en/news/press-releases/siemens-revolutionizes-parcel-unloading.
Stoyanov, Todor, Narunas Vaskevicius, Christian A. Mueller, Tobias Fromm, Robert Krug, Vinicio
Tincani, Rasoul Mojtahedzadeh, et al. 2016. “No More Heavy Lifting: Robotic Solutions to
the Container Unloading Problem.” IEEE Robotics and Automation Magazine 23 (4): 94
106. https://doi.org/10.1109/MRA.2016.2535098.
United Nations. 2020. “Review of Maritime Transport 2020.” Geneva. https://unctad.org/
webflyer/review-maritime-transport-2020.
Wilhelm, Jasper, Thies Beinke, and Michael Freitag. 2020. “Improving Human-Machine Inter-
action with a Digital Twin.” In Dynamics in Logistics. Proceedings of the 7th International
Conference on Dynamics in Logistics (LDIC 2020), ed. by M. Freitag, H.-D. Haasis, H.
Kotzab, and J. Pannek, 52740. Cham: Springer. https://doi.org/10.1007/978-3-030-
44783-0_49.
World Trade Organization. 2020. “World Trade Statistical Review 2020.” Geneva. https://www.
wto.org/english/res_e/statis_e/wts2020_e/wts20_toc_e.htm.
Xiao, Bin, Haiping Wu, and Yichen Wei. 2018. “Simple Baselines for Human Pose Estimation and
Tracking.” ArXiv, 116.
Wilhelm, Hoppe et al.
22 2021 International Scientific Symposium on Logistics
Zhao, Xiaozhou, Julia A. Bennell, Tolga Bektaş, and Kath Dowsland. 2016. “A Comparative
Review of 3D Container Loading Algorithms.” International Transactions in Operational
Research 23 (12): 287320. https://doi.org/10.1111/itor.12094.
... Due to the different ways tasks are processed, this approach can only be applied to machines to a limited extent [20]. In [21], a to their resting position, with the middle module sinking below the conveyor belts. Finally, the conveyor modules unload all the parcels that are pulled onto the platform to the back of the robot (convey), where an external material-handling technology is installed on the robot. ...
... As presented in [21], the total unloading time is the result of the individual unloading steps. Assuming the parcels are loaded homogeneously, the number of rows in a container is the number of layers n x in length times the number of rows n z . ...
... By combining the individual times with the parameters of the unloading process (e.g., parcel size), we can estimate the total unloading time for different conditions. According to [21], the parcel size is the most affecting parameter in a majority of tasks to be tested. ...
Article
Full-text available
In the business-to-business sector large quantities of the internationally traded goods are transported by hundreds in identical parcels inside shipping containers. These parcels are often unloaded manually, which is labor-intensive and physically demanding. In previous papers, we presented a novel semi-autonomous container unloading system and an estimation of the performance. In this paper we evaluate the proposed system. Tests are performed in a laboratory testbed with a container and different sized parcels. We use this testbed to record the times for individual tasks of the system as well as the acceleration of the parcels. For comparison, we test the acceleration in the manual and bulk unloading scenario. Following a line-by-line approach, the proposed system’s performance ranges from 261 to 3,014 parcels per hour, between manual and bulk performance. The throughput depends on the parcel’s size and is higher for small parcels. With a maximum shock on the parcels of less than the proposed system induces significantly less force than other approaches (manual and bulk).
Article
Full-text available
To facilitate physically exhausting tasks, semi-autonomous systems are introduced at manual industrial workplaces. This changes the role of the operator from a manual worker to a supervisor and intervener in case of problems. New human-machine interaction strategies providing operators with the means to solve problems are required to enable this role change seamlessly. For this purpose, we propose a novel control architecture based on a digital twin, present a newly developed semi-autonomous system for the industrial sector of container unloading, and apply the presented control architecture hereupon. Besides, a morphology for classification of unloading systems is proposed.
Article
Full-text available
Human pose estimation refers to the estimation of the location of body parts and how they are connected in an image. Human pose estimation from monocular images has wide applications (e.g., image indexing). Several surveys on human pose estimation can be found in the literature, but they focus on a certain category; for example, model-based approaches or human motion analysis, etc. As far as we know, an overall review of this problem domain has yet to be provided. Furthermore, recent advancements based on deep learning have brought novel algorithms for this problem. In this paper, a comprehensive survey of human pose estimation from monocular images is carried out including milestone works and recent advancements. Based on one standard pipeline for the solution of computer vision problems, this survey splits the problem into several modules: feature extraction and description, human body models, and modeling methods. Problem modeling methods are approached based on two means of categorization in this survey. One way to categorize includes top-down and bottom-up methods, and another way includes generative and discriminative methods. Considering the fact that one direct application of human pose estimation is to provide initialization for automatic video surveillance, there are additional sections for motion-related methods in all modules: motion features, motion models, and motion-based methods. Finally, the paper also collects 26 publicly available data sets for validation and provides error measurement methods that are frequently used.
Article
Full-text available
This article discusses the scientifically and industrially important problem of automating the process of unloading goods from standard shipping containers. We outline some of the challenges barring further adoption of robotic solutions to this problem: ranging from handling a vast variety of shapes, sizes, weights, appearance and packing arrangement of the goods, through hard demands on unloading speed and reliability, to ensuring fragile goods are not damaged. We propose a modular and reconfigurable software framework in an attempt at efficiently addressing some of these challenges. We outline the general framework design, as well as the basic functionality of the core modules developed and present two instantiations of the software system on two different fully integrated demonstrators. While one is coping with an industrial scenario, namely the automated unloading of coffee sacks, with an already econom ically interesting performance, the other scenario is used to demonstrate the capabilities of our scientific and technological developments in the context of medium- to long-term prospects of automation in logistics. We performed evaluations which allow us to summarize several important lessons learned and to identify future directions of research on autonomous robots for handling of goods in logistics applications.
Article
Full-text available
In many branches of production and manufacturing the level of automation has reached saturation. Further automation would be possible but would also affect negatively their economic efficiency. On the contrary, in logistics the level of automation is low due to the lack of process standardization. This is caused on the one hand by the high number of protagonists involved in global supply chains and on the other hand by the weak demand for standardization that characterizes areas with low costs of labor. Among the logistics processes the highest challenge for automation is the unloading of goods from carriers, i. e. containers, which are used for international and intermodal transports. In containers a wide range of goods is transported among which, unpalettized mass goods have the highest relevance for autonomous unloading, because of their size, shape and weight. Mass goods are arbitrarily packed neatly or loose, stuck or slack patterns and with respect to their properties homogeneous or heterogeneous. Nowadays these containers are unloaded mostly manually. A system for the autonomous unloading of all the kinds of mass goods from containers doesnt exist as a market ready product. This paper presents a kinematic suited to unload all kinds of mass goods from a container.
Conference Paper
Die Entladung von Seecontainern erfolgt in der Regel manuell. Die körperlich anstrengende und monotone Arbeit führt hierbei zu einer hohen Mitarbeiterfluktuation. Autonome Entladesysteme konnten sich zur Reduktion dieses Unsicherheitsfaktors bisher jedoch nicht etablieren. Der Beitrag geht auf die Rahmenbedingungen und den Prozess der Containerentladung ein. Dazu werden grundlegende Anforderungen beschrieben. Darauf basierend wird ein alternativer Ansatz für die automatisierte Entladung von Seecontainern vorgestellt, wobei einige neu entwickelten Subsysteme detailliert beschrieben werden.
Chapter
The unloading of containers is a tedious task that a decreasing number of workers is willing to take on. (Semi-)autonomous systems are already available but limited to clearly defined scenarios due to a rigid level of autonomy. This paper focuses on an adaptive concept for a human-centered semi-autonomous unloading process. First, available systems are analyzed regarding their level of autonomy and the integration of the human operator. Following this, a concept integrating the digital twin and adaptive automation is presented. The usage of a digital representation with adapting autonomy allows combining the strength of humans and machines. In the presented system, the operator uses his cognitive advantage to provide specific support when the machine reaches its limits.
Article
Vision-based monocular human pose estimation, as one of the most fundamental and challenging problems in computer vision, aims to obtain posture of the human body from input images or video sequences. The recent developments of deep learning techniques have been brought significant progress and remarkable breakthroughs in the field of human pose estimation. This survey extensively reviews the recent deep learning-based 2D and 3D human pose estimation methods published since 2014. This paper summarizes the challenges, main frameworks, benchmark datasets, evaluation metrics, performance comparison, and discusses some promising future research directions.
Article
Realtime multi-person 2D pose estimation is a key component in enabling machines to have an understanding of people in images and videos. In this work, we present a realtime approach to detect the 2D pose of multiple people in an image. The proposed method uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn to associate body parts with individuals in the image. This bottom-up system achieves high accuracy and realtime performance, regardless of the number of people in the image. In previous work, PAFs and body part location estimation were refined simultaneously across training stages. We demonstrate that using a PAF-only refinement is able to achieve a substantial increase in both runtime performance and accuracy. We also present the first combined body and foot keypoint detector, based on an annotated foot dataset that we have publicly released. We show that the combined detector not only reduces the inference time compared to running them sequentially, but also maintains the accuracy of each component individually. This work has culminated in the release of OpenPose, the first open-source realtime system for multi-person 2D pose detection, including body, foot, hand, and facial keypoints.
Conference Paper
We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in understanding an object's precise 2D location. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old along with per-instance segmentation masks. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.