Content uploaded by Jasper Wilhelm
Author content
All content in this area was uploaded by Jasper Wilhelm on Jun 28, 2021
Content may be subject to copyright.
The 2021 International Scientific Symposium on Logistics
is a joint event of Bundesvereinigung Logistik
and Fraunhofer Institute for Material Flow and Logistics.
Method for the Evaluation of
An Autonomous Handling System
For Improving the Process Efficiency of Container Unloading
Jasper Wilhelm, Research Associate,
BIBA - Bremer Institut für Produktion und Logistik GmbH, University of Bremen, Germany
Nils Hendrik Hoppe, Research Associate,
Faculty of Production Engineering, University of Bremen, Germany
Paul Kreuzer, Senior Developer,
Framos GmbH, Taufkirchen, Germany
Christoph Petzoldt, Department Head,
BIBA - Bremer Institut für Produktion und Logistik GmbH, University of Bremen, Germany
Lennart Rolfs, Research Associate,
BIBA - Bremer Institut für Produktion und Logistik GmbH, University of Bremen, Germany
Michael Freitag, Director,
BIBA - Bremer Institut für Produktion und Logistik GmbH, University of Bremen, Germany,
and Professor, Faculty of Production Engineering, University of Bremen, Germany
Summary. Rising trade volume creates an increasing need for automatic unloading solutions
for containers. Some systems are already on the market but not widely used due to lack of
robustness and difficult-to-predict performance. We present the first approach towards a uni-
versal estimation of unloading performance and apply it to a new system. We divide the un-
loading process into five steps, made up of six individual tasks, and present the ten parameters
affecting these tasks. We show how the total unloading time and performance can be calculated
based on the task times, reducing the number of necessary tests. Using this method, we
calculate the unloading performance of a system gripping multiple cartons. The estimated
performance ranges from 341 to 3,252 cartons per hour. This shows that for many systems, the
unloading performance depends on multiple parameters. We anticipate this contribution to serve
as the first step towards a standardized calculation of unloading performance for containers.
1. Introduction
International trade volume is steadily rising (World Trade Organization 2020, 12). The majority
of this cargo is transported in containers, mainly by ship, and packed and emptied in the hinter-
land (United Nations 2020, 9). To increase transport efficiency, both on ship and truck, cargo in
Wilhelm, Hoppe et al.
14 2021 International Scientific Symposium on Logistics
containers are loaded with individual cartons instead of pallets, complicating the emptying of
containers (Bortfeldt and Wäscher 2013; Zhao et al. 2016).
Several automatic handling solutions for unloading containers are available on the market but
are not economical, as fully autonomous solutions often fail due to the varying loading patterns
(Wilhelm, Beinke, and Freitag 2020). Existing solutions either pick items individually, limiting
the potential throughput, or handle cartons in bulk, potentially damaging fragile goods. In this
contribution, we present an overview of available systems for container unloading and provide
a first evaluate of the unloading performance of a newly developed solution for the container
unloading. We propose a process segmentation of unloading-tasks for an objective calculation
of unloading performance and robustness in different scenarios. We evaluate this method on a
newly developed system for the autonomous unloading of containers.
The remainder of this work is organized as follows. Section 2 reviews currently available con-
tainer-unloading systems and describes a semi-autonomous system recently developed by the
authors. The method for evaluating the unloading system and its performance is presented in
Section 3. In Section 4 we describe the test-bed and experiments and present the resulting task
time and unloading performance. Section 5 concludes this article and presents both future work
and further perspectives.
2. State of the Art and System Description
2.1 State of the Art
Recent overviews of autonomous unloading systems for stacked cartons in containers are pre-
sented in Wilhelm, Beinke, and Freitag (2020) and Freitag et al. (2020). Despite the large
number of unloading systems for containers or trucks, none of these solutions have achieved
widespread use yet (Wilhelm, Beinke, and Freitag 2020). Reasons are the high variability of
packing patterns and short process times, which is currently hard to achieve for fully autonomous
systems (Petzoldt et al. 2020). Especially in complex scenarios this leads to system downtimes
and costly manual interventions (Freitag et al. 2020).
The available solutions can be classified by various characteristics (Petzoldt et al. 2020; Freitag
et al. 2020). One feature that directly affects the process time is the type of unloading. Tech-
niques are the individual picking of items (Boston Dynamics 2021; Bastian Solutions 2018;
Stoyanov et al. 2016), gripping of multiple cartons stored in a row (Honeywell Intelligrated
2019), and bulk-unloading of the entire content of the container on conveyor belts (Siemens
Logistics GmbH 2019; Honeywell Intelligrated 2019). In the bulk-unloading scenario, the system
does not pick up individual items, but unloads the entire cargo of the container via conveyor
technology in the floor, potentially damaging the cartons due to falls. The unloading speed
ranges from 500 (Echelmeyer, Bonini, and Rohde 2014) to 1,000 cartons per hour (Bastian
Solutions 2018) for individually picked items to 25,000 cartons per hour in the bulk unloading
scenario (Siemens Logistics GmbH 2019). The unloading performance of human operators
ranges from 420 to 840 cartons per hour (Petzoldt et al. 2020). The unloading speed for all
systems presented are extracted from commercial publications and therefore not reviewed. An
independent evaluation or methods for calculating the unloading speeds are not available.
Method for the Evaluation of an Autonomous Handling System
Conference Volume 15
Unloading Type
Unloading speed in cartons/h
Manual
420 .. 840
Petzoldt et al. 2020
Individual picking
500
1,000
Echelmeyer, Bonini, and Rohde 2014;
Bastian Solutions 2018
Multi-grip
1,500
* Krantz 2021
Bulk
1,500
25,000
* Krantz 2021;
Siemens Logistics GmbH 2019
Table 1. Performance of different unloading methods
* values given as the upper bound for a system
that can perform both multi-grip and bulk-unloading
2.2 System for Multi-Grip Unloading
The method for the calculation of unloading times will be evaluated on a newly developed system
for unloading loose-loaded containers in which the cargo is stacked in multiple layers (Petzoldt
et al. 2020). To improve the throughput over unloading systems that pick items one at a time,
the authors proposed a solution the unloading multiple items packed in a row (Petzoldt et al.
2020). This increases the unloading performance without high impact-loads on the items as in
the bulk-unloading scenario.
The systems consists of a omnidirectional mobile chassis, a vertically moveable platform with
tilt-adjustment, and three individually movable gripping-modules with vacuum suction cups to
grip and pull cartons. The platform and center of the robot are equipped with conveyors to move
the unloaded items to external material-handling technology at the back of the system. Figure 1
highlights the controllable parts of the system.
Figure 1. Unloading system with its degrees of freedom modules
(in accordance with Petzoldt et al. 2020)
The system is equipped with an array of four RGB-D cameras, placed at the top and bottom, left
and right corners of the vehicle. To increase robustness, the detection process runs in-
dependently on the four individual camera frames. After the detection of the box corners in the
3D color-frames, the positions of the corners are transformed into the common coordinate
system of the robot. For carton identification we adapted a methodology based on deep neural
networks (DNNs) which has been very successful in 2D Multi-Person Skeleton Estimation (Gong
et al. 2016; Chen, Tian, and He 2020; Xiao, Wu, and Wei 2018; Cao et al. 2021). In this scenario,
Wilhelm, Hoppe et al.
16 2021 International Scientific Symposium on Logistics
the four front-facing corners of the cartons present the skeleton to be detected. The key
advantage of this concept over traditional computer vision methods is the long-term sustain-
ability. This approach allows for a quick re-training in case the type of cargo changes, compared
to traditional methods, in which an expert would have to adjust or even re-develop the
algorithms.
2.3 Unloading Process
The unloading process performed by the robot consists of multiple steps, each built from unique
tasks (Hoppe et al. 2020). Table 2 presents the list of process steps and their corresponding
tasks. First, an array of four depth-cameras scans the area in front of the robot. A carton de-
tection algorithm identifies the individual cartons in each of these four images and creates a
skeleton representation of all identified objects. After merging this array of four skeletal images,
the cartons for the next grip are chosen based on their reachability and the optimal unloading
pose of the robot is calculated. Second, the robot approaches the cartons by moving its chassis
and platform concurrently. To grab the cartons, the robot moves the gripping-modules to the
front of the platform and starts the vacuum once the carton-front is in proximity of the suction
cups. The vacuum is individually monitored and controlled so that the robot can distinguish
between failed and successful grips. Once either all packages have been successfully grabbed or
non-gripping suction cups are deactivated, the gripping-modules move to the rear. The robot
switches off the vacuum and moves the gripper modules to their rest position, with the center
module sinking below the conveyor belts. Finally, the conveyor modules unload all cartons pulled
onto the platform.
Step
Tasks ()
Conc.+
1
Identify
Carton detection (1), Carton selection*, Pose calculation*
2
Approach
Chassis motion (2), Platform motion (3)
yes
3
Grab
Gripper motion (4), Vacuum effect (5)
4
Pull
Gripper motion (4), Vacuum effect (5)
5
Convey
Conveyor motion (6)
Table 2. Individual tasks of the system to be tested
based on the unloading process
* task time and robustness independent of parameters,
negligible impact on unloading time
+ Tasks in this step are performed concurrently
3. Method
3.1 Experimental Design
Before the performance of the system is determined under real conditions in a field-test, inte-
gration and laboratory tests are performed to provide an initial estimate. We place particular
emphasis on the coverage of as many potential scenarios as possible and the determination of
the robustness of the solution.
Method for the Evaluation of an Autonomous Handling System
Conference Volume 17
To minimize testing and create universally valid results, we propose a modular design of
experiments. The separation of the overall process into individual steps and their basic tasks
allows to specifically test each task with a reduced number of parameters. For each step only
the relevant parameters of the tasks need to be adapted. The unloading time for one row of
cartons is the sum of the times of the different steps based on the individual parameters of
their underlying tasks . In first approximation, each step is performed once for each row, since
we assume that all cartons of a row are unloaded at once.
(1)
with
(2a)
(2b)
is the set of all steps with sequential subtasks (see Table 2).
Due to the multi-grip performed by the system, the total unloading time for a given container is
unloading time of an individual row times the total number of rows in a container. Assuming
homogeneously stacked cartons, the number of rows in a container is the number of layers
in length times the number of layers in height. Therefore, the total unloading time is
(3)
3.2 Parameter Identification
In a first step, we identified the external parameters of influence for each task by interviewing
system and process experts. For each task, we deducted their elementary parameters from first-
order principles and the expert evaluations. Thus, we can vary only task-relevant parameters,
drastically reducing the number of tests necessary for each task. By combining the individual
times with parameters of the unloading process (e.g., carton size), we can estimate the overall
unloading time for varying conditions. Table 3 shows the relevant parameters affecting unloading
performance and robustness for each task.
Task
Performance factors
1
Carton Detection
object size (carton width, carton height), lightning (brightness,
contrast), refractions, reflection (carton surface)
2
Chassis motion
distance (carton depth), resistance (floor inclination)
3
Platform Motion
distance (carton height), resistance (carton mass)
4
Gripper Motion
resistance (carton mass, platform inclination)
5
Control Vacuum
resistance (carton surface: porosity, carton mass, platform
inclination)
6
Control Conveyor
distance, resistance (carton mass)
Table 3. Parameters affecting unloading performance and robustness of tasks
In the second step, we designed individual tests to analyze the effect of the corresponding
parameters. For each parameter, we created discrete variations described in Table 4.
Wilhelm, Hoppe et al.
18 2021 International Scientific Symposium on Logistics
Parameter
Parameter value
1
Carton surface
matte; laminated
2
Brightness
no ambient lighting; bright ambient lightning
3
Contrast
equally distributed light; mixed lighting (spots)
4
Atmosphere
no fog/dust (clean laboratory); Reduced visibility (fog)
5
Floor inclination
-4%; 4%
6
Platform inclination
0%; 40%
7
Carton width
200 mm; 800 mm
8
Carton depth
200 mm; 800 mm
9
Carton height
200 mm; 800 mm
10
Carton mass
0 kg; 35 kg
Table 4. Range of values for the parameters
4 Experiments and results
For this contribution, we identified the times of the tasks that significantly affect the unloading
time (tasks 2, 3, 4, 6). Each task was performed ten times for each combination of parameters.
Since all controllers are velocity controller, we only changed the parameters affecting the dis-
tance. The task time for the conveyor motion was determined on theoretical grounds. The total
distance of can be covered in assuming a conveyor speed of
. We performed all
tests in a laboratory test-bed with a container of cartons of different sizes.
4.1 Preliminary Task Times
Table 5 lists the results of the experiments performed. It presents the mean time and its devi-
ation for the slowest and fastest combination of parameters for each task. The total unloading
time is given by the sum of all steps of the unloading process (Eq. 1). The time of each step
is defined by the total time or maximum time of all tasks as given in Table 2 (Eq. 2). Table
5 gives the minimal and maximal time of step task.
Task
in s
in s
2
Chassis motion
4.0
12.0
3
Platform Motion
4.9
13.3
4
Gripper Motion
6.4
6.4
6
Control Conveyor
5.6
5.6
Table 5. Preliminary results of the unloading task times. The time for the
conveyor motion was determined theoretically
Method for the Evaluation of an Autonomous Handling System
Conference Volume 19
4.2 Performance Evaluation
The unloading performance is defined as the number of cartons per time (Table 1). The number
of cartons per container depends on the size of the cartons. The size of conventional cartons are
between 300×200×100 mm (small) and 800×640×600 mm (large). Therefore, the maximum
number of cartons in a 1AA 40-feet container
1
is 9,009 with the long side oriented to the back.
The maximum number of large cartons is 126 also with the long side oriented to the back.
Table 6 presents the different scenarios and the stacking pattern for these scenarios and the
estimated unloading performance. The total unloading time is calculated with Eq. (3). It should
be noted that chassis and platform motion are performed concurrently and the gripper motion
is performed twice, both when gripping and unloading.
Carton
size
Number of
cartons in
length
Number of
cartons in
width
Number of
cartons in
height
Total
number of
cartons
Unloading
time
per row in s
Unloading
performance
in carton/h
Small
39
21
11
9,009
23.2
3,252
Large
14
3
3
126
31.7
341
Table 6. Carton pattern for standard sized containers and the
preliminary unloading performance
4.3 Limitations
The aforementioned test-setup allows for a flexible test of multiple criteria with a reduced over-
head due to the evaluation of individual tasks. With equations (1–3), the final unloading time
can be evaluated for a wide range of scenarios. This flexibility comes at the cost of lacking full-
service evaluations. The estimated times are the result of distinct tests and present only an
expected value for the total unloading time under various conditions. Since we did not perform
full factorial tests and so far only tested for parcel size and mass, potential correlation between
the parameters might affect the unloading performance. Additionally, the conveyor time was
estimated based on conveyor velocity.
Additional effects will be evaluated in a field-test. There, actual 40ft-containers in the receiving
area of a large logistics service provider will serve as the testbed for the system. With this test,
we will evaluate the robustness over longer periods of time as well as the systems approach to
unforeseen situations.
5. Summary
This paper presents a list of available solutions and their performance as well as a first approach
towards a standardized evaluation of unloading throughput for automatic unloading solutions.
In the presented method for throughput estimation, the process is divided into multiple steps
and the tasks in each steps are evaluated in terms of performance and robustness. We propose
1
internal dimensions of 11.998 × 2.330 × 2.350 m (International Organization for Standardization 2020)
Wilhelm, Hoppe et al.
20 2021 International Scientific Symposium on Logistics
distinct tests for each task under variation of all parameters affecting performance and robust-
ness and evaluate this division on a new unloading system, resulting in a setup with five different
tasks and ten parameters.
In a first test, we estimate the unloading performance of the system to range from 341 cartons
per hour for very large items to over 3,200 cartons per hour for small items. Next, we will
evaluate all parameters affecting robustness in a laboratory environment. In field-tests we will
evaluate the robustness and performance of the system under varying conditions.
Acknowledgments
This work was supported by the German Federal Ministry of Transport and Digital Infrastructure
(BMVI) under Grant 19H17016C.
References
Bastian Solutions. 2018. “Robotic Truck Trailer – Case Loader/Unloader.” Indianapolis.
https://www.bastiansolutions.com/solutions/service/industrial-robotics/industrial-robotic-
solutions/mobile-robotics/robotic-truck-unloading-loading/.
Bortfeldt, Andreas, and Gerhard Wäscher. 2013. “Constraints in Container Loading – A State-of-
the-Art Review.” European Journal of Operational Research 229 (1): 1–20. https://doi.org/
10.1016/j.ejor.2012.12.006.
Boston Dynamics. 2021. “Boston dynamics unveils new robot for warehouse automation.” 2021.
https://www.bostondynamics.com/new-robot-for-warehouse-automation.
Cao, Zhe, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2021. “OpenPose: Real-
time Multi-Person 2D Pose Estimation Using Part Affinity Fields.” IEEE Transactions on
Pattern Analysis and Machine Intelligence 43 (1): 172–86. https://doi.org/10.1109/
TPAMI.2019.2929257.
Chen, Yucheng, Yingli Tian, and Mingyi He. 2020. “Monocular Human Pose Estimation: A Survey
of Deep Learning-Based Methods.” Computer Vision and Image Understanding 192
(March): 102897. https://doi.org/10.1016/j.cviu.2019.102897.
Echelmeyer, Wolfgang, Marco Bonini, and Moritz Rohde. 2014. “From Manufacturing to Logistics:
Development of a Kinematic for Autonomous Unloading of Containers.” Advanced Materials
Research 903: 245–51. https://doi.org/10.4028/www.scientific.net/AMR.903.245.
Freitag, Michael, Nils Hoppe, Christoph Petzoldt, Jasper Wilhelm, Lennart Rolfs, Rafael Mortensen
Ernits, and Thies Beinke. 2020. “Digitaler Zwilling Zur Mensch-Technik-Interaktion.” In
Mensch-Technik-Interaktion in Der Digitalisierten Arbeitswelt, 165–82. GITO Verlag.
https://doi.org/10.30844/wgab_2020_8.
Garrido-Jurado, S., R. Muñoz-Salinas, F.J. Madrid-Cuevas, and M.J. Marín-Jiménez. 2014.
“Automatic Generation and Detection of Highly Reliable Fiducial Markers under Occlusion.”
Pattern Recognition 47 (6): 2280–92. https://doi.org/10.1016/j.patcog.2014.01.005.
Method for the Evaluation of an Autonomous Handling System
Conference Volume 21
Gong, Wenjuan, Xuena Zhang, Jordi Gonzàlez, Andrews Sobral, Thierry Bouwmans, Changhe
Tu, and El-hadi Zahzah. 2016. “Human Pose Estimation from Monocular Images: A
Comprehensive Survey.” Sensors 16 (12): 1966. https://doi.org/10.3390/s16121966.
Honeywell Intelligrated. 2019. “Robotic Unloader.” Mason. https://www.intelligrated.com/
en/solutions/technology/robotics/robotic-unloader.
Hoppe, Nils, Jasper Wilhelm, Christoph Petzoldt, Lennart Rolfs, Thies Beinke, and Michael Freitag.
2020. “Design Eines Robotiksystems Zur Entleerung VonSeecontainern.” Logistics Journal
Proceedings. Wissenschaftliche Gesellschaft Für Technische Logistik (WGTL), 1–12.
International Organization for Standardization. 2020. “Series 1 Freight Containers — Classi-
fication, Dimensions and Ratings (ISO Standard No. 668:2020).” https://www.iso.org/
standard/76912.html.
Krantz, Eric. 2021. “Honeywell Announces Fully Automated Robotic Unloader for Distribution
Centers”. https://www.honeywell.com/us/en/press/2019/04/honeywell-announces-fully-
automated-robotic-unloader-for-distribution-centers.
Lin, Tsung-yi, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr
Dollár, and C Lawrence Zitnick. 2014. “Microsoft COCO: Common Objects in Context.” In
Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, Vol 8693.,
edited by D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, 740–55. Springer, Cham.
https://doi.org/10.1007/978-3-319-10602-1_48.
Petzoldt, Christoph, Jasper Wilhelm, Nils Hendrik Hoppe, Lennart Rolfs, Thies Beinke, and
Michael Freitag. 2020. “Control Architecture for Digital Twin-Based Human-Machine
Interaction in a Novel Container Unloading System.” Procedia Manufacturing 52: 215–20.
https://doi.org/10.1016/j.promfg.2020.11.037.
Plackett, R. L., and J. P. Burman. 1946. “The Design of Optimum Multifactorial Experiments.”
Biometrika 33 (4): 305–25. https://doi.org/10.1093/biomet/33.4.305.
Siemens Logistics GmbH. 2019. “Siemens Revolutionizes Parcel Unloading.” 2019. https://www.
siemens-logistics.com/en/news/press-releases/siemens-revolutionizes-parcel-unloading.
Stoyanov, Todor, Narunas Vaskevicius, Christian A. Mueller, Tobias Fromm, Robert Krug, Vinicio
Tincani, Rasoul Mojtahedzadeh, et al. 2016. “No More Heavy Lifting: Robotic Solutions to
the Container Unloading Problem.” IEEE Robotics and Automation Magazine 23 (4): 94–
106. https://doi.org/10.1109/MRA.2016.2535098.
United Nations. 2020. “Review of Maritime Transport 2020.” Geneva. https://unctad.org/
webflyer/review-maritime-transport-2020.
Wilhelm, Jasper, Thies Beinke, and Michael Freitag. 2020. “Improving Human-Machine Inter-
action with a Digital Twin.” In Dynamics in Logistics. Proceedings of the 7th International
Conference on Dynamics in Logistics (LDIC 2020), ed. by M. Freitag, H.-D. Haasis, H.
Kotzab, and J. Pannek, 527–40. Cham: Springer. https://doi.org/10.1007/978-3-030-
44783-0_49.
World Trade Organization. 2020. “World Trade Statistical Review 2020.” Geneva. https://www.
wto.org/english/res_e/statis_e/wts2020_e/wts20_toc_e.htm.
Xiao, Bin, Haiping Wu, and Yichen Wei. 2018. “Simple Baselines for Human Pose Estimation and
Tracking.” ArXiv, 1–16.
Wilhelm, Hoppe et al.
22 2021 International Scientific Symposium on Logistics
Zhao, Xiaozhou, Julia A. Bennell, Tolga Bektaş, and Kath Dowsland. 2016. “A Comparative
Review of 3D Container Loading Algorithms.” International Transactions in Operational
Research 23 (1–2): 287–320. https://doi.org/10.1111/itor.12094.