Content uploaded by Fabio Ruggiero
Author content
All content in this area was uploaded by Fabio Ruggiero on Apr 04, 2018
Content may be subject to copyright.
IEEE ROBOTICS AND AUTOMATION MAGAZINE, VOL. , NO. , 1
Nonprehensile manipulation of deformable objects:
Achievements and perspectives from the RoDyMan
project
Fabio Ruggiero, Antoine Petit, Diana Serra, Aykut C. Satici, Jonathan Cacace, Alejandro Donaire, Fanny
Ficuciello, Luca R. Buonocore, G. Andrea Fontanelli, Vincenzo Lippiello, Luigi Villani, Bruno Siciliano
Abstract—The goal of this work is to disseminate the results
achieved so far within the RODYMAN project related to planning
and control strategies for robotic nonprehensile manipulation.
The project aims at advancing the state of the art of nonpre-
hensile dynamic manipulation of rigid and deformable objects to
future enhance the possibility of employing robots in anthropic
environments. The final demonstrator of the RODYMAN project
will be an autonomous pizza maker. This article is a milestone
to highlight the lessons learned so far and pave the way towards
future research directions and critical discussions.
I. OV ERV IE W A BO UT NO NPR EH E NS ILE M ANI PU L ATIO N IN
ROB OTIC S
MANIPULATING an object roughly entails changing its
currentstate to a desired one. Suppose to have an object
on a table that has to be moved from configuration Ato
configuration Bon the same table. How would you move it?
Maybe, the two most rated answers would be: (i) grab the
object, lift it up and place it back on the table in B; (ii) push
the object on the table from configuration Ato B. In case
(i), the object is completely grabbed between the fingertips
and/or the palm, and the hand is theoretically able to resist
any external disturbance wrench applied to the object. Hence
form closure occurs even when infinitesimal motions of the
object are prevented by the hand, otherwise force closure
occurs [1]. Bilateral constraints are exhibited by the grasp in
both closures. In case (ii), the object is instead pushed by
one or more fingertips and these last are able to resist only
to forces counteracting the direction of the pushing. Hence,
nonprehensile manipulation, or graspless manipulation, occurs
since only unilateral constraints are involved. These tasks can
also be endorsed as dynamic when the dynamics of both the
object and the robot are essential to successful task execution.
Therefore, a nonprehensile dynamic manipulation task can
be generally described as a task in which the object is
subject only to unilateral constraints and the dynamics of
both the object and the manipulating hand, as well as the
F. Ruggiero, J. Cacace, F. Ficuciello, G.A. Fontanelli, V. Lippiello, L.
Villani and B. Siciliano are with CREATE Consortium and with Department
of Electrical Engineering and Information Technology, University of Naples
Federico II, via Claudio 21, 80125, Naples, Italy. A. Petit is with MIMESIS
team, INRIA, 1, place de l’Hopital, 67000 Strasbourg, France. D. Serra is
with Rete Ferroviaria Italiana SpA, Via De’Cattani 1, 50145, Osmannoro.
A.C. Satici is with Boise State University, 1910 W University Dr, Boise,
United States. A. Donaire is with Queensland University of Technology, 2
George st, 4000, Brisbane, QLD, Australia. L.R. Buonocore is with CERN,
Route de Meyrin 385, Geneve 23, Switzerland.
related kinematics and the (quasi-)static forces play a crucial
role. Pushing objects, folding clothes, carrying items on a
tray, cooking in pan, performing some surgery operations are
examples of nonprehensile manipulation tasks. From a robotic
point of view, most of nonprehensile manipulation systems
are underactuated, raising controllability challenges. However,
dynamic nonprehensile manipulation benefits of several ad-
vantages such as the increase of available robot actions, bigger
operative workspaces, enhanced dexterity in dynamic tasks.
On one hand, the robotics literature is well-established
regarding grasping techniques [1] and control methods for ma-
nipulation tasks with grasp [2]. Within industrial applications,
where simplicity and costs are relevant above all, grippers
or special-purpose devices are widely used. Nevertheless, the
necessity for robots working in anthropic environments is
growing fast as shown by the European Strategic Resarch
Agenda1(eSRA), where it is outlined that robots will pervade a
portion of the market domain in domestic appliances, assisting
living, entertainment and education. Therefore, robots should
not need specific tools for each actions, but they should exploit
multi-purpose devices, like multi-fingered robotic hands, and
rely upon the dexterity conferred by the designed control
algorithms. Manipulation dexterity is one of the main research
challenge currently addressed by the robotic community. As
explained above, a nonprehensile manipulation task is a dex-
terous task par excellence. As a matter of fact, some tasks are
intrinsically prehensile like, for instance, (un)screwing a bottle
cap. Other tasks can be instead tackled both in a prehensile
or a nonprehensile manner, like the aforementioned example
of moving an object on a table. Other tasks are inherently
nonprehensile, like carrying a glass full of water on a plate.
Other tasks are hybrid, in the sense that to reach the goal
both prehensile and nonprehensile actions are required, like
a juggler that has to repetitively catch-and-throw balls in a
cascade juggling pattern.
On the other hand, the robotics literature is not fully
developed for nonprehensile manipulation tasks. The classic
way to cope with them is to split a task in simpler subtasks,
referred to as nonprehensile manipulation primitives [3], such
as throwing [4], dynamic catching [5], batting [6], juggling [7],
dribbling [8], pushing [9], sliding [10], rolling [11] and so
on. Each primitive, equipped with its own motion planner
and controller, is then turned on and off during a complex
1https://www.eu-robotics.net/cms/upload/topic˙groups/SRA2020˙SPARC.pdf
IEEE ROBOTICS AND AUTOMATION MAGAZINE, VOL. , NO. , 2
manipulation task by a high-level supervisor [12]. Among
the mentioned nonprehensile manipulation primitives, only
rolling and batting boast a considerable amount of work
in the literature. There is also a lack of a general unified
theoretical framework in the field, causing the continuous
investigation of ad-hoc motion planners and controllers to
solve the specific tasks one by one. The main reason may
be found in the possible change of the contact status during
a nonprehensile manipulation task, leading to non-smooth
dynamics of the whole system, which complicates the control
design. For this reason, dynamic nonprehensile manipulation
may be considered as the most complex manipulation action,
deserving attention as requested by the eSRA, and posing
many research challenges to be solved.
In the described context, the RODYMAN project aims at the
development of a service robot able to manipulate elastic and
soft objects, as well as to manipulate both rigid and non-rigid
objects in a nonprehensile way, with the ambitious goal of
bridging the gap between robotic and human task execution
capability. In order to reach the planned goals, three main
research challenges have been identified within the RODYMAN
project. Namely:
•Mechatronic development and assembly. A mobile
robotic platform equipped with two commercial arms and
multi-fingered hands is necessary to perform the dynamic
manipulation tasks planned for the project.
•Modelling and perception. Real-time requirements
posed by robot interaction with deformable objects during
dynamic nonprehensile manipulation actions are essential
to fulfil the required tasks.
•Control techniques for nonprehensile dynamic manip-
ulation. The goal of the project is to advance the state
of the art in controlling rigid objects in a nonprehensile
way, and starting investigating the problems relative to
the prehensile and nonprehensile manipulation control of
deformable objects as well.
The final demonstrator of the project will be an autonomous
pizza maker since preparing a pizza involves an extraordinary
level of manual dexterity.
Some other projects have tried to address nonprehensile
manipulation problems using different approaches. The RIBA
robot2is able to lift up and set down patients from/to their beds
and/or wheelchairs. The soft body of the robot is designed
to make safe the interaction with the humans. The performed
transporting task is indeed nonprehensile, but the manipulation
task is not dynamic since the patient’s body is considered
as a rigid object, and only motion planning techniques for
lifting up the body are investigated. The task is very similar
to a pick-and-place operation where the transporting motion is
addressed in a nonprehensile fashion. Also the ERC SHRINE
project3goals are focused to enhance robot manipulation
capabilities, so as to overcome barriers preventing robots from
safe and smooth operations within anthropic environments.
The sought robot dynamic manipulation, in some cases also
performed in a nonprehensile way, is addressed to cooperate
2http://rtc.nagoya.riken.jp/RIBA/index-e.html
3http://www.shrine-project.eu
Fig. 1: The RODYMAN platform handling the peel with
two arms and two proper grippers at the end-effectors. The
displayed tool is a real pizza peel employed by chefs to
cook the dough in the oven. A blue silicon disk, usually
employed by acrobatic pizza chefs for training, is employed
in the experiments.
with humans. In the following, the results achieved so far
within the RODYMAN project for the above three outlined
research challenges are described. Videos of the related ex-
periments can be found in the related YouTube channel4.
II. ROBOT DES IGN A ND A RCH IT E CT URE
A. Mechatronic design
The built mechatronic set-up, referred to as RODYMAN like
the project name, is a 21-degree-of-freedom (DoF) humanoid-
like robot (see Fig. 1). An omnidirectional mobile platform
allows the robot to move in the space. An actuated mechanism
gives the ability to enlarge the support polygon during the
execution of dynamic and rapid movements of the upper body.
The battery pack and UPS unit used to provide power to
all the devices are housed within the mobile platform, and
provide at the same time the weight to stabilize the platform.
Two standard PCs are also located in the base. One is a
QNX-based PC used for real-time and low-level of the motors
and the implementation of safety procedures. The second
4https://www.youtube.com/user/ThePRISMAlab
IEEE ROBOTICS AND AUTOMATION MAGAZINE, VOL. , NO. , 3
one is a Linux-based PC used for perception and high-level
planning and control algorithms. The upper-body limbs of the
robot are two SCHUNK LWA 4P arms with 6 DoFs each.
The seventh joint of each arm, required to add human-like
redundancy, is provided by a SCHUNK PRL-100 integrated
into the shoulder. To the best of the authors’ knowledge,
the SCHUNK were the only arms on the market to have
both dimensions similar to the human arms and the control
directly on the CAN bus without an external controller box.
Nevertheless, experimental results show that the high friction
and the low joint velocities exhibited by these arms represent a
limitation on the execution of particular and complex tasks like
tossing. It is worth pointing out that this solution represents
only a first prototype, and the design of new arms, with
advanced dynamical characteristics, is within the RODYMAN
project plan.
Moreover, the dynamic model of the whole structure has
been derived in a symbolic form. The LMI method in [13]
has been employed to obtain the identification of the dynamic
parameters by absorbing the physical constraints within the op-
timization procedure. Experimental results have indeed shown
that friction, mostly the static part, is the dominant component
in the measured torque. Therefore, a friction identification
has been firstly performed separately, and then the friction
parameters have been used as constraints within the LMI
optimization procedure.
The RODYMAN platform is completed by two motors to
actuate the torso and one for the pan-tilt neck. In order to
provide enhanced dexterous manipulation skills, two anthropo-
morphic SCHUNK Servo-electric 5-Finger SVH hands can be
applied at the end- effector tip of the two arms. However, these
hands are very delicate and they are replaced with suitable 3D-
printed tools for those tasks requiring non-trivial weights in
action, like the pizza-peel task described afterwards. From the
perception point of view, the platform is equipped with two
laser scanners in the base for odometry operation; two force
sensors can be mounted on the wrist to measure the interaction
forces between the end effector and the environment, while
the interaction forces exerted on the robot structure can also
be obtained using proper estimators [14]. Finally, the head is
equipped with a stereo camera system, an RGB-D sensor and
a time-of-flight camera to obtain a precise depth estimation.
B. High-level software architecture
In order to carry out the expected activities involving com-
plex manipulation actions, a control architecture is designed
to handle high-level planning tasks. A sketch of the control
architecture is shown in Fig. 2, and it is described below.
The Human Robot Interaction (HRI) Interface module is
used to specify high-level tasks as inputs for the system
(e.g. the pizza tossing). The Supervisor module is responsible
of the task decomposition process, splitting the high-level
actions received by the HRI Interface in lower-level actions
considering both the state of the robot and the information
generated by the Perception module. After the decomposition
process, each lower level action can be executed. Exam-
ples of high-level tasks are Grasp(Object), Search(Object) or
Toss(Object), and sequences of nonprehensile manipulation
primitives. In order to suitably perform the task decomposition
process, the supervisor module is provided with a library of
hierarchical tasks, analogous to Hierarchical Task Networks
(HTNs), which can be composed by the system to achieve
the desired goal [15]. In this context, if a nonprehensile
manipulation action is required, the related low-level controller
is invoked from the dynamic manipulation task list while
the supervisor waits its termination. Otherwise, the Executors
module is responsible for the action by implementing both
the path and the motion planning functionalities to find an
obstacle-free way for the end effectors of the robot and its
base. This module relies on MoveIT! [16], a framework that
integrates Universal Robot Description File,Open Motion
Planning Library and other toolkits. The generated trajectories
for the joints and the base of the robot are streamed to the robot
actuators from the Controller modules. Finally, information
about the robot environment is extracted from the Perception
module via image elaboration algorithms.
The proposed high-level control architecture perfectly
matches the requirements of the RODYMAN robotic platform,
statically allocating the best low-level controller to accomplish
desired actions. This improves the current literature since very
few ways of decomposing a high-level task for nonprehensile
manipulation have been developed [12]. Future directions are
inclined to increase the level of collaboration of the robot with
human operators allowing shared task planning and execution.
III. PER CE PTI ON O F DEF ORMAB LE O BJ ECT S
Using the point cloud data provided by the RGB-D sen-
sor, a method to cope with real-time (35 fps) tracking of
a deformable object is presented. Several contributions are
proposed such as handling various large elastic deformations
while ensuring physical consistency, coping with fractures,
rigid motions and occlusions [17]. The case of a pizza dough
being stretched and tossed is taken as example.
Since the considered system attempts to deal with large de-
formations and elastic volumetric strains, a realistic mechani-
cal model, based on continuum mechanics and on a volumetric
tetrahedral Finite Element Method (FEM), is employed. This
model can be suitably used for real-time applications through
the SOFA simulator. Besides, an explicit physical modelling
would enable a reliable prediction of internal forces undergone
by the object.
In order to model elastic deformations, the infinitesimal
strain theory and Hooke’s law are taken into account, pro-
viding a linear relation between the displacement of the
tetrahedral elements of the mesh and the internal forces exerted
on their nodes. The co-rotational approach is used as a good
compromise between the ability to model large deformations
of the elements and computational efficiency.
Based on the FEM co-rotational model, fractures in the
mesh are detected by decomposing the internal forces on the
nodes into tensile and compressive forces to measure pure
tensile forces acting on each node, through a so-called sepa-
ration tensor. The fracture is propagated by simply removing
attached elements intersected by the fracture plane. The model
is illustrated in Fig. 3.
IEEE ROBOTICS AND AUTOMATION MAGAZINE, VOL. , NO. , 4
Fig. 2: RODYMAN high-level control architecture.
(a) (b)
Fig. 3: Volumetric tetrahedral mesh (elements in blue colors)
in (a), and modeling of fractures in (b).
The frame-by-frame tracking framework in Fig. 4 relies on a
prior visual segmentation of the object in the image, based on a
graph-cut based segmentation technique using color cues. The
corresponding segmented point cloud is first registered through
a classical Iterative Closest Point (ICP) method, and then by
fitting the known mesh of the object on the point cloud. The
basic idea is to derive external forces exerted by the point
cloud on the mesh and to integrate them with the internal
forces computed using the physical model into Lagrangian
mechanical equations: M¨
x+C˙
x+K′x+f0=fext,where
x∈Rncontains the positions of the nvertices, M∈Rn×n,
C∈Rn×nand K′∈Rn×nare the mass, damping and
stiffness matrices, fext ∈Rnis the external forces vector, and
f0∈Rnis an offset on the internal forces due to rotational
effects. An Euler implicit integration scheme and a conjugate
gradient method are used to solve the system with respect x.
The elastic forces fext based on geometrical correspondences
between the point cloud and the mesh [17].
To validate the method, some results have been obtained
with various soft objects, deformations due to bending, stretch-
ing or compression actions, and fractures, and under challeng-
ing conditions, like occlusions or fast motions (see Fig. 5).
Some preliminary experiments integrating the method into a
robotic manipulation task of a silicon pizza dough, have also
been carried out (see Fig.5c and Section IV-B).
RGB image
Visual segmentation
of the object
Segmentation of the
depth map
Sampling
Backprojection
Deformable registration
using closest point
correspondences and
FEM model
Segmented
image
Rigid registration
using ICP
Point cloud
Rigidly trasformed mesh
Point cloud
Depth
map
Deformed
mesh
Previous state
of the mesh
Fig. 4: Overview of the developed approach for deformable
object tracking.
IV. NON PRE HE N SI LE OBJE CTS M OTIO N PL ANN IN G AND
CO NT RO L
Four manipulation primitives have been considered so far
within the RODYMAN project. Namely, nonprehensile rolling,
sliding, tossing and batting/juggling. Sliding and tossing
take into account deformable objects, while rolling and bat-
ting/juggling only rigid ones. It is indeed difficult to find
relevant applications involving deformable objects in pure
rolling and juggling tasks. In the following, the controller
and/or the motion planner designed for the aforementioned
primitives are described.
IEEE ROBOTICS AND AUTOMATION MAGAZINE, VOL. , NO. , 5
(a) (b) (c) (d)
Fig. 5: Results of the tracking process with the input images (4different objects in the first row), and the corresponding
registered re-projected mesh (second row, in red for object (b), in blue for objects (b) and (d), in white for object (c)).
A. Nonprehensile rolling
An actuated manipulator of a given shape, referred to as
hand, manipulates an object only through purely rotations,
without grasping or caging it. Therefore, the object can only
roll upon the shape of the hand. Case studies like the manip-
ulation of a ball on a plate, the ball on a beam, and so on, are
deeply examined in the literature. In this article, only planar
rolling is described.
Since highly-geared harmonic drives are present within the
RODYMAN mechatronic platform, it is suitable to suppose the
acceleration of the hand ¨
θh∈Ras input ah∈Rfor the
system. The dynamic model for a nonprehensile planar rolling
manipulation system, in which the hand can only rotate around
its center of mass, is described by
¨
θh=ah,(1a)
¨sh=−b−1
22 b12ah+c21 ˙
θh+c22 ˙sh+g2,(1b)
where sh∈Ris the contact position of the object on the
hand, whose shape is parametrized through arclength, b12 ∈R
and b22 ∈Rare entries of the inertia matrix B∈R2×2,
while c21 ∈Rand c22 ∈Rare entries of the (2×2)
Coriolis matrix of the mechanical system, and g2∈Ris the
second element of the (2×1) gravity-force vector. Detailed
expressions of each term are provided in [18], where it is also
noticed that if the Coriolis terms are zero, a nonprehensile
planar rolling manipulation system is differentially flat with
the output b12
b22 θh+sh. Among the class of systems for which
the aforementioned assumption is true, it is worth recalling the
ball-on-disk (BoD) system, which is mathematically equivalent
to the disk-on-disk (DoD) system in the transversal plane [11].
The BoD consists of a ball rolling on a disk and arranged
one on top of the other as shown in Fig 6. The disk is the
actuated hand and the ball rolling on the hand is the object.
The control problem for the BoD is to balance the object at the
Fig. 6: The RODYMAN platform actuating the BoD system.
The disk is actuated by the movement of the RODYMAN
joints. The displayed structure, made by three connected bars,
is employed to make possible to start the experiments with
the ball in a position which is different from the desired
equilibrium on the top of the disk. The world frame is depicted
in red, the one attached to the rotating wheel is in green, θh
represents the angle between the two, while shmeasures the
contact position of the ball on the wheel.
IEEE ROBOTICS AND AUTOMATION MAGAZINE, VOL. , NO. , 6
upright position while driving the hand to a desired angular
set-point. This problem is solved using passivity-based control
(PBC) for port-Hamiltonian (pH) systems. In its standard form,
this approach applied to nonprehensile rolling aims at finding
a control law for system (1a)-(1b) such that the closed-loop
dynamics can be written as follows
˙
q
˙
p=0B−1Bd
−BdB−1J2(q,p)−Rd(q,p)∇Hd(q,p),
(2)
where q∈R2and p∈R2are the generalized coordinate and
moment vectors, Hd=1
2p⊤B−1
d(q)p+Vd(q)∈Ris the
desired total energy of the closed-loop system, Bd∈R2×2
and Vd∈Rare the desired mass matrix and potential energy,
respectively, and J2∈R2×2and Rd∈R2×2represent the
gyroscopic forces and damping injection of the closed loop,
respectively. The objective is to shape the desired energy of
the closed-loop dynamics to get a minimum of the potential
energy at the desired equilibrium. The asymptotically stability
of the closed loop is ensured by using the desired energy as
Lyapunov function and the detectability of the passive output.
The full development of the control design is reported in [11]
for the DoD example.
Within the RODYMAN project, control laws related to the
nonprehensile rolling primitive have been developed also for
the 3D case, like the stabilization of a ball on a flat plate and
the control of a robotic hula-hoop [19]. As a milestone, it is
possible to affirm that nonprehensile rolling can be success-
fully modelled through the pH formalism and consequently
controlled with PBC approaches. This is relevant because it
means that there exists a unified framework at least for such
class of nonprehensile manipulation primitive.
B. Friction-induced manipulation primitive
As a case study, in order to uniformly cook a pizza, the
dough must be rotated through a peel inside a wooden oven, in
which the heat source is present only in a side of the structure.
Similar actions are performed by chefs when some food must
be brown or rotated in a pan.
From a dynamic point of view, friction plays a key role
because of the sliding manipulation primitive between the
tool and the part to be manipulated. In the literature, friction-
induced manipulation has been extensively studied to create
virtual velocity fields on a vibrating plate actuated by a
mechanical system equal or similar to a Stewart platform [10].
A similar concept has been suitably modified for the pizza
case [20].
The RODYMAN platform has successfully achieved a bi-
manual nonprehensile manipulation task through sliding by
handling a peel to rotate the pizza placed on it. With ref-
erence to Fig. 1, the peel is chosen to be only translated
(and rotated) along (and around) its longitudinal direction.
A suitable combination of these two movements creates the
desired motion of the object on the peel: an acceleration along
the longitudinal direction moves the object back and forth on
the peel once static friction is overcome; while, an angular
acceleration around the same axis creates a non-uniform
pressure distribution on the object. This, together with the
linear acceleration, creates a rotation of the object. It is worth
noticing that the object rotation is not decoupled from a linear
displacement on the peel. Adaptations from [20] have been
necessary to apply the concepts on the RODYMAN platform.
Namely, two suitable smooth sinusoidal accelerations, with the
same tunable frequency and different tunable amplitudes and
phases, are planned for the linear and angular accelerations
of the peel. The motion of the RODYMAN joints are then
retrieved by means of a standard closed-loop inverse kinematic
algorithm. The above-described tracking of deformable objects
has been employed to control the center of mass of the pizza
towards the center of the peel through a simple PI-controller,
while a complete rotation of the circular shape is requested.
Friction estimation is crucial within this task and several
tests have been performed to suitably tune all the parameters
of the control model to fit the real set-up. Current work aims at
finding structural properties for the controller, like the design
of orbital stabilization for the object on the peel (i.e., reach a
desired rotational velocity).
C. Tossing task
The problem of tossing and catching a deformable object,
like a pizza dough, is a procedure that is frequently dexterously
performed by human pizza chefs. There are at least three
reasons why tossing the dough during the preparation of the
pizza is attractive: (i) the dough is stretched to a desired size,
(ii) the dough naturally assumes a configuration that is thicker
at the ends and thinner in the middle, and (iii) as the spinning
dough freely falls, the outside of the dough dries, making it
crunchy in the outside but light in the middle. The pizza chef
is trained to perform a streamlined hand motion to toss and
catch the dough, while a similar feat is a desired achievement
for the RODYMAN robot.
The combined model of the dough grasped with robotic
fingers through unilateral constraints, and the kinematics and
dynamics of the robot manipulator has been derived in [4].
Upon that, a control law achieving the desired tossing motion
can be designed. Furthermore, with a perfect knowledge of
the motion of the dough, optimal trajectories can be generated
in SE (3) for the catching phase. The optimal trajectory
generation is repeated as new sensor information is avail-
able. The trajectories are generated in such a way that the
initial position, velocity, acceleration and final velocity and
accelerations are matched, therefore, it is at least thrice con-
tinuously differentiable. An optimal trajectory, whose initial
and final accelerations are prescribed, has to satisfy a sixth-
order Boundary Value Problem (BVP). Such BVP is generated
by using the necessary conditions for a path to minimize a
convex combination of the jerk and acceleration functionals.
While minimizing the jerk functional reduces the vibrations
in the structure of the robotic manipulator, minimizing the
acceleration functional reduces the total amount of energy
expended during the catching motion. Only the case where
the final position is left free and is part of the minimization
problem is considered. More details can be found in [4].
Experimental validations are in progress. Nevertheless, pre-
liminary results show that such kind of task requires high
IEEE ROBOTICS AND AUTOMATION MAGAZINE, VOL. , NO. , 7
peak currents in the motors to toss the dough for more than
10 cm. As anticipated above, the motors of RODYMAN do
not have such skills. As a matter of fact, analogies between
tossing and walking gaits can be found within mathematical
models. Similarly to robotic legs, hydraulic actuators seem
to be more performing and the same might hold for tossing
tasks. The stretching-the-dough task can be also performed in
an alternative form which will be explored in the future.
D. Batting/Juggling skills
A very challenging primitive from the control view point
is the one involving impacts. Inside batting, for instance, an
object (a ball) is intercepted by the end-effector (a paddle)
without grasping it, and it is thrown towards a precise goal.
This motion primitive is typically used by athletes, such
as baseball or table tennis players. Also jugglers use such
primitive when their hands control the continuous motion
of one or more objects through intermittent contacts. These
dynamic motions require high velocity and precision. The
design of planning and control methods to deal with them
would strongly enhance capabilities of robot manipulators,
extending the workspace size and enhancing dexterity.
The batting task dynamics is typically defined as hybrid,
since it consists of the continuous aerodynamics of the ma-
nipulated ball (a differential equation), and the discontinuous
reset of the velocity at impact time (two difference equations),
given by
¨
pb=−g−kd|| ˙
pb|| ˙
pb+klS(ωb)˙
pb,(3a)
˙
p+
b=vp+Γa(˙
p−
b−vp) + Γbω−
b,(3b)
ω+
b=Γc(˙
p−
b−vp) + Γdω−
b,(3c)
where pb∈R3and ωb∈R3are the position and the
spin of the ball, respectively; vp∈R3is the paddle ve-
locity; Γj(Rp)∈R3×3,j={a, b, c, d}, are transformation
matrices dependent on the rebound parameters and on the
orientation of the paddle Rp∈SO(3) at the impact time;
kd(˙
pb,ωb)∈Rand kl(˙
pb,ωb)∈Rare, respectively, drag
and lift parameters; g∈R3is gravity acceleration vector;
||·|| denotes the Euclidean norm; S(·)∈R3×3is the skew-
symmetric matrix, while superscripts −and +represent the
state before and after the impact, respectively. The matrices
Γjcan be detailed on the basis of the addressed rebound (ball
impacting the table and/or a rubber paddle). Their expression
may become complicated in non-trivial situations (like non-
spherical objects): this leads to the use of some (string)
assumptions and model reductions.
Five different phases have been considered to solve the
batting problem by using the RODYMAN platform simulator.
Firstly, a vision system is assumed to measure the trajectory
of the ball. By assigning the impact time, the prediction of
the impact position and pre-impact velocity of the ball are ob-
tained numerically solving its aerodynamic model (3a). Then,
the post-impact velocity of the ball, such that it goes towards a
desired goal in a predefined time, is computed solving equation
(3a) backward in time. The configuration of the paddle to
generate such velocity of the ball results from the analytic
solution of the discontinuous part of the ball-paddle model,
Fig. 7: The renowned pizza chef Enzo Coccia wearing the
Xsens MVN suite, with the RODYMAN avatar in the back-
ground acquiring and repeating the movements of the chef.
given by the difference equations (3b) and (3c). Thereafter, the
motion of the paddle to reach the desired configuration is a
result of the minimization of its linear and angular acceleration
with a coordinate-free approach, assuming that the path is
generated on an arbitrary Riemannian manifold, similarly to
the tossing primitive. Finally, the motion of the RODYMAN
joints is derived from a classical second-order closed-loop
kinematic inversion. More details can be found in [6].
A similar algorithm can be thought to accomplish different
juggling patterns. While doing this, the lesson learned is
that these techniques may also be applied to other dynamic
tasks which share the same hybrid nature with impacting
manipulations, such as walking or running tasks.
V. FINAL DI SC USS IO N
Despite several progresses have been addressed within the
RODYMAN project so far, like real-time tracking of deformable
objects employed in tossing and sliding tasks, several issues
have to be tackled yet. The mechatronic platform should
be revised to cope with issues given by the high-velocity
of some nonprehensile manipulation primitives. In general,
experiments involving nonprehensile actions are not easy to
solve due to the uncertain dynamics mainly due to friction:
parameters estimation and/or robust controllers are thus essen-
tial. Moreover, physics terms causing non-smooth behaviour
are often neglected when deriving the mathematical model
of a given nonprehensile task: this makes the nonprehensile
system look like a prehensile one. This happens for instance
within rolling, sliding and pushing nonprehensile manipulation
primitives. The proof that the designed controller does not vio-
late the given assumptions is usually performed a-posteriori. A
method to directly control the contact forces should be indeed
addressed and this might be a future research direction, leading
to the design of non-smooth and hybrid controllers which are
also a new frontier for the research community.
Another approach might instead be the observation of pizza
chef activities to learn task simplification and synthesize
human-inspired control strategies. For instance, an integrated
IEEE ROBOTICS AND AUTOMATION MAGAZINE, VOL. , NO. , 8
robotic platform able to acquire and transfer human body mo-
tion to a robotic system is obtained by interfacing RODYMAN
with a low-cost motion capture system (see Fig. 7). Once the
teleoperation algorithm for real-time replication of human mo-
tion on RODYMAN is developed, a comprehensive taxonomy
of dynamic prehensile and non-prehensile tasks, ordered for
different levels of hand-arm and dual-arm coordination, can
be built from scratch. To this aim, taking inspiration from
the researches conducted on anthropomorphic hands [21] or a
single hand-arm system [22], a study on postural synergies for
dual-arm robotic manipulation can be conducted to develop a
framework where to simplify learning strategies from human
imitation. Such approach will also take advantage of dimen-
sionality reduction strategy to successfully apply supervised
reinforcement learning algorithms using synergistic motion.
Today, from these observations, it has been learned that the
motion planner is crucial for nonprehensile tasks since the
repetitive actions seem well imprinted in the pizza chef’s
mind, while the corrections made by the hands are very small
despite the difference between various doughs. Therefore, it is
reasonable to state that a good motion planner is the essential
instrument within nonprehensile manipulation.
One further question that may arise is: why has the pizza-
making procedure been taken as example? Is there the need
of having a robot making pizza? In truth, the pizza-making
process is only a media expedient with scientific purposes. It is
indeed clear that if a robot is able to manipulate a pizza dough,
it might be able to perform similar difficult manipulation
tasks. For instance, in 1997 the RoboCup started certainly
not to replace the real soccer players, but rather to advance
the state of the art while facing both gaming and difficult
problems for robots. With the same aim, RODYMAN is trying
to mimic the artistic ability of a pizza chef. While facing this
big challenge, many sub-problems have to be addressed in
parallel which could have an impact in other domains. The
perception of elastic objects is currently being applied in the
medical context to shape variations of muscles and organs. The
manipulation performed while tossing the deformable dough
is currently under investigation to improve the automation
of gluing the shoes’ lower surfaces. The batting process has
similar dynamics to the walking gaits, and as such it could
be used to improve autonomy of humanoids, or employed for
actuated prostheses.
ACK NOWL EDG ME N T
The research leading to these results has been supported by
the RoDyMan project, which has received funding from the
European Research Council FP7 Ideas under Advanced Grant
agreement number 320992. The authors are solely responsible
for the content of this manuscript.
REF ERE NC ES
[1] D. Prattichizzo and J. Trinkle, “Grasping,” in Springer Handbook of
Robotics, B. Siciliano and O. Khatib, Eds. Springer International
Publishing, 2016, pp. 955–988.
[2] R. M. Murray, Z. Li, and S. S. Sastry, A mathematical introduction to
robotic manipulation. CRC press, 1994.
[3] K. Lynch and T. D. Murphey, “Control of nonprehensile manipulation,”
in Control Problems in Robotics, ser. Springer Tracts in Advanced
Robotics, A. Bicchi, D. Prattichizzo, and H. Christensen, Eds. Springer
Berlin Heidelberg, 2003, vol. 4, pp. 39–57.
[4] A. Satici, F. Ruggiero, V. Lippiello, and B. Siciliano, “A coordinate-
free framework for robotic pizza tossing and catching,” in 2016 IEEE
International Conference on Robotics and Automation, Stockholm, S,
2016, pp. 3932–3939.
[5] G. B ¨atz, A. Yaqub, H. Wu, K. Kuhnlenz, D. Wollherr, and M. Buss,
“Dynamic manipulation: Nonprehensile ball catching,” in 18th Mediter-
ranean Conference on Control and Automation, Marrakech, MA, 2010,
pp. 365–370.
[6] D. Serra, A. Satici, F. Ruggiero, V. Lippiello, and B. Siciliano, “An
optimal trajectory planner for a robotic batting task: the table tennis
example,” in 13th International Conference on Informatics in Control,
Automation and Robotics, Lisbon, P, 2016, pp. 90–101.
[7] P. Reist and R. D’Andrea, “Design and analysis of a blind juggling
robot,” IEEE Transactions on Robotics, vol. 28, no. 6, pp. 1228–1243,
2012.
[8] G. B¨atz, U. Mettin, A. Schimdts, M. Scheint, D. Wollherr, and
A. Shiriaev, “Ball dribbling with an underactuated continuous-time
control phase: Theory & experiments,” in 2010 IEEE/RSJ International
Conference on Intelligent Robots and Systems, Taipei, Taiwan, 2010, pp.
2890–2895.
[9] K. Lynch and M. Mason, “Stable pushing: Mechanics, controllability,
and planning,” The International Journal of Robotics Research, vol. 15,
no. 6, pp. 533–556, 1996.
[10] T. Vose, P. Umbanhowar, and K. Lynch, “Friction-induced velocity fields
for point parts sliding on a rigid oscillated plate,” The International
Journal of Robotics Research, vol. 28, no. 8, pp. 1020–1039, 2009.
[11] A. Donaire, F. Ruggiero, L. Buonocore, V. Lippiello, and B. Siciliano,
“Passivity-based control for a rolling-balancing system: The nonprehen-
sile disk-on-disk,” IEEE Transactions on Control Systems Technology,
2016, in press.
[12] J. Woodruff and K. Lynch, “Planning and control for dynamic, non-
prehensile, and hybrid manipulation tasks,” in 2017 IEEE International
Conference on Robotics and Automation, Singapore, 2017, pp. 4066–
4073.
[13] C. D. Sousa and R. Cortes ˜ao, “Physically feasibility of robot base
inertial parameters identification: A linear matrix inequality approach,”
International Journal of Robotics Research, vol. 33, no. 6, pp. 931–944,
2014.
[14] A. De Luca, A. Albu-Schaffer, S. Haddadin, and G. Hirzinger, “Collision
detection and safe reaction with the DLR-III lightweight manipulator
arm,” in 2006 IEEE/RSJ International Conference on Intelligent Robots
and Systems, Beijing, C, 2006, pp. 1623–1630.
[15] J. Cacace, A. Finzi, V. Lippiello, G. Loianno, and D. Sanzone, “Aerial
service vehicles for industrial inspection: Task decomposition and plan
execution,” Applied Intelligence, vol. 42, no. 1, pp. 49–62, 2015.
[16] S. Chitta, I. Sucan, and S. Cousings, “MoveIt! [ROS topics],” IEEE
Robotics & Automation Magazine, vol. 19, no. 1, pp. 18–19, 2014.
[17] A. Petit, V. Lippiello, G. A. Fontanelli, and B. Siciliano, “Tracking
elastic deformable objects with an RGB-D sensor for a pizza chef robot,”
Robotics and Autonomous Systems, vol. 88, pp. 187–201, 2017.
[18] V. Lippiello, F. Ruggiero, and B. Siciliano, “The effects of shapes in
input-state linearization for stabilization of nonprehensile planar rolling
dynamic manipulation,” Robotics and Automation Letters, vol. 1, no. 1,
pp. 492–499, 2016.
[19] A. Gutirrez-Giles, F. Ruggiero, V. Lippiello, and B. Siciliano, “Modeling
and control of a robotic hula-hoop system without velocity measure-
ments,” in 20th World Congress of the International Federation of
Automatic Control, Toulouse, F, 2017.
[20] M. Higashimori, K. Utsumi, Y. Omoto, and M. Kaneko, “Dynamic ma-
nipulation inspired by the handling of a pizza peel,” IEEE Transactions
on Robotics, vol. 25, no. 4, pp. 829–838, 2009.
[21] F. Ficuciello, G. Palli, C. Melchiorri, and B. Siciliano, “Postural syn-
ergies of the UB Hand IV for human-like grasping,” Robotics and
Autonomous Systems, vol. 62, no. 4, pp. 515–527, 2014.
[22] F. Ficuciello, D. Zaccara, and B. Siciliano, “Synergy-based policy
improvement with path integrals for anthropomorphic hands,” in 2016
IEEE/RSJ International Conference on Intelligent Robots and Systems,
Daejeon, K, 2016, pp. 1940–1945.
