Available via license: CC BY 4.0
Content may be subject to copyright.
Sequential Manipulation Planning for Over-actuated UAMs
Yao Su1˚, Jiarui Li1,2˚, Ziyuan Jiao1˚, Meng Wang1, Chi Chu1,3, Hang Li1, Yixin Zhu4, Hangxin Liu1:
Abstract— We investigate the sequential manipulation plan-
ning problem for unmanned aerial manipulators (UAMs).
Unlike prior UAM work that primarily focuses on one-step ma-
nipulation tasks, sequential manipulations require coordinated
motions of the floating base, the manipulator, and the object
being manipulated, entailing a unified kinematics and dynamics
model for motion planning under designated constraints. By
leveraging a VKC-based motion planning framework that con-
solidates components’ kinematics into one chain, the sequential
manipulation task of a UAM can be planned as a whole
with more coordinated motions. Integrating the kinematics and
dynamics models with a hierarchical control framework, we
demonstrate, for the first time, an over-actuated UAM achieves
a series of new sequential manipulation capabilities in both
simulation and experiment.
I. INTRODUCTION
Combining the agility of unmanned aerial vehicles (UAVs)
and the flexibility of manipulators, UAMs can conduct
manipulation tasks across rough terrains and in regions
unreachable by ground robots [1–3]. Oftentimes, a fully-
or even over-actuated UAV serves as the UAMs’ flying
vehicle [4–6]; this type of UAVs can track position and
orientation independently to provide the UAM with more
agile maneuver, achieve a larger reachable workspace, and
obtain better dynamic properties compared with traditional
multirotors. Existing UAMs leverage a bi-level schema by
combining (i) a controller to stabilize the system and track
the desired trajectory under forceful contacts with the en-
vironment and (ii) a motion planner to produce trajectories
satisfying task-related constraints. Such a bi-level schema of
UAMs has succeeded in a wide range of aerial manipulation
tasks, such as pick-and-place [3, 7], inspection [8, 9], valve
operation [10], and door-like articulated object manipula-
tion [11, 12].
To date, UAMs are limited to tasks with one-step plan-
ning. To endow UAMs with multi-step sequential manip-
ulation capability, UAM platform ought to (i) coordinate
the motions of its floating base and the manipulator that
consists of a series of revolute/prismatic joints, and (ii)
effectively produce varied motion patterns at different steps
of a sequential task, especially when it interacts with ob-
jects with diverse kinematic structures. Developing such a
sequential manipulation planning schema for UAMs remains
an unexplored topic.
˚Yao Su, Jiarui Li, and Ziyuan Jiao contributed equally to this work.
:Corresponding author. 1National Key Laboratory of General Artificial
Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI).
2Department of Advanced Manufacturing and Robotics, College of
Engineering, Peking University. 3Department of Automation, Tsinghua
University. 4Institute for Artificial Intelligence, Peking University.
Emails: {suyao, lijiarui, jiaoziyuan, wangmeng,
chuchi, lihang}@bigai.ai, yixin.zhu@pku.edu.cn,
liuhx@bigai.ai
Fig. 1: Two sequential manipulation tasks completed by the
UAM. They require an over-actuated UAM platform for more agile
motion, a VKC modeling technique for manipulation planning, and
an effective hierarchical control algorithm.
Planning sequential manipulation is challenging even for
ground mobile manipulators [13–15]. In particular, consol-
idating the kinematics of the mobile base, the manipulator,
and the manipulated object into one kinematics chain—
constructing a virtual kinematic chain (VKC)—emerges as
an effective means; it plans the mobile manipulator as a
whole, yielding more coordinated manipulations [16–19].
Inspired by VKC, we extend the whole-body sequential
manipulation from ground robots to UAMs. Here, “whole-
body” refers to the unification of the trajectory planning for
the floating base and the motion planning for the manipulator.
First, we devise a novel UAM [20–22] by integrating a 4-
degree-of-freedom (DoF) manipulator with an over-actuated
UAV that can be easily replicated by composing four modular
quadcopters. Next, through a dedicated nullspace-based con-
trol allocation framework, this new UAM platform possesses
high thrust efficiency, can achieve arbitrary attitudes control,
and is robust against controller sampling frequency and
measurement noise [20, 21]. Finally, after inserting virtual
linkages and joints and abstracting the object being ma-
nipulated by its kinematic structure, we derive the (virtual)
kinematics and dynamics of this new UAM and solve the
corresponding motion planning problems on the VKC via
arXiv:2306.14105v1 [cs.RO] 25 Jun 2023
𝒛!
𝓕𝑩
𝒚#
𝒙#
𝒛#
𝓕𝑾
𝑙
𝒙!
𝒚!
4×quadcopter
thrust generator 1
thrust generator 2
thrust generator 3
thrust generator 4
𝒒𝑴
𝟏
𝒒𝑴
𝟐
𝒒𝑴
𝟑
𝒒𝑴
𝟒
link 1
link 2
6-DoF virtual base
UAM o riginal trans
𝒚%
𝒙%
𝒛%
𝓕𝒊
𝜶𝒊
𝜷𝒊4×BLHeli electronic
speed controller
4×Emax AVAN Mini
3-inch propeller
4×carbon fiber tubes
Raspberry Pi Zero 2W
Dynamixel U2D2 USB converter
700mAh 5V lithium battery
3-DoF manipulator with a 1-DoF gripper,
actuated by 4×Dynamixel XC330-M228-T,
connected by 3D-printed connector
versatile gripp ers that can be
kitted with different accessories
Crazyflie 2.1 control board
Crazyflie BigQuad deck
4×Emax RS1108
brushless motor
power distribution board
11.1V 45C 1000mAh battery
Fig. 2: Hardware design and attached coordinate frames of our over-actuated UAM platform. The flying vehicle consists of four
omnidirectional thrust generators. Each thrust generator has a 2-DoF passive gimbal mechanism and a quadcopter for over-actuation. The
manipulator has three serial rotational DoFs and a parallel gripper.
trajectory optimization [16, 17]. A hierarchical control archi-
tecture is further developed.
Together, our new UAM platform with VKC-based plan-
ning framework and a hierarchical control architecture
demonstrate various sequential aerial manipulation tasks
involving multiple steps in simulations and physical experi-
ments. Fig. 1 illustrates an example of relocating an object
into a closed drawer and a closed cabinet, requiring six
manipulation steps. For the first time, this task demonstrates
the plausibility and potential of planning sequential aerial
manipulation tasks for UAMs.
This paper makes three contributions: (i) We put forward
a novel mechanical design of an over-actuated UAM and
derive the dynamics model of the system. (ii) We devise a
manipulation planning and hierarchical control framework
for our UAM. (iii) We demonstrate the entire pipeline’s
sequential manipulation capabilities in both simulations and
experiments.
The remainder of the paper is organized as follows. Sec. II
presents the hardware design of our UAM platform. Secs. III
to V describe the system dynamics, manipulation planning,
and control framework of this platform, respectively. Simu-
lation and experimental results are summarized in Secs. VI
and VII, respectively. Sec. VIII concludes the paper.
II. HARDWARE DESIGN
As shown in Fig. 2, our UAM platform consists of an
over-actuated flying vehicle and a 4-DoF robotic manipulator
connected at the bottom; it weights 1.21 kg with a maximum
payload of 3kg. Due to limited computing power, the plat-
form’s controller runs on a remote PC that sends commands
to the platform wirelessly.
A. Flying Vehicle
The flying vehicle’s central frame is a rigid body made of
a resin block fixed with four carbon-fiber tubes. Each tube
is connected to an omnidirectional thrust generator with two
added passive DoFs to a generic quadcopter through a 3D-
printed gimbal mechanism; see Fig. 2. Each quadcopter com-
prises a Crazyflie 2.1 control board, a Bitcraze’s BigQuad
Deck, and a power distribution board connected to an 11.1v
Lithium battery. Four Emax RS1108 brushless motors actuate
3-inch propellers with a maximum thrust force of tmax “
2.6N. Their speeds are controlled by an Electronic Speed
Controller (ESC). This flying vehicle has been fully verified
in prior work [21–23], capable of independently tracking 6-
DoF position and attitude trajectory and achieving arbitrary
attitude rotations with high thrust efficiency.
B. Robotic Manipulator
The robotic manipulator is installed at the bottom of the
flying vehicle. It comprises three serial rotational DoFs and a
parallel gripper. Four Dynamixel XC330-M228-T motors are
utilized to actuate the manipulator, and a Raspberry Pi Zero
(RPi Zero) and a Dynamixel U2D2 converter are equipped
on the flying vehicles to receive the control command wire-
lessly; RPi Zero then send these signals to control the motors.
The manipulator subsystem is powered by a 5Vbattery. The
system’s physical properties are tabulated in Tab. I.
III. DYNAM IC S MODELLING
The complete dynamics model of the UAM platform is
too complex for controller design while dividing it into two
decoupled subsystems—the arm and the flying vehicle—
introduces severe disturbance to the platform. As a result,
we simplify the flying vehicle’s dynamics by concentrating
on compensating the gravity torque introduced by the shift
of center of mass (CoM) when the manipulator is in motion.
A. Platform Configuration and Notation
Fig. 2 illustrates related coordination frames. The world
frame and the UAM’s body frame are denoted as FWand
FB, respectively. We define the body frame’s position as
p
p
p“ rx, y, zsT, the attitude in the roll-pitch-yaw conven-
tion as θ
θ
θ“ rϕ, θ, ψsT, and the angular velocity in FBas
ω
ω
ω“ rp, q, rsT[24]. Frame Fiis attached to the ith thrust
TABLE I: Physical Parameters of the UAM platform. m0and
I0denote the mass and inertia matrix of the flying vehicle’s
mainframe, respectively. miand Iidenote each thrust generator’s
mass and inertia matrix, respectively. mj
Mand Ij
Mdenote the mass
and inertia matrix of the manipulator link j, respectively.
Group Parameter Value
flying vehicle
m0
B{kg 0.168
mi
B{kg 0.222
diagpI0
Bq{kg ¨cm2r0.30 0.30 0.60s
diagpIi
Bq{kg ¨cm2r2.23 2.84 4.51s
l{m0.21
tmax{N2.6
manipulator
m1
M{kg 0.044
m2
M{kg 0.040
m3
M{kg 0.043
diagpI1
Mq{kg ¨cm2r0.22 0.21 0.04s
diagpI2
Mq{kg ¨cm2r0.22 0.19 0.06s
diagpI3
Mq{kg ¨cm2r0.82 0.80 0.15s
griper range/mm 4´35
others
remote PC control rate/Hz 100
quadcopter control rate/Hz 500
manipulator control rate/Hz 500
communication delay/ms 20
generator’s center. We combine the flying vehicle’s pose and
velocity as q
q
qB“ rWp
p
pT,Bθ
θ
θTsTand 9q
9
q
9
qB“ rWv
v
vT,Bω
ω
ωTsT. Let
q
q
qMPR4ˆ1be the manipulator’s joint angles.
B. Flying Vehicle Dynamics
The dynamics model of the flying vehicle is simplified as:
:q
:
q
:qB“„1
m
W
BR
R
R0
0BJ
J
Jpq
q
qMq´1ȷu
u
u`„gˆz
ˆz
ˆz
Bτ
τ
τgpq
q
qMq´Bω
ω
ωˆBJ
J
Jpq
q
qMqBω
ω
ωȷ,(1)
where gis the gravitational acceleration, mthe whole
platform’s total mass, J
J
Jthe whole platform’s inertia matrix,
Bτ
τ
τgis the gravitational torque due to the displacement of its
CoM from the geometric center, and ˆ
z
z
z“ r0,0,1sTis the unit
vector in the vertical direction in the world frame. Of note,
J
J
Jand Bτ
τ
τgare functions of the manipulator’s joint angles
q
q
qMdefined by kinematic relationships. And
u
u
u“„ř4
i“1
B
iR
R
R Tiˆz
ˆz
ˆz
ř4
i“1pd
d
diˆB
iR
R
R Tiˆz
ˆz
ˆzqȷ“„J
J
Jvpα
α
α, β
β
βq
J
J
Jωpα
α
α, β
β
βqȷT
T
T , (2)
where Ti,αi, and βidenote the magnitude of thrust, tilting,
and twisting angles of the ith thrust generator, respectively.
d
d
diis the distance vector from FB’s center to each Fi.
C. Manipulator Dynamics
The dynamics of the manipulator are modeled following
Luo et al. [25], formally as:
M
M
MMpq
q
qMq:q
:q
:qM`C
C
CMpq
q
qM,9q
9q
9qMq `G
G
GMpq
q
qMq “ τ
τ
τM`J
J
Jext F
F
Fext,(3)
where M
M
MMPR4ˆ4is the manipulator’s inertia matrix, C
C
CMP
R4ˆ1the vector of the Coriolis and centrifugal terms, G
G
GMP
R4ˆ1the gravitational force vector, τ
τ
τMthe torque command
of each joint actuator, F
F
Fext external forces, and J
J
Jext the
related Jacobian matrix.
IV. SEQUENTIAL AERIAL MANIPULATION PLANNING
In this section, we start by describing three essential steps
to construct VKCs [16, 17, 26] for our UAM platform. Next,
we formulate the sequential manipulation planning problem
on VKCs and solve it through trajectory optimization for
aerial manipulation tasks.
A. Modeling UAMs with VKCs
Kinematic inversion reverses the kinematic model of
an articulated object by converting the attachable link into
the new root in the inverted kinematic model. Of note, in
addition to reversing the parent-child relationship for every
two adjacent frames between the base link and the attachable
link of the object, the spatial transformation of each joint
must also be updated appropriately since a joint typically
constrains the child link’s motion w.r.t. child link’s frame.
Virtual joint defines the spatial transformation between
two body frames and the joint type that constrains the relative
motion between them. The manipulator and the manipulated
object form a single serial kinematic chain by inserting a
virtual joint between the manipulator’s end-effector and an
attachable link in the object model. If a manipulated object
is articulate, its kinematic model has to be inverted for the
constructed kinematic chain to remain serial.
Virtual base reflects the motion constraints imposed on
the floating base. In our UAM platform, the floating base
is an over-actuated UAV that can achieve free motion in
space. Specifically, starting from the ground, we add three
perpendicular prismatic joints for linear motion, followed by
three revolute joints at the center of the UAV body frame
for angular motions. These six joints together form a virtual
chain that mimics the possible motions of the floating base.
A VKC for aerial manipulation planning is constructed
by augmenting a virtual base to the UAM’s kinematic
model; see Fig. 2. During the manipulation, the end-effector
connects to the inverted object model via a virtual joint. From
this VKC perspective, performing an aerial manipulation task
is treated as altering the VKC’s state, equivalent to solving
a motion planning problem on VKCs.
B. Motion Planning on VKCs
The state vector x
x
xPXfree describes the state of a VKC,
where Xfree PRnis the collision-free configuration space. The
motion planning problem on VKCs is equivalent to finding a
T-step path x
x
x1:TPXfree, which can be formulated and solved
by trajectory optimization. Following Jiao et al. [16, 17], the
objective function of the trajectory optimization is:
min
x
x
x1:T
T´1
ÿ
t“1
}W
W
Wvδx
x
xt}2
2`
T´1
ÿ
t“2
}W
W
Waδ9
x
x
xt}2
2,(4)
where we penalize the overall traveled distance and overall
smoothness of the trajectory x
x
x1:T.W
W
Wvand W
W
Waare two diag-
onal weighting matrices for each DoF, δx
x
xtand δ9
x
x
xtare finite
forward difference and second-order finite central difference
of x
x
xt, respectively. An equality constraint is imposed on the
constructed VKC, which specifies the physical constraints of
the object or the environment:
hchainpx
x
xtq “ 0,@t“1,2,...,T. (5)
Failing to account for this type of constraint (e.g., the
kinematic constraint of the manipulator or the object) may
damage the UAM or the object being manipulated, resulting
in failed executions.
The goal of a sequential aerial manipulation task is for-
radio
position
controller
attitude
controller
control
allocation
flying vehicle
trajectory
planner
𝑢!
𝑢"
𝛼#,#𝛽#
𝑇#,𝜃
feedback
linearization
controller
𝑢#
𝑞$
%, 𝑞)$
%
𝑝%, 𝑣%
𝜃%, 𝜔%
dynamics
controller
𝑞&
%, 𝑞)&
%, 𝑞-&
%
4-DoF
manipulator
𝑝, 𝑣
𝜃, 𝜔
𝑞&, 𝑞)&
𝜏&
quadcopter
controller
motor
torque
controller
Wi-Fi
𝑞&
planner and high-level controller at remote PC (100 HZ) UAM onboard
controller (500 Hz)
Fig. 3: Hierarchical control architecture of the UAM platform.
The high-level controller of the flying vehicle (i) calculates desired
wrench command u
u
udfor trajectory tracking and (ii) allocates it
to desired thrusts and joint angles of thrust generators through
control allocation. Each quadcopter has its own onboard controller
to regulate the joint angles and thrust to desired values.
mulated as an inequality constraint:
›
›ftaskpx
x
xTq´Ggoal›
›
2
2ďξgoal,(6)
which bounds the final state x
x
xTof the VKC and the task
goal Ggoal PGwith a small tolerance ξgoal. The function
ftask :RnÑRkmaps x
x
xTfrom the configuration space Xto
the task-dependent goal space GPRk. For example, in an
object-picking task, ftask represents the forward kinematics
of the VKC, and Ggoal is the end-effector pose prior to
grasping [27].
Additional safety constraints are further imposed on the
motion planning problem:
x
x
xmin ďx
x
xtďx
x
xmax,@t“1,2, . . . , T (7)
}δx
x
xt}8ď9
x
x
xmax,}δ9
x
x
xt}8ď:
x
x
xmax,@t“2,3, . . . , T ´1(8)
Nlink
ÿ
i“1
Nobj
ÿ
j“1
|distsafe ´fdistpLi, Ojq|`ďξdist ,(9)
Nlink
ÿ
i“1
Nlink
ÿ
j“1
|distsafe ´fdistpLi, Ljq|`ďξdist ,(10)
where |¨|`is defined as |x|`“maxpx, 0q. Eqs. (7) and (8)
are inequality constraints that define the joint capability and
implicitly constrain the workspace of a UAM. Eq. (9) and
Eq. (10) penalize collisions with obstacles and self-collisions,
respectively. distsafe is a predefined safety distance, and fdist
is a function that calculates the signed distance between a
pair of objects.
V. CONTRO L
Using a hierarchical control architecture, we devise the
UAM’s overall controller with two subsystems, shown in
Fig. 3. The high-level controller calculates the desired com-
mands for trajectory tracking remotely and sends them wire-
lessly to the low-level controller that runs on the platform
with high frequency.
A. Flying Vehicle Control
High-level control: The feedback-linearization method
is applied to Eq. (1) to transfer the nonlinear system dynam-
ics to a linear double integrator [28–30]:
u
u
ud“«mW
BR
R
RT0
0
0
0BJ
J
Jpq
q
qMqffˆ„u
u
uv
u
u
uωȷ´„gˆz
ˆz
ˆz
Bτ
τ
τgpq
q
qMq´Bω
ω
ωˆBJ
J
Jpq
q
qMqBω
ω
ωȷ˙,(11)
where the superscript dindicates the desired values, u
u
uv
and u
u
uωare two virtual inputs that can be designed with
translational and rotational errors to track the reference
position and attitude trajectory:
u
u
uv“9v
9v
9vr`Kv1e
e
ev`Kv2e
e
ep`Kv3że
e
epdt,
u
u
uω“9ω
9ω
9ωr`Kω1e
e
eω`Kω2e
e
eθ`Kω3że
e
eθdt,
(12)
where Kpi and Kωi are constant gain matrices, and the
superscript rindicates the reference value from the VKC-
based motion planning; see Sec. IV. The error terms are
defined following Su et al. [31]:
e
e
ep“p
p
pr´p
p
p, e
e
ev“v
v
vr´v
v
v,
e
e
eθ“1
2rR
R
Rpθ
θ
θqTR
R
Rpθ
θ
θrq´R
R
Rpθ
θ
θrqTR
R
Rpθ
θ
θqs_,
e
e
eω“R
R
Rpθ
θ
θqTR
R
Rpθ
θ
θrqω
ω
ωr´ω
ω
ω,
(13)
where R
R
Rp¨q is the transformation from Euler angles to a
standard rotation matrix, and r¨s_is the mapping from SO(3)
to R3. Combining Eqs. (11) to (13) with Eq. (1), we have
the error dynamics as:
9
e
e
ev`Kv1e
e
ev`Kv2e
e
ep`Kv3że
e
epdt “0,
9
e
e
eω`Kω1e
e
eω`Kω2e
e
eθ`Kω3że
e
eθdt “0,
(14)
which is an asymptotically stable system.
Control allocation and low-level control: The control
allocation solves for desired command αd
i,βd
i, and Td
ifor
each 3-DoF thrust generator from total wrench command of
whole flying vehicle u
u
ud. Among various approaches [20–
22, 28, 32], we implement the downwash-aware control allo-
cation method [21] to avoid the large disturbance caused by
downwash flows that counteract other thrust generators while
maintaining high thrust efficiency, critical for a smooth aerial
manipulation, especially when interacting with an object.
In low-level control, two separated PID controllers are
designed to allow each quadcopter to track the desired
tilting and twisting angles, αd
iand βd
i, with tilting torque
commands. This is combined later with the thrust force
command Td
ito determine each actuator’s angular velocity.
Finally, it is converted to a PWM command to drive the
actuators [22].
B. Manipulator Control
With Eq. (3), we design the manipulator’s controller,
τ
τ
τM“M
M
MMpq
q
qMq:q
:q
:qd
M`C
C
CMpq
q
qM,9q
9q
9qMq `G
G
GMpq
q
qMq,(15)
where
:
q
:
q
:
qd
M“:q
:
q
:
qr
M`KM1e
e
eM`KM2
9
e
e
eM`KM3że
e
eMdt,
e
e
eM“q
q
qr
M´q
q
qM,
(16)
where KMi are constant gain matrices.
(a) Keyframes in simulation
(b) Simulation results
Fig. 4: Simulation Task 1: Install a light bulb. The action
sequence of the UAM can be divided into four steps: 1
approach,
2
pick-up, 3
rotate and translate, and 4
feed in.
VI. SIMULATION
A. Setup
Before conducting experiments, we develop a simulation
platform in MATLAB Simulink/Simscape to evaluate the
proposed sequential manipulation planning framework on
our customized over-actuated UAM platform [33, 34]. To
make the simulation as close to the real system as possible,
the simulator incorporates the UAM’s physical parameters,
the dynamics of propeller motors and saturation, control
frequencies, communication noise, measurement noise and
delays (see Tab. I).
B. Results
Fig. 4 and 5 summarize the simulation results of accom-
plishing two sequential aerial manipulation tasks—installing
a light bulb and relocating an object into a closed cabinet—
using the proposed manipulation planning framework.
Task 1: As shown in Fig. 4a, the light bulb installation
task was divided into four steps in our VKC-based motion
planning framework: 1
approach the light bulb, 2
pick it
up with the manipulator, 3
flip the platform to transport the
light bulb to the bottom of the target position, and 4
move
up to install it. A 10-DoF VKC for the UAM platform is
①
⑥
②③
④ ⑤
(a) Keyframes in simulation
(b) Simulation results
Fig. 5: Simulation Task 2: Relocate an object into a cabinet. The
action sequence of the UAM are divided into six steps: 1
approach
to the door, 2
open the door, 3
pick up the object, 4
put the
object into the cabinet, 5
approach to the door, and 6
close the
door.
built—six for the flying vehicle and four for the manipulator.
A feasible and collision-free reference trajectory is acquired
within the physical constraints. As shown in Fig. 4b, with
the hierarchical controller introduced in Sec. V, the proposed
UAM platform can accurately track the reference trajectory
to accomplish the task.
Task 2: Relocating an object into the cabinet requires a
six-step action sequence of the UAM: 1
approach the door
and grasp the handle with the manipulator, 2
open the door,
3
ungrasp the handle and move to pick up the object, 4
put the object into the cabinet, 5
approach and grasp to the
handle again, and 6
close the door. Some keyframes are
shown in Fig. 5, and the planned reference trajectory by the
VKC-based motion planner and the tracking performance are
shown in Fig. 5b.
These simulation results indicate that the VKC-based
motion planning framework and the proposed UAM platform
effectively achieve sequential aerial manipulation.
VII. EXP ER IM EN T
A. Setup
To further demonstrate the sequential aerial manipulation
capability, we conduct experiments by implementing a table
arrangement task in the physical world. Specifically, we use
the Vicon motion capture system (MoCap) to measure the
position and attitude of the UAM platform. The trajectory
planner and main controller of the UAM systems runs on a
remote PC (AMD Ryzen9 5950X CPU, 64 GB RAM), which
communicates with the MoCap through Ethernet and solves
the controller commands with high efficiency. The flying
vehicle’s primary controller is modified from the Crazyflie
python library; it calculates the desired thrust T
T
Td, tilting
angles α
α
αd, and twisting angles β
β
βdfor all quadcopter modules
of the thrust generators and sends them through Crazy Radio
PA antennas (2.4G Hz). Each quadcopter is embedded with
an onboard IMU module. Its firmware is modified to estimate
the rotation angle given the attitude of central frame θ
θ
θ,
regulates the tilting and twisting angles to desired values
with two PID loops, and provides the required thrust with
500 Hz for a fast low-level response.
The manipulator controller is modified from the Robotis
Dynamixel SDK, which runs on the RPi Zero. It receives
commands from the remote PC via WiFi and sends com-
mands to the motors through the Dynamixel U2D2 converter.
Each Dynamixel XC330-M228-T motor has its own con-
troller, and the control mode is set as current control. The
measurement rate of the motion capture system, the remote
PC controller, and the data communication with the UAM
platform is all set to 100 Hz.
B. Results
Fig. 6 summarizes the experimental results of relocating
an object into the drawer, which was divided into six steps:
1
approach to the drawer and grasp the handle with the
manipulator, 2
open the drawer, 3
ungrasp with the handle
and move to pick up the toy, 4
move to the opened drawer
and drop off the toy inside, 5
approach and grasp the
handle again, and 6
close the drawer. The 10-DoF reference
trajectory provided by our VKC-based motion planner was
accurately tracked by the UAM with the hierarchical con-
troller, and the desired commands for each omnidirectional
thrust generator are plotted in Fig. 6b. Fig. 1 and 6a show
some experiment keyframes.
VIII. CONCLUSION
In this paper, we presented a solution to the sequential
aerial manipulation problem of UAMs, an unexplored topic
until now. Unlike previous work in UAM that solves the
motion planning and control problems of one-step manip-
ulation tasks, accomplishing sequential aerial manipulation
③ ④ ⑤
(a) Keyframes in experiment
(b) Experiment results
Fig. 6: Experiment Task: Relocate an object into the drawer. The
action sequence of the UAM is divided into six steps: 1
approach
to the drawer, 2
open the drawer, 3
approach the toy and pick it
up, 4
drop off the toy to the drawer, 5
approach to the drawer
handle, and 6
close the drawer.
requires (i) a highly efficient UAM platform, (ii) a special-
ized motion planner that can well-coordinate motions of the
flying vehicle, the manipulator, and the manipulated object
under varied settings, and (iii) an effective control scheme to
track the desired trajectory. To jointly tackle these challenges,
we designed a novel UAM platform based on an over-
actuated UAV that can achieve omnidirectional flight with
high thrust efficiency. To produce a long sequence of motions
that coordinates well with each other, we extended the idea
of VKC used for ground mobile manipulators and developed
a VKC-based aerial manipulation planning framework for
UAMs. Together with a hierarchical control scheme, we
validated our solution in both simulation and experiment.
The results demonstrated that our approach endowed a new
capability of sequential aerial manipulation for UAMs and
could open up new venues in the field of aerial manipulation.
The integration of a wireless tactile senor [35] with the
manipulator will be the future work for our research, which
extends the manipulation capability of our UAM platform
and enlarges the application range of our the VKC-based
planning framework to more complicated sequential aerial
manipulation tasks.
Acknowledgement: The authors would like to thank Zeyu
Zhang, Zhen Chen, Yangyang Wu, Zihang Zhao, and Hao
Liang at BIGAI and Qing Lei at PKU for their help on
Vicon, figures, and hardware design. This work is sup-
ported in part by the National Key R&D Program of China
(2021ZD0150200) and the Beijing Nova Program.
REFERENCES
[1] M. Fumagalli, R. Naldi, A. Macchelli, F. Forte, A. Q. Keemink,
S. Stramigioli, R. Carloni, and L. Marconi, “Developing an aerial
manipulator prototype: Physical interaction with the environment,”
IEEE Robotics and Automation Magazine (RA-M), vol. 21, no. 3,
pp. 41–50, 2014.
[2] A. Ollero, M. Tognon, A. Suarez, D. Lee, and A. Franchi, “Past,
present, and future of aerial robotic manipulators,” IEEE Transactions
on Robotics (T-RO), 2021.
[3] M. Ryll and R. K. Katzschmann, “Smors: A soft multirotor uav
for multimodal locomotion and robust interaction,” in International
Conference on Robotics and Automation (ICRA), 2022.
[4] G. Jiang, R. Voyles, K. Sebesta, and H. Greiner, “Estimation and
optimization of fully-actuated multirotor platform with nonparallel ac-
tuation mechanism,” in International Conference on Intelligent Robots
and Systems (IROS), 2017.
[5] S. Park, J. Lee, J. Ahn, M. Kim, J. Her, G.-H. Yang, and D. Lee,
“Odar: Aerial manipulation platform enabling omnidirectional wrench
generation,” Transactions on Mechatronics (TMECH), vol. 23, no. 4,
pp. 1907–1918, 2018.
[6] S. Yi, K. Watanabe, and I. Nagai, “Modeling and control of a
fully-actuated quadrotor manipulator with tiltable rotors,” in Recent
Advances in Intelligent Computational Systems (RAICS), 2020.
[7] M. Zhao, K. Okada, and M. Inaba, “Versatile articulated aerial
robot dragon: Aerial manipulation and grasping by vectorable
thrust control,” International Journal of Robotics Research (IJRR),
p. 02783649221112446, 2022.
[8] M. Tognon, E. Cataldi, H. A. T. Chavez, G. Antonelli, J. Cortés, and
A. Franchi, “Control-aware motion planning for task-constrained aerial
manipulation,” IEEE Robotics and Automation Letters (RA-L), vol. 3,
no. 3, pp. 2478–2484, 2018.
[9] K. Bodie, M. Tognon, and R. Siegwart, “Dynamic end effector tracking
with an omnidirectional parallel aerial manipulator,” IEEE Robotics
and Automation Letters (RA-L), vol. 6, no. 4, pp. 8165–8172, 2021.
[10] M. Zhao, K. Nagato, K. Okada, M. Inaba, and M. Nakao, “Force-
ful valve manipulation with arbitrary direction by articulated aerial
robot equipped with thrust vectoring apparatus,” IEEE Robotics and
Automation Letters (RA-L), vol. 7, no. 2, pp. 4893–4900, 2022.
[11] M. Brunner, G. Rizzi, M. Studiger, R. Siegwart, and M. Tognon, “A
planning-and-control framework for aerial manipulation of articulated
objects,” IEEE Robotics and Automation Letters (RA-L), 2022.
[12] N. Sugito, M. Zhao, T. Anzai, T. Nishio, K. Okada, and M. Inaba,
“Aerial manipulation using contact with the environment by thrust
vectorable multilinked aerial robot,” in International Conference on
Robotics and Automation (ICRA), 2022.
[13] D. Berenson, J. Kuffner, and H. Choset, “An optimization approach
to planning for mobile manipulation,” in International Conference on
Robotics and Automation (ICRA), 2008.
[14] S. Chitta, B. Cohen, and M. Likhachev, “Planning for autonomous
door opening with a mobile manipulator,” in International Conference
on Robotics and Automation (ICRA), 2010.
[15] K. Gochev, A. Safonova, and M. Likhachev, “Planning with adaptive
dimensionality for mobile manipulation,” in International Conference
on Robotics and Automation (ICRA), 2012.
[16] Z. Jiao, Z. Zhang, X. Jiang, D. Han, S.-C. Zhu, Y. Zhu, and H. Liu,
“Consolidating kinematic models to promote coordinated mobile ma-
nipulations,” in International Conference on Intelligent Robots and
Systems (IROS), 2021.
[17] Z. Jiao, Z. Zhang, W. Wang, D. Han, S.-C. Zhu, Y. Zhu, and H. Liu,
“Efficient task planning for mobile manipulation: a virtual kinematic
chain perspective,” in International Conference on Intelligent Robots
and Systems (IROS), 2021.
[18] J. Haviland, N. Sünderhauf, and P. Corke, “A holistic approach to
reactive mobile manipulation,” IEEE Robotics and Automation Letters
(RA-L), 2022.
[19] A. Röfer, G. Bartels, W. Burgard, A. Valada, and M. Beetz, “Kin-
everse: A symbolic articulation model framework for model-agnostic
mobile manipulation,” IEEE Robotics and Automation Letters (RA-L),
2022.
[20] Y. Su, P. Yu, M. Gerber, L. Ruan, and T.-C. Tsao, “Nullspace-based
control allocation of overactuated uav platforms,” IEEE Robotics and
Automation Letters (RA-L), vol. 6, no. 4, pp. 8094–8101, 2021.
[21] Y. Su, C. Chu, M. Wang, J. Li, Y. Liu, Y. Zhu, and H. Liu,
“Downwash-aware control allocation for over-actuated uav platforms,”
in International Conference on Intelligent Robots and Systems (IROS),
2022.
[22] P. Yu, Y. Su, M. J. Gerber, L. Ruan, and T.-C. Tsao, “An over-actuated
multi-rotor aerial vehicle with unconstrained attitude angles and high
thrust efficiencies,” IEEE Robotics and Automation Letters (RA-L),
vol. 6, no. 4, pp. 6828–6835, 2021.
[23] P. Yu, Y. Su, L. Ruan, and T.-C. Tsao, “Compensating aerodynamics
of over-actuated multi-rotor aerial platform with data-driven iterative
learning control,” IEEE Robotics and Automation Letters (RA-L)
(submitted), 2023.
[24] C. Pi, L. Ruan, P. Yu, Y. Su, S. Cheng, and T. Tsao, “A simple six
degree-of-freedom aerial vehicle built on quadcopters,” in Proceedings
of IEEE Conference on Control Technology Applications (CCTA),
2021.
[25] J. Luo, Z. Gong, Y. Su, L. Ruan, Y. Zhao, H. H. Asada, and C. Fu,
“Modeling and balance control of supernumerary robotic limb for
overhead tasks,” IEEE Robotics and Automation Letters (RA-L), vol. 6,
no. 2, pp. 4125–4132, 2021.
[26] Z. Jiao, Y. Niu, Z. Zhang, S.-C. Zhu, Y. Zhu, and H. Liu, “Sequential
manipulation planning on scene graph,” in International Conference
on Intelligent Robots and Systems (IROS), 2022.
[27] Z. Jiao, A Virtual Kinematic Chain Perspective for Robot Task and
Motion Planning. PhD thesis, University of California, Los Angeles,
2022.
[28] L. Ruan, C.-H. Pi, Y. Su, P. Yu, S. Cheng, and T.-C. Tsao, “Control
and experiments of a novel tiltable-rotor aerial platform comprising
quadcopters and passive hinges,” Mechatronics, vol. 89, p. 102927,
2023.
[29] Y. Su, L. Ruan, P. Yu, C.-H. Pi, M. J. Gerber, and T.-C. Tsao, “A fast
and efficient attitude control algorithm of a tilt-rotor aerial platform
using inputs redundancies,” IEEE Robotics and Automation Letters
(RA-L), vol. 7, no. 2, pp. 1214–1221, 2021.
[30] Y. Su, P. Yu, M. Gerber, L. Ruan, and T. Tsao, “Fault-tolerant control
of an over-actuated uav platform built on quadcopters and passive
hinges,” Transactions on Mechatronics (TMECH), 2023.
[31] Y. Su, Compensation and Control Allocation with Input Saturation
Limits and Rotor Faults for Multi-Rotor Copters with Redundant
Actuations. PhD thesis, University of California, Los Angeles, 2021.
[32] M. Ryll, H. H. Bülthoff, and P. R. Giordano, “A novel overactuated
quadrotor unmanned aerial vehicle: Modeling, control, and experimen-
tal validation,” IEEE Transactions on Control Systems Technology,
vol. 23, no. 2, pp. 540–556, 2014.
[33] Y. Su, Z. Jiao, Z. Zhang, C. Chu, J. Li, H. Li, M. Wang, and
H. Liu, “Flight structure optimization of modular reconfigurable uavs,”
in International Conference on Robotics and Automation (ICRA)
(submitted), 2024.
[34] Y. Su, C. Chu, J. Zhang, H. Li, M. Wang, and H. Liu, “Marvel:
Modular unmanned aerial vehicles can achieve full actuation through
self-reconfiguration,” in International Conference on Robotics and
Automation (ICRA) (submitted), 2024.
[35] W. Li, M. Wang, J. Li, Y. Su, D. K. Jia, X. Qian, A. Kaspar, and H. Liu,
“L3 f-touch: A wireless gelsight with decoupled tactile and three-axis
force sensing,” IEEE Robotics and Automation Letters (RA-L), 2023.
