Conference PaperPDF Available

Edge Computing for Mobile Robots: Multi-Robot Feature-Based Lidar Odometry with FPGAs

Authors:

Abstract

Offloading computationally intensive tasks such as lidar or visual odometry from mobile robots has multiple benefits. Resource constrained robots can make use of their network capabilities to reduce the data processing load and be able to perform a larger number tasks in a more efficient manner. However, previous works have mostly focused on cloud offloading, which increases latency and reduces reliability, or high-end edge devices. Instead, we explore the utilization of FPGAs at the edge for computational offloading with minimal latency and high parallelism. We present the potential for modelling feature-based odometry in VHDL and utilizing FPGA implementations.
Edge Computing for Mobile Robots:
Multi-Robot Feature-Based Lidar Odometry with FPGAs
L. Qingqing1, F. Yuhong1, J. Pe˜
na Queralta2, T. N. Gia2, H. Tenhunen3, Z. Zou1and T. Westerlund2
1School of Information Science and Technology, Fudan Universtiy, China
2Department of Future Technologies, University of Turku, Finland
3Department of Electronics, KTH Royal Institute of Technology, Sweden
Emails: 1{qingqingli16, zhuo}@fudan.edu.cn, 2{yuhofu, jopequ, tunggi, tovewe}@utu.fi, 3hannu@kth.se
Abstract—Offloading computationally intensive tasks such as
lidar or visual odometry from mobile robots has multiple benefits.
Resource constrained robots can make use of their network
capabilities to reduce the data processing load and be able to
perform a larger number tasks in a more efficient manner.
However, previous works have mostly focused on cloud offloading,
which increases latency and reduces reliability, or high-end edge
devices. Instead, we explore the utilization of FPGAs at the
edge for computational offloading with minimal latency and high
parallelism. We present the potential for modelling feature-based
odometry in VHDL and utilizing FPGA implementations.
Index Terms—Odometry; Autonomous Robots; FPGA; Lidar;
Lidar Odometry; Laser rangefinder; SLAM; Mobile Robots;
I. INTRODUCTION
The penetration of lidars in the robotic industries has
considerably grown over the last two decades [1]. Efficient
lidar odometry was proposed by Zhang et al. and has been
extensively used in the development of autonomous robots
and vehicles [2]. However, processing lidar data for odometry
and mapping purposes requires on-board computers. A recent
approach is to exploit the robot’s connectivity to offload
computationally intensive tasks to cloud servers. However,
this increases latency and reduces reliability. Therefore, we
study the offloading at the local network level, in order to
tightly control the communication latency and provide a robust
solution. With this approach, a single gateway can perform
odometry calculations for multiple robots in real-time [3].
However, as the number of robots connected to the same
gateway increases, a much more powerful processor is needed,
and the control over the system latency can be partially lost.
We propose the utilization of FPGAs as their parallelism
enables rapid scaling without an impact on performance.
The edge computing paradigm is a distributed network
paradigm that aims at moving computational processing power
and data analytics closer to where the data originates, in
the so-called edge of the network. In a hybrid Edge-Cloud
architecture, part of the data processing is done at the local
network level and only the analysis results are transmitted
to cloud servers. This allows for reduced bandwidth and
increased reliability. In robotics, this paradigm can increase
the capabilities of resource-constrained robots by streaming
sensor data and performing the analysis at edge gateways with
higher parallelist and power efficiency [4].
II. RE LATE D WORK
Offloading computationally intensive sensor data processing
has been extensively studied in mobile robot navigation.
Through the simplification of on-board hardware, significant
saving in energy consumption can be achieved, incrementing
the robot’s battery life [3]. Integrating edge computing with
mobile robot navigation is a relatively recent field and the
related works are few. From a more generic point of view,
Dey et al. studied the potential benefits of offloading compu-
tationally intensive tasks within simultaneous localization and
mapping (SLAM) algorithms with edge computing [5]. The
authors concluded that edge computing can bring significant
enhancements to localization and positioning algorithms with
low latency and high reliability services. Most other works
have been focusing on offloading to cloud servers. Nonethe-
less, network connectivity to cloud servers can be unreliable
and latency uncontrollable in some situations. Therefore, it is
not suitable for time-critical applications such as localization
and movement estimation in autonomous robots.
III. FEATU RE -BASED LIDAR O DO METRY ON FPGAS
Lidar odometry algorithms use the lidar’s data to compute
the motion of a robot between two consecutive sweeps. In
general, lidar odometry algorithms can be divided into three
steps. The first step is to extract features from lidar data, the
features can be the geometric distributions, or some stable
points which can be observed in the two consecutive sweeps.
The next step is to find the feature correspondence through the
position difference between sweeps. The last step is estimating
the lidar movement through the time between two sweeps [2].
FPGAs have the ability to process lidar data in real time
with a limited resource utilization, and naturally parallelize
the processing of data from multiple lidars [6].
Our goal in this work is to design a pure VHDL implemen-
tation. Instead of comparing complete lidar sweeps, as most
recent implementations do, we aim at a implementation that
analyzes lidar data as it is available and compares features in
real-time. With this, we expect to be able to increase odometry
accuracy and the positioning update frequency.
IV. EXP ER IM EN T AN D RES ULTS
In order to test the efficiency and usability of the feature-
based odometry algorithm, we have first implemented in
LUT LUTRAM FF IO DSP
9.14
0.55 0.74
8.8
15.45
”Resource Utilization (%)”
Fig. 1. FPGA Resource Utilization Summary
C++ within the Robot Operating System (ROS). ROS is the
most popular open-source meta-operating system for robots
to support code reuse in the development of robots. We have
utilized the inexpensive RPLidar A1 for our experiment, a
360° two-dimensional lidar with 1° resolution at 10Hz.
A. FPGA Implementation
We have utilized a Zybo Z7-20 board for implementing
our algorithm. This relatively small board is built around the
Z-7010, the smallest chip in the Xilinx Zynq-7000 family.
Even though the board integrates a dual-core ARM Cortex-A9
processor, we only use the FPGA logic in order to provide a
generic design that can be easily ported to other platforms.
Implementing the proposed algorithm using VHDL hard-
ware modelling presents some challenges. On one side, the
need for calculation of trigonometric functions. In order to
solve this, we utilize coordinate rotation digital computer
(CORDIC) algorithms implemented in VHDL. In particular,
we generate sine samples and from those calculate the values
for cosine. On the other side, conversion types and integrating
the CORDIC calculations into a state machine, requiring
complimentary intermediate signals.
B. FPGA Resource Utilization
In order to study the potential of the FPGA-based imple-
mentation to be parallelized, we have synthesized an initial
and unoptimized version of the design. The preliminary re-
sults show that the main resource utilization occurs with the
IO banks (8.8%), Logic LUTs (9.14%) and DSP modules
(15.45%). Nonetheless, the IO banks can be easily multi-
plexed, and additional lidars can be connected with a single
input, so they do not represent a significant limitation. More-
over, a single nRF or Wi-Fi receiver can be utilized to receive
information from multiple units. In terms of DSP utilization,
the modules are used in the CORDIC implementations. A
single CORDIC module can be shared among multiple parallel
processes with a relatively simple state machine. Therefore, the
proposed algorithm is not limited by the number of available
DSPs. Similarly, we have estimated that around 90% of the
LUTs can be shared. This is possible because the timing
constraints that the frequency of the lidar scanner impose
are very relaxed compared to the maximum performance that
the VHDL implementation can deliver. Therefore, we expect
that a single FPGA board will be able to run in parallel
over 50 lidar odometry calculations. This can be combined
TABLE I
PERFORMANCE COMPARISON BETWE EN FPGA AND CPU
IMPLEMENTATIONS
Xilinx Zynq XC7Z010 Intel Atom x5-Z8350
(VHDL Impl.) (C++ Impl.)
Aprox. Resource 10% (fixed) + 1% 4% (fixed) + 7% CPU
Max. concurrency >20-50* <5-15
*Expected range with optimized code.
TABLE II
FPGA RESOURCE UTI LIZ ATIO N BREA KD OWN
Resource Type Used Available
Slice LUTs 4961 53200
LUT as Logic 4865 53200
LUT as Memory 96 17400
Slice Registers 791 106400
F7 Muxes 32 26600
F8 Muxes 0 13300
Bonded IOB 11 125
DSPs 34 220
with wireless communication solutions that provide enough
available channels, such as Wi-Fi or nRF. In contrast, on an
Intel Atom x5-Z8350 CPU @ 1.44GHz × 4, the proposed
algorithm can be executed approximately 5 to 15 times in
parallel, depending on whether other processes are being run.
V. CONCLUSION AND FUTURE WORK
We have presented preliminary work on an odometry of-
floading solution for multi-robot systems. We have designed
and implemented a feature-based lidar odometry algorithm that
is flexible and can accommodate to a variety of indoors or
outdoors environments, and implemented in an FPGA with
pure VHDL modeling. The results presented in this paper show
potential for high parallelism and low-latency odometry.
In future work, we will integrate this solution within a
real team of mobile robots and test their navigability with
minimum hardware on-board and edge offloading.
ACKNOWLEDGMENT
This work has been supported by NFSC grant No.
61876039, and the Shanghai Platform for Neuromorphic and
AI Chip (NeuHeilium).
REFERENCES
[1] S. A. Hiremath et al. Laser range finder model for autonomous navigation
of a robot in a maize field using a particle filter. Computers and
Electronics in Agriculture, 100:41 – 50, 2014.
[2] J. Zhang et al. Loam: Lidar odometry and mapping in real-time. In
Robotics: Science and Systems, volume 2, page 9, 2014.
[3] V. K. Sarker et al. Offloading slam for indoor mobile robots with edge-
fog-cloud computing. In ICASERT, 2019.
[4] S. Biookaghazadeh et al. Are fpgas suitable for edge computing? In
USENIX HotEdge Workshop, 2018.
[5] S. Dey et al. Robotic slam: a review from fog computing and mobile edge
computing perspective. In MobiQuitous, pages 153–158. ACM, 2016.
[6] J. Pe˜
na Queralta et al. FPGA-based Architecture for a Low-Cost 3D Lidar
Design and Implementation from Multiple Rotating 2D Lidars with ROS.
In IEEE SENSORS 2019 (SENSORS), 2019.
... By placing the time-sensitive robotics applications at the edge of the network, MEC and IIC can ensure high computation capabilities while leveraging low-latency communication and high bandwidth. Most of the recent experimental work focuses on distributing computation between the robots and the edge of the network, in particular, deep object recognition and grasp planning [16], visual odometry [17], [25], real-time control [18], [27], [28], [19], [21] and object detection [26], [20]. Besides the experimental work, several studies have motivated the application of MEC and IIC for different robotics systems by proposing collaborative architectures (e.g., healthcare [9], autonomy [10], tele-surgery [11], manufacturing [12][13][14] and multi robot systems [24]). ...
... The close proximity of MEC and IIC has been studied in some recent existing experimental studies to offload the location-based robotics services from mobile robots with limited computational and low-energy resources [17], [25], [29][30][31]. Additionally, [12][13][14], [22], [24] elaborate on the reduced network pressure and improved security and privacy of the robot sensor data that can be offered by restricting the access within a trusted private infrastructure. ...
Article
Full-text available
Cloud-based robotics systems leverage a wide range of Information Technologies (IT) to offer tangible benefits like cost reduction, powerful computational capabilities, data offloading, etc. However, the centralized nature of cloud computing is not well-suited for a multitude of Operational Technologies (OT) nowadays used in robotics systems that require strict real-time guarantees and security. Edge computing and fog computing are complementary approaches that aim at mitigating some of these challenges by providing computing capabilities closer to the users. The goal of this work is hence threefold: i) to analyze the current edge computing and fog computing landscape in the context of robotics systems, ii) to experimentally evaluate an end-to-end robotics system based on solutions proposed in the literature, and iii) to experimentally identify current benefits and open challenges of edge computing and fog computing. Results show that, in the case of an exemplary delivery application comprising two mobile robots, the robot coordination and range can be improved by consuming real-time radio information available at the edge. However, our evaluation highlights that the existing software, wireless and virtualization technologies still require substantial evolution to fully support edge-based robotics systems.
... The first real-time FPGA application field is the FeatureBased Visual Odometry (FBVO), which is a system in which a robot determines its pose in a certain space, through the visualized images. A LiDAR-based FBVO implementation on FPGA was performed by Qingqing et al. 84 , evaluating the parallelization of this architecture in order to host more than one LiDAR. These estimations were made considering the usage of the FPGA internal resources and comparing it with an Intel Atom x5-Z8350 processor. ...
Article
This work aims to provide details on the latest technological developments regarding LiDAR (Light Imaging Detection And Ranging) systems, with particular reference to the techniques, architectures, and methodologies partially or entirely implemented by means of the FPGA (Field Programmable Gate Array) environment. Currently, LiDAR technology is considered of great interest as it is widely employed in a variety of application fields, such as automotive, seismology, archaeology, metrology, and military. For this reason, the required performances are gradually increasing, which leads to complex and stringent solutions. The growth in LiDAR systems’ complexity suggests the use of high-end general-purpose computing units such as central processing units to perform very complex tasks and FPGAs to perform multiple tasks in real-time through the implementation of dedicated computational blocks. The latter, in recent architectures, are therefore used for the execution of specific tasks that require high computational speed and system flexibility. This paper reports some case studies recently applied in the LiDAR field, with the aim of illustrating the role of FPGA technology and its benefits.
... Active research areas in TIERS include multi-robot coordination [1], [2], [3], [4], [5], swarm design [6], [7], [8], [9], UWB-based localization [10], [11], [12], [13], [14], [15], localization and navigation in unstructured environments [16], [17], [18], lightweight AI at the edge [19], [20], [21], [22], [23], distributed ledger technologies at the edge [24], [25], [26], [27], [28], [29], edge architectures [30], [31], [32], [33], [34], [35], offloading for mobile robots [36], [37], [38], [39], [40], [41], [42], LPWAN networks [43], [44], [45], [46], sensor fusion algorithms [47], [48], [49], and reinforcement and federated learning for multi-robot systems [50], [51], [52], [53]. ...
... All these systems are highly distributed. The evolution from the conventional cloud-centric architecture has been accelerated by the emergence of the edge computing technologies [19,20]. A unified standard classification is defined to ensure the consistency of the development and structures of IoT. ...
Article
Full-text available
Providing security and privacy to the Internet of Things (IoT) networks while achieving it with minimum performance requirements is an open research challenge. Blockchain technology, as a distributed and decentralized ledger, is a potential solution to tackle the limitations of the current peer-to-peer IoT networks. This paper presents the development of an integrated IoT system implementing the permissioned blockchain Hyperledger Fabric (HLF) to secure the edge computing devices by employing a local authentication process. In addition, the proposed model provides traceability for the data generated by the IoT devices. The presented solution also addresses the IoT systems’ scalability challenges, the processing power and storage issues of the IoT edge devices in the blockchain network. A set of built-in queries is leveraged by smart-contracts technology to define the rules and conditions. The paper validates the performance of the proposed model with practical implementation by measuring performance metrics such as transaction throughput and latency, resource consumption, and network use. The results show that the proposed platform with the HLF implementation is promising for the security of resource-constrained IoT devices and is scalable for deployment in various IoT scenarios.
Chapter
Mobile edge computing (MEC) and next-generation mobile networks are set to disrupt the way intelligent and autonomous systems are interconnected. This will have an effect on a wide range of domains, from the Internet of Things to autonomous mobile robots. The integration of such a variety of MEC services in an inherently distributed architecture requires a robust system for managing hardware resources, balancing the network load and securing the distributed applications. Blockchain technology has emerged a solution for managing MEC services, with consensus protocols and data integrity checks that enable transparent and efficient distributed decision-making. In addition to transparency, the benefits from a security point of view are evident. Nonetheless, blockchain technology faces significant challenges in terms of scalability. In this chapter, we review existing consensus protocols and scalability techniques in both well-established and next-generation blockchain architectures. From this, we evaluate the most suitable solutions for managing MEC services and discuss the benefits and drawbacks of the available alternatives.
Article
Fog robotics is an emerging topic that derives from cloud robotics, but similarly as fog computing, the applications require low latency connections in order to be useful in real life environments. This article presents a new perspective to the topic in which not only robotics takes advantage of the fog computing paradigm, but fog computing is able to leverage the robotics technology in order to enhance its features. This work highlights the benefits obtained by each technology when it is mixed with the other and sketches the relevant topics to research in order to make this partnership possible.
Article
This review paper focuses on cooperative robotic arms with mobile or drone bases performing cooperative tasks. This is because cooperative robots are often used as risk-reduction tools to human life. For example, they are used to explore dangerous places such as minefields and disarm explosives. Drones can be used to perform tasks such as aerial photography, military and defense missions, agricultural surveys, etc. The bases of the cooperative robotic arms can be stationary, mobile (ground), or drones. Cooperative manipulators allow faster performance of assigned tasks because of the available “extra hand”. Furthermore, a mobile base increases the reachable ground workspace of cooperative manipulators while a drone base drastically increases this workspace to include the aerial space. The papers in this review are chosen to extensively cover a wide variety of cooperative manipulation tasks and industries that use them. In cooperative manipulation, avoiding self-collision is one of the most important tasks to be performed. In addition, path planning and formation control can be challenging because of the increased number of components to be coordinated.
Article
Proliferation of Industrial Internet has dramatically changed the way we live and work. It brings convenience to our society and sometimes requires real-time processing of dramatic data at the same time. However, traditional paradigm of computing on the center cloud can’t always meet such requirement, for the non-negligible time delay of data transmission and communication. Edge computing is a novel computing paradigm proposed to resolve such a problem. As a promising technology, it extends computing from cloud center to the edge of network. Edge computing has the advantage of low latency to achieve a shorter response time, as well as potential to address the concerns of energy consuming, bandwidth burden and security issue. In this paper, we give a survey about edge computing from the aspect of methodologies, application scenarios and its role in Industrial Internet. Some open issues of edge computing are also introduced in this paper. At the end of the manuscript, a discussion about future direction is proposed. The shallow network algorithms such as broad learning system (BLS), which have achieved great improvement in computing efficiency, show an optimistic outlook in this area. We propose our conceive about future applications when shallow network methods like BLS are applied in edge computing and hope the paper will inspire research in relative directions.
Conference Paper
Full-text available
Three-dimensional representations and maps are the key behind self-driving vehicles and many types of advanced autonomous robots. Localization and mapping algorithms can achieve much higher levels of accuracy with dense 3D point clouds. However, the cost of a multiple-channel three-dimensional lidar with a 360°field of view is at least ten times the cost of an equivalent single-channel two-dimensional lidar. Therefore, while 3D lidars have become an essential component of self-driving vehicles, their cost has limited their integration and penetration within smaller robots. We present an FPGA-based 3D lidar built with multiple inexpensive RPLidar A1 2D lidars, which are rotated via a servo motor and their signals combined with an FPGA board. A C++ package for the Robot Operating System (ROS) has been written, which publishes a 3D point cloud. The mapping of points from the two-dimensional lidar output to the three-dimensional point cloud is done at the FPGA level, as well as continuous calibration of the motor speed and lidar orientation based on a built-in landmark recognition. This inexpensive design opens a wider range of possibilities for lower-end and smaller autonomous robots, which can be able to produce three-dimensional world representations. We demonstrate the possibilities of our design by mapping different environments.
Conference Paper
Full-text available
Indoor mobile robots are widely used in industrial environments such as large logistic warehouses. They are often in charge of collecting or sorting products. For such robots, computation-intensive operations account for a significant percentage of the total energy consumption and consequently affect battery life. Besides, in order to keep both the power consumption and hardware complexity low, simple micro-controllers or single-board computers are used as onboard local control units. This limits the computational capabilities of robots and consequently their performance. Offloading heavy computation to Cloud servers has been a widely used approach to solve this problem for cases where large amounts of sensor data such as real-time video feeds need to be analyzed. More recently, Fog and Edge computing are being leveraged for offloading tasks such as image processing and complex navigation algorithms involving non-linear mathematical operations. In this paper, we present a system architecture for offloading computationally expensive localization and mapping tasks to smart Edge gateways which use Fog services. We show how Edge computing brings computational capabilities of the Cloud to the robot environment without compromising operational reliability due to connection issues. Furthermore, we analyze the power consumption of a prototype robot vehicle in different modes and show how battery life can be significantly improved by moving the processing of data to the Edge layer.
Article
Full-text available
The rapid growth of Internet-of-things (IoT) and artificial intelligence applications have called forth a new computing paradigm--edge computing. In this paper, we study the suitability of deploying FPGAs for edge computing from the perspectives of throughput sensitivity to workload size, architectural adaptiveness to algorithm characteristics, and energy efficiency. This goal is accomplished by conducting comparison experiments on an Intel Arria 10 GX1150 FPGA and an Nvidia Tesla K40m GPU. The experiment results imply that the key advantages of adopting FPGAs for edge computing over GPUs are three-fold: 1) FPGAs can provide a consistent throughput invariant to the size of application workload, which is critical to aggregating individual service requests from various IoT sensors; (2) FPGAs offer both spatial and temporal parallelism at a fine granularity and a massive scale, which guarantees a consistently high performance for accelerating both high-concurrency and high-dependency algorithms; and (3) FPGAs feature 3-4 times lower power consumption and up to 30.7 times better energy efficiency, offering better thermal stability and lower energy cost per functionality.
Conference Paper
Offloading computationally expensive Simultaneous Localization and Mapping (SLAM) task for mobile robots have attracted significant attention during the last few years. Lack of powerful on-board compute capability in these energy constrained mobile robots and rapid advancement in compute cloud access technologies laid the foundation for development of several Cloud Robotics platforms that enabled parallel execution of computationally expensive robotic algorithms, especially involving multiple robots. In this work the Cloud Robotics concept is extended to include the current emphasis of computing at the network edge nodes along with the Cloud. The requirements and advantages of using edge nodes for computation offloading over remote cloud or local robot clusters are discussed with reference to the ETSI 'Mobile-Edge Computing' initiative and OpenFog Consortium's 'OpenFog Architecture'. A Particle Filter algorithm for SLAM is modified and implemented for offloading in a multi-tier edge+cloud setup. Additionally a model is proposed for offloading decision in such a setup with experiments and results demonstrating the efficacy of the proposed dynamic offloading scheme over static offloading strategies.
Article
Autonomous navigation of robots in an agricultural environment is a difficult task due to the inherent uncertainty in the environment. Many existing agricultural robots use computer vision and other sensors to supplement Global Positioning System (GPS) data when navigating. Vision based methods are sensitive to ambient lighting conditions. This is a major disadvantage in an outdoor environment. The current study presents a novel probabilistic sensor model for a 2D range finder (LIDAR) from first principles. Using this sensor model, a particle filter based navigation algorithm (PF) for autonomous navigation in a maize field was developed. The algorithm was tested in various field conditions with varying plant sizes, different row patterns and at several scanning frequencies. Results showed that the Root Mean Squared Error of the robot heading and lateral deviation were equal to 2.4 degrees and 0.04 m, respectively. It was concluded that the performance of the proposed navigation method is robust in a semi-structured agricultural environment.
Are fpgas suitable for edge computing?
  • S Biookaghazadeh
S. Biookaghazadeh et al. Are fpgas suitable for edge computing? In USENIX HotEdge Workshop, 2018.