Conference PaperPDF Available

Edge Computing for Mobile Robots: Multi-Robot Feature-Based Lidar Odometry with FPGAs

Authors:

Abstract

Offloading computationally intensive tasks such as lidar or visual odometry from mobile robots has multiple benefits. Resource constrained robots can make use of their network capabilities to reduce the data processing load and be able to perform a larger number tasks in a more efficient manner. However, previous works have mostly focused on cloud offloading, which increases latency and reduces reliability, or high-end edge devices. Instead, we explore the utilization of FPGAs at the edge for computational offloading with minimal latency and high parallelism. We present the potential for modelling feature-based odometry in VHDL and utilizing FPGA implementations.
Edge Computing for Mobile Robots:
Multi-Robot Feature-Based Lidar Odometry with FPGAs
L. Qingqing1, F. Yuhong1, J. Pe˜
na Queralta2, T. N. Gia2, H. Tenhunen3, Z. Zou1and T. Westerlund2
1School of Information Science and Technology, Fudan Universtiy, China
2Department of Future Technologies, University of Turku, Finland
3Department of Electronics, KTH Royal Institute of Technology, Sweden
Emails: 1{qingqingli16, zhuo}@fudan.edu.cn, 2{yuhofu, jopequ, tunggi, tovewe}@utu.fi, 3hannu@kth.se
Abstract—Offloading computationally intensive tasks such as
lidar or visual odometry from mobile robots has multiple benefits.
Resource constrained robots can make use of their network
capabilities to reduce the data processing load and be able to
perform a larger number tasks in a more efficient manner.
However, previous works have mostly focused on cloud offloading,
which increases latency and reduces reliability, or high-end edge
devices. Instead, we explore the utilization of FPGAs at the
edge for computational offloading with minimal latency and high
parallelism. We present the potential for modelling feature-based
odometry in VHDL and utilizing FPGA implementations.
Index Terms—Odometry; Autonomous Robots; FPGA; Lidar;
Lidar Odometry; Laser rangefinder; SLAM; Mobile Robots;
I. INTRODUCTION
The penetration of lidars in the robotic industries has
considerably grown over the last two decades [1]. Efficient
lidar odometry was proposed by Zhang et al. and has been
extensively used in the development of autonomous robots
and vehicles [2]. However, processing lidar data for odometry
and mapping purposes requires on-board computers. A recent
approach is to exploit the robot’s connectivity to offload
computationally intensive tasks to cloud servers. However,
this increases latency and reduces reliability. Therefore, we
study the offloading at the local network level, in order to
tightly control the communication latency and provide a robust
solution. With this approach, a single gateway can perform
odometry calculations for multiple robots in real-time [3].
However, as the number of robots connected to the same
gateway increases, a much more powerful processor is needed,
and the control over the system latency can be partially lost.
We propose the utilization of FPGAs as their parallelism
enables rapid scaling without an impact on performance.
The edge computing paradigm is a distributed network
paradigm that aims at moving computational processing power
and data analytics closer to where the data originates, in
the so-called edge of the network. In a hybrid Edge-Cloud
architecture, part of the data processing is done at the local
network level and only the analysis results are transmitted
to cloud servers. This allows for reduced bandwidth and
increased reliability. In robotics, this paradigm can increase
the capabilities of resource-constrained robots by streaming
sensor data and performing the analysis at edge gateways with
higher parallelist and power efficiency [4].
II. RE LATE D WORK
Offloading computationally intensive sensor data processing
has been extensively studied in mobile robot navigation.
Through the simplification of on-board hardware, significant
saving in energy consumption can be achieved, incrementing
the robot’s battery life [3]. Integrating edge computing with
mobile robot navigation is a relatively recent field and the
related works are few. From a more generic point of view,
Dey et al. studied the potential benefits of offloading compu-
tationally intensive tasks within simultaneous localization and
mapping (SLAM) algorithms with edge computing [5]. The
authors concluded that edge computing can bring significant
enhancements to localization and positioning algorithms with
low latency and high reliability services. Most other works
have been focusing on offloading to cloud servers. Nonethe-
less, network connectivity to cloud servers can be unreliable
and latency uncontrollable in some situations. Therefore, it is
not suitable for time-critical applications such as localization
and movement estimation in autonomous robots.
III. FEATU RE -BASED LIDAR O DO METRY ON FPGAS
Lidar odometry algorithms use the lidar’s data to compute
the motion of a robot between two consecutive sweeps. In
general, lidar odometry algorithms can be divided into three
steps. The first step is to extract features from lidar data, the
features can be the geometric distributions, or some stable
points which can be observed in the two consecutive sweeps.
The next step is to find the feature correspondence through the
position difference between sweeps. The last step is estimating
the lidar movement through the time between two sweeps [2].
FPGAs have the ability to process lidar data in real time
with a limited resource utilization, and naturally parallelize
the processing of data from multiple lidars [6].
Our goal in this work is to design a pure VHDL implemen-
tation. Instead of comparing complete lidar sweeps, as most
recent implementations do, we aim at a implementation that
analyzes lidar data as it is available and compares features in
real-time. With this, we expect to be able to increase odometry
accuracy and the positioning update frequency.
IV. EXP ER IM EN T AN D RES ULTS
In order to test the efficiency and usability of the feature-
based odometry algorithm, we have first implemented in
LUT LUTRAM FF IO DSP
9.14
0.55 0.74
8.8
15.45
”Resource Utilization (%)”
Fig. 1. FPGA Resource Utilization Summary
C++ within the Robot Operating System (ROS). ROS is the
most popular open-source meta-operating system for robots
to support code reuse in the development of robots. We have
utilized the inexpensive RPLidar A1 for our experiment, a
360° two-dimensional lidar with 1° resolution at 10Hz.
A. FPGA Implementation
We have utilized a Zybo Z7-20 board for implementing
our algorithm. This relatively small board is built around the
Z-7010, the smallest chip in the Xilinx Zynq-7000 family.
Even though the board integrates a dual-core ARM Cortex-A9
processor, we only use the FPGA logic in order to provide a
generic design that can be easily ported to other platforms.
Implementing the proposed algorithm using VHDL hard-
ware modelling presents some challenges. On one side, the
need for calculation of trigonometric functions. In order to
solve this, we utilize coordinate rotation digital computer
(CORDIC) algorithms implemented in VHDL. In particular,
we generate sine samples and from those calculate the values
for cosine. On the other side, conversion types and integrating
the CORDIC calculations into a state machine, requiring
complimentary intermediate signals.
B. FPGA Resource Utilization
In order to study the potential of the FPGA-based imple-
mentation to be parallelized, we have synthesized an initial
and unoptimized version of the design. The preliminary re-
sults show that the main resource utilization occurs with the
IO banks (8.8%), Logic LUTs (9.14%) and DSP modules
(15.45%). Nonetheless, the IO banks can be easily multi-
plexed, and additional lidars can be connected with a single
input, so they do not represent a significant limitation. More-
over, a single nRF or Wi-Fi receiver can be utilized to receive
information from multiple units. In terms of DSP utilization,
the modules are used in the CORDIC implementations. A
single CORDIC module can be shared among multiple parallel
processes with a relatively simple state machine. Therefore, the
proposed algorithm is not limited by the number of available
DSPs. Similarly, we have estimated that around 90% of the
LUTs can be shared. This is possible because the timing
constraints that the frequency of the lidar scanner impose
are very relaxed compared to the maximum performance that
the VHDL implementation can deliver. Therefore, we expect
that a single FPGA board will be able to run in parallel
over 50 lidar odometry calculations. This can be combined
TABLE I
PERFORMANCE COMPARISON BETWE EN FPGA AND CPU
IMPLEMENTATIONS
Xilinx Zynq XC7Z010 Intel Atom x5-Z8350
(VHDL Impl.) (C++ Impl.)
Aprox. Resource 10% (fixed) + 1% 4% (fixed) + 7% CPU
Max. concurrency >20-50* <5-15
*Expected range with optimized code.
TABLE II
FPGA RESOURCE UTI LIZ ATIO N BREA KD OWN
Resource Type Used Available
Slice LUTs 4961 53200
LUT as Logic 4865 53200
LUT as Memory 96 17400
Slice Registers 791 106400
F7 Muxes 32 26600
F8 Muxes 0 13300
Bonded IOB 11 125
DSPs 34 220
with wireless communication solutions that provide enough
available channels, such as Wi-Fi or nRF. In contrast, on an
Intel Atom x5-Z8350 CPU @ 1.44GHz × 4, the proposed
algorithm can be executed approximately 5 to 15 times in
parallel, depending on whether other processes are being run.
V. CONCLUSION AND FUTURE WORK
We have presented preliminary work on an odometry of-
floading solution for multi-robot systems. We have designed
and implemented a feature-based lidar odometry algorithm that
is flexible and can accommodate to a variety of indoors or
outdoors environments, and implemented in an FPGA with
pure VHDL modeling. The results presented in this paper show
potential for high parallelism and low-latency odometry.
In future work, we will integrate this solution within a
real team of mobile robots and test their navigability with
minimum hardware on-board and edge offloading.
ACKNOWLEDGMENT
This work has been supported by NFSC grant No.
61876039, and the Shanghai Platform for Neuromorphic and
AI Chip (NeuHeilium).
REFERENCES
[1] S. A. Hiremath et al. Laser range finder model for autonomous navigation
of a robot in a maize field using a particle filter. Computers and
Electronics in Agriculture, 100:41 – 50, 2014.
[2] J. Zhang et al. Loam: Lidar odometry and mapping in real-time. In
Robotics: Science and Systems, volume 2, page 9, 2014.
[3] V. K. Sarker et al. Offloading slam for indoor mobile robots with edge-
fog-cloud computing. In ICASERT, 2019.
[4] S. Biookaghazadeh et al. Are fpgas suitable for edge computing? In
USENIX HotEdge Workshop, 2018.
[5] S. Dey et al. Robotic slam: a review from fog computing and mobile edge
computing perspective. In MobiQuitous, pages 153–158. ACM, 2016.
[6] J. Pe˜
na Queralta et al. FPGA-based Architecture for a Low-Cost 3D Lidar
Design and Implementation from Multiple Rotating 2D Lidars with ROS.
In IEEE SENSORS 2019 (SENSORS), 2019.
... By placing the time-sensitive robotics applications at the edge of the network, MEC and IIC can ensure high computation capabilities while leveraging low-latency communication and high bandwidth. Most of the recent experimental work focuses on distributing computation between the robots and the edge of the network, in particular, deep object recognition and grasp planning [16], visual odometry [17], [25], real-time control [18], [27], [28], [19], [21] and object detection [26], [20]. Besides the experimental work, several studies have motivated the application of MEC and IIC for different robotics systems by proposing collaborative architectures (e.g., healthcare [9], autonomy [10], tele-surgery [11], manufacturing [12][13][14] and multi robot systems [24]). ...
... The close proximity of MEC and IIC has been studied in some recent existing experimental studies to offload the location-based robotics services from mobile robots with limited computational and low-energy resources [17], [25], [29][30][31]. Additionally, [12][13][14], [22], [24] elaborate on the reduced network pressure and improved security and privacy of the robot sensor data that can be offered by restricting the access within a trusted private infrastructure. ...
Article
Full-text available
Cloud-based robotics systems leverage a wide range of Information Technologies (IT) to offer tangible benefits like cost reduction, powerful computational capabilities, data offloading, etc. However, the centralized nature of cloud computing is not well-suited for a multitude of Operational Technologies (OT) nowadays used in robotics systems that require strict real-time guarantees and security. Edge computing and fog computing are complementary approaches that aim at mitigating some of these challenges by providing computing capabilities closer to the users. The goal of this work is hence threefold: i) to analyze the current edge computing and fog computing landscape in the context of robotics systems, ii) to experimentally evaluate an end-to-end robotics system based on solutions proposed in the literature, and iii) to experimentally identify current benefits and open challenges of edge computing and fog computing. Results show that, in the case of an exemplary delivery application comprising two mobile robots, the robot coordination and range can be improved by consuming real-time radio information available at the edge. However, our evaluation highlights that the existing software, wireless and virtualization technologies still require substantial evolution to fully support edge-based robotics systems.
... Edge computing can provide more benefits in some special cases; for example, a self-driving car needs to make a decision within a short time, so edge computing is the best solution. For autonomous driving of mobile robots, the authors in [216,217], used to reduce latency and increase reliability. In addition, the car can communicate with the infrastructure and can also use its on-board processing system to talk to other vehicles around it [218]. ...
... It is important to compare the accuracy and efficiency of various models on these devices. The test results can provide necessary insights to advance technologies in edge AI applications such as autonomous vehicles [5] and mobile robots [102]. ...
Article
Full-text available
Moving object segmentation (MOS) is the process of identifying dynamic objects from video frames, such as moving vehicles or pedestrians, while discarding the background. It plays an essential role in many real-world applications such as autonomous driving, mobile robots, and surveillance systems. With the availability of a huge amount of data and the development of powerful computing infrastructure, deep learning-based methods have shown remarkable improvements in MOS tasks. However, as the dimension of data becomes higher and the network architecture becomes more complicated, deep learning-based MOS models are computationally intensive, which limits their deployment on resource-constrained devices and in delay-sensitive applications. Therefore, more research started to develop fast and lightweight models. This paper aims to provide a comprehensive review of deep learning-based MOS models, with a focus on efficient model design techniques. We summarize a variety of MOS datasets, and conduct a thorough review of segmentation accuracy metrics and model efficiency metrics. Most importantly, we compare the performance of efficient MOS models on popular datasets, identify competitive models and analyze their essential techniques. Finally, we point out existing challenges and present future research directions.
... The first real-time FPGA application field is the FeatureBased Visual Odometry (FBVO), which is a system in which a robot determines its pose in a certain space, through the visualized images. A LiDAR-based FBVO implementation on FPGA was performed by Qingqing et al. 84 , evaluating the parallelization of this architecture in order to host more than one LiDAR. These estimations were made considering the usage of the FPGA internal resources and comparing it with an Intel Atom x5-Z8350 processor. ...
Article
This work aims to provide details on the latest technological developments regarding LiDAR (Light Imaging Detection And Ranging) systems, with particular reference to the techniques, architectures, and methodologies partially or entirely implemented by means of the FPGA (Field Programmable Gate Array) environment. Currently, LiDAR technology is considered of great interest as it is widely employed in a variety of application fields, such as automotive, seismology, archaeology, metrology, and military. For this reason, the required performances are gradually increasing, which leads to complex and stringent solutions. The growth in LiDAR systems’ complexity suggests the use of high-end general-purpose computing units such as central processing units to perform very complex tasks and FPGAs to perform multiple tasks in real-time through the implementation of dedicated computational blocks. The latter, in recent architectures, are therefore used for the execution of specific tasks that require high computational speed and system flexibility. This paper reports some case studies recently applied in the LiDAR field, with the aim of illustrating the role of FPGA technology and its benefits.
Article
Full-text available
The COVID-19 pandemic has had a massive impact on the global aviation industry. As a result, the airline industry has been forced to embrace new technologies and procedures in order to provide a more secure and bio-safe travel. Currently, the role of smart technology in airport systems has expanded significantly as a result of the contemporary Industry 4.0 context. The article presents a novel construction of an intelligent mobile robot system to guide passengers to take the plane at the departure terminals at busy airports. The robot provides instructions to the customer through the interaction between the robot and the customer utilizing voice communications. The usage of the Google Cloud Speech-to-Text API combined with technical machine learning to analyze and understand the customer's requirements are deployed. In addition, we use a face detection technique based on Multi-task Cascaded Convolutional Networks (MTCNN) to predict the distance between the robot and passengers to perform the function. The robot can guide passengers to desired areas in the terminal. The results and evaluation of the implementation process are also mentioned in the article and show promise.
Article
Full-text available
The collaboration between ground and flying robots can take advantage of the capabilities of each system to enhance the performance of robotic applications, including military, healthcare, disaster management, etc. Unfortunately, this cooperation is restricted by the physical constraint that all computation should be performed on robots. The concept of computation offloading has great potential to improve the performance of both ground and flying robots. Unfortunately, external conditions like the network conditions, the robot’s mobility, and the availability of the processing resources may lead to new challenges. Accordingly, these challenges should be addressed from different perspectives, like security, network communication, response time, and energy consumption. Recently, most computation offloading solutions aim to optimize further the robot’s energy consumption. Several research works are designed 1.) to satisfy the real-time requirements of robotic applications and 2.) to solve the trade-off between the energy consumed by computation and communication. To better understand these concepts, we present a comprehensive overview of the computation offloading process for ground and flying robots. We also devised a taxonomy explaining the factors affecting the offloading decision in a robotic scenario. The taxonomy presents guidelines to recognize the scope of research in offloading decisions that were designed for robots. Then, we discuss the state-of-the-art techniques of computation offloading from an architectural point of view, and we survey works related to offloading decisions for robots.
Article
The optical phased array (OPA) chip is becoming the core component of all-solid-state lidars. In order to realize the wide-angle range beam scanning and rapid control of OPA-based solid-state lidars, we design and fabricate a 64-channel silicon-based OPA chip based on thermo-optical modulation. In accordance with the optical phase control characteristics of the multi-channel thermo-optic OPA chip, we propose a parallel random access memory (RAM) array technique to realize high-speed phase control for the OPA chip and build the hardware system with a field-programmable logic gate array (FPGA). This system is used to test the main performance indicators of the OPA chip and realizes the high-speed scanning control of the wide-angle range of the OPA output beam. The test results show that the OPA chip we designed can achieve fast beam scanning within a large angle range between -32.5 ~ +32.5°, and the chip has high working reliability. Combined with the control system we build, it can meet the high-speed radar scanning requirements of all-solid-state lidars. This work lays a good foundation for the subsequential development of our thermo-optical and electro-optic OPA all-solid-state lidar prototypes.
Conference Paper
Full-text available
Three-dimensional representations and maps are the key behind self-driving vehicles and many types of advanced autonomous robots. Localization and mapping algorithms can achieve much higher levels of accuracy with dense 3D point clouds. However, the cost of a multiple-channel three-dimensional lidar with a 360°field of view is at least ten times the cost of an equivalent single-channel two-dimensional lidar. Therefore, while 3D lidars have become an essential component of self-driving vehicles, their cost has limited their integration and penetration within smaller robots. We present an FPGA-based 3D lidar built with multiple inexpensive RPLidar A1 2D lidars, which are rotated via a servo motor and their signals combined with an FPGA board. A C++ package for the Robot Operating System (ROS) has been written, which publishes a 3D point cloud. The mapping of points from the two-dimensional lidar output to the three-dimensional point cloud is done at the FPGA level, as well as continuous calibration of the motor speed and lidar orientation based on a built-in landmark recognition. This inexpensive design opens a wider range of possibilities for lower-end and smaller autonomous robots, which can be able to produce three-dimensional world representations. We demonstrate the possibilities of our design by mapping different environments.
Conference Paper
Full-text available
Indoor mobile robots are widely used in industrial environments such as large logistic warehouses. They are often in charge of collecting or sorting products. For such robots, computation-intensive operations account for a significant percentage of the total energy consumption and consequently affect battery life. Besides, in order to keep both the power consumption and hardware complexity low, simple micro-controllers or single-board computers are used as onboard local control units. This limits the computational capabilities of robots and consequently their performance. Offloading heavy computation to Cloud servers has been a widely used approach to solve this problem for cases where large amounts of sensor data such as real-time video feeds need to be analyzed. More recently, Fog and Edge computing are being leveraged for offloading tasks such as image processing and complex navigation algorithms involving non-linear mathematical operations. In this paper, we present a system architecture for offloading computationally expensive localization and mapping tasks to smart Edge gateways which use Fog services. We show how Edge computing brings computational capabilities of the Cloud to the robot environment without compromising operational reliability due to connection issues. Furthermore, we analyze the power consumption of a prototype robot vehicle in different modes and show how battery life can be significantly improved by moving the processing of data to the Edge layer.
Article
Full-text available
The rapid growth of Internet-of-things (IoT) and artificial intelligence applications have called forth a new computing paradigm--edge computing. In this paper, we study the suitability of deploying FPGAs for edge computing from the perspectives of throughput sensitivity to workload size, architectural adaptiveness to algorithm characteristics, and energy efficiency. This goal is accomplished by conducting comparison experiments on an Intel Arria 10 GX1150 FPGA and an Nvidia Tesla K40m GPU. The experiment results imply that the key advantages of adopting FPGAs for edge computing over GPUs are three-fold: 1) FPGAs can provide a consistent throughput invariant to the size of application workload, which is critical to aggregating individual service requests from various IoT sensors; (2) FPGAs offer both spatial and temporal parallelism at a fine granularity and a massive scale, which guarantees a consistently high performance for accelerating both high-concurrency and high-dependency algorithms; and (3) FPGAs feature 3-4 times lower power consumption and up to 30.7 times better energy efficiency, offering better thermal stability and lower energy cost per functionality.
Conference Paper
Offloading computationally expensive Simultaneous Localization and Mapping (SLAM) task for mobile robots have attracted significant attention during the last few years. Lack of powerful on-board compute capability in these energy constrained mobile robots and rapid advancement in compute cloud access technologies laid the foundation for development of several Cloud Robotics platforms that enabled parallel execution of computationally expensive robotic algorithms, especially involving multiple robots. In this work the Cloud Robotics concept is extended to include the current emphasis of computing at the network edge nodes along with the Cloud. The requirements and advantages of using edge nodes for computation offloading over remote cloud or local robot clusters are discussed with reference to the ETSI 'Mobile-Edge Computing' initiative and OpenFog Consortium's 'OpenFog Architecture'. A Particle Filter algorithm for SLAM is modified and implemented for offloading in a multi-tier edge+cloud setup. Additionally a model is proposed for offloading decision in such a setup with experiments and results demonstrating the efficacy of the proposed dynamic offloading scheme over static offloading strategies.
Article
Autonomous navigation of robots in an agricultural environment is a difficult task due to the inherent uncertainty in the environment. Many existing agricultural robots use computer vision and other sensors to supplement Global Positioning System (GPS) data when navigating. Vision based methods are sensitive to ambient lighting conditions. This is a major disadvantage in an outdoor environment. The current study presents a novel probabilistic sensor model for a 2D range finder (LIDAR) from first principles. Using this sensor model, a particle filter based navigation algorithm (PF) for autonomous navigation in a maize field was developed. The algorithm was tested in various field conditions with varying plant sizes, different row patterns and at several scanning frequencies. Results showed that the Root Mean Squared Error of the robot heading and lateral deviation were equal to 2.4 degrees and 0.04 m, respectively. It was concluded that the performance of the proposed navigation method is robust in a semi-structured agricultural environment.
Are fpgas suitable for edge computing?
  • S Biookaghazadeh
S. Biookaghazadeh et al. Are fpgas suitable for edge computing? In USENIX HotEdge Workshop, 2018.