Erdal Kayacan

• PhD
• Professor (Full) at Paderborn University

Autonomous underwater vehicles (AUVs) present several challenges due to the complex and simultaneous interplay of various factors, including but not limited to unmodeled dynamics , highly nonlinear behavior, intercouplings, communication delays, and environmental disturbances. In particular, environmental disturbances degrade trajectory tracking pe...

With the rise of deep learning, there is a fundamental change in visual SLAM algorithms toward developing different modules trained as end-to-end pipelines. However, regardless of the implementation domain, visual SLAM's performance is subject to diverse environmental challenges, such as dynamic elements in outdoor environments, harsh imaging condi...

This study presents the conflict-aware multi-agent estimated time of arrival (CAMETA) framework, a novel approach for predicting the arrival times of multiple agents in unstructured environments without predefined road infrastructure. The CAMETA framework consists of three components: a path planning layer generating potential path suggestions, a m...

With increasingly challenging applications for quadrotors, higher requirements are emerging for tracking accuracy and safety. While high accuracy is a prerequisite for complex tasks, safety is ensured through tolerance to actuator faults and resistance to external disturbances. In this article, adaptive robust control (ARC) integrated with Gaussian...

Automated visual inspection of on- and off-shorewind turbines using aerial robots provides several benefits, namely, a safe working environment by circumventing the need for workers to be suspended high above the ground, reduced inspection time, preventive maintenance, and access to hard-to-reach areas. A novel nonlinear model predictive control (N...

We propose an anomaly detection framework that generates an anomaly score for the current observation of a task. The proposed framework can overcome two major problems in smart manufacturing systems, where a robot learns task execution by human expert demonstrations. First, it can function as a warning system to alert human operators when the robot...

In this study, a novel end-to-end path planning algorithm based on deep reinforcement learning is proposed for aerial robots deployed in dense environments. The learning agent finds an obstacle-free way around the provided rough, global path by only depending on the observations from a forward-facing depth camera. A novel deep reinforcement learnin...

In autonomous and mobile robotics, one of the main challenges is the robust on-the-fly perception of the environment, which is often unknown and dynamic, like in autonomous drone racing. In this work, we propose a novel deep neural network-based perception method for racing gate detection – PencilNet ¹ ¹ Source code, trained...

In autonomous and mobile robotics, one of the main challenges is the robust on-the-fly perception of the environment, which is often unknown and dynamic, like in autonomous drone racing. In this work, we propose a novel deep neural network-based perception method for racing gate detection -- PencilNet -- which relies on a lightweight neural network...

Event-based vision has already revolutionized the perception task for robots by promising faster response, lower energy consumption, and lower bandwidth without introducing motion blur. In this work, a novel deep learning method based on gated recurrent units utilizing sparse convolutions for detecting gates in a race track is proposed using event-...

In this work, we present the design, fabrication, and experimental validation of a lightweight, low inertia dual-arm manipulator with a center of gravity (COG) balancing mechanism, specifically designed for aerial manipulation missions. The developed system, consisting of the dual-arm base and two arms with 6 degrees of freedom (DOFs) each, weighs...

Existing Deep Learning (DL) frameworks typically do not provide ready-to-use solutions for robotics, where very specific learning, reasoning, and embodiment problems exist. Their relatively steep learning curve and the different methodologies employed by DL compared to traditional approaches, along with the high complexity of DL models, which often...

A adaptive artificial neural network (ANN) boundary control scheme is developed for an autonomous aerial refueling hose system in this study. The objectives of this paper are 1) to guarantee the state of the system uniformly bounded, 2) to make the system output meet the prescribed performance, that is, −χa(t)<y(l(t),t)<χb(t). The proposed novel co...

Long-term autonomy is of great importance in various real-world applications of aerial robotics, including, but not limited to, search and rescue missions in underground mines, detection, and monitoring of victims trapped under collapsed buildings, aerial radiation detection in nuclear power plants after an accident, or extraterrestrial exploration...

The papers in this special section focus on artificial intelligence (AI) applications for robotics. AI technologies, covering cognition, analysis, inference, and decision-making, enable robots to act smartly and greatly enhance robots’ capabilities to assist and support humans. As a primary carrier of AI technologies, robotics is one of the applica...

Motivated by the difficulty roboticists experience while tuning model predictive controllers (MPCs), we present an automated weight set tuning framework in this work. The enticing feature of the proposed methodology is the active exploration approach that adopts the exploration–exploitation concept at its core. Essentially, it extends the trial-and...

In this work we propose a holistic framework for autonomous aerial inspection tasks, using semantically-aware, yet, computationally efficient planning and mapping algorithms. The system leverages state-of-the-art receding horizon exploration techniques for next-best-view (NBV) planning with geometric and semantic segmentation information provided b...

In this work we propose a holistic framework for autonomous aerial inspection tasks, using semantically-aware, yet, computationally efficient planning and mapping algorithms. The system leverages state-of-the-art receding horizon exploration techniques for next-best-view (NBV) planning with geometric and semantic segmentation information provided b...

In this work, we propose a centralized nonlinear model predictive control (NMPC) method to facilitate fault-tolerant control (FTC) of an over-actuated quadrotor against a propeller failure. Thanks to the novel mechanical design, the hyperdynamic quadrotor can independently command and control all 6-degrees-of-freedom (DoFs). Additionally, the under...

In the maritime sector, safe vessel navigation is of great importance, particularly in congested harbors and waterways. The focus of this work is to estimate the distance between an object of interest and potential obstacles using a companion UAV. The proposed approach fuses GPS data with long-range aerial images. First, we employ semantic segmenta...

In this work we showcase the design and assessment of the performance of a multi-camera UAV, when coupled with state-of-the-art planning and mapping algorithms for autonomous navigation. The system leverages state-of-the-art receding horizon exploration techniques for Next-Best-View (NBV) planning with 3D and semantic information, provided by a rec...

Gradient descentGradient Descent (GD) is a computational optimization method which is based on the first-order Taylor expansion of nonlinear functions. In order to find a local minimum for a nonlinear function, this algorithm uses the initial parameters of the nonlinear function and updates these parameters in the negative direction of the gradient...

The feedback linearization method can be used to control nonlinear systems without linearizing them around a specific equilibrium point. This controller includes a nonlinear feedback controller to eliminate the nonlinear terms. The mathematical preliminaries to design a feedback linearization controller is established in this chapter. The following...

Fuzzy logicFuzzy logic has proved itself as an advanced model-free approach with tremendous impact on control community. Fuzzy logicFuzzy logic has the ability to handle uncertainties, lack of modeling, and operational disturbances in a control system using expert knowledge. In this chapter, rule-based sliding mode fuzzy logic controllers are desig...

Optimization is the selectionSelection process of the best elements with respect to some criterion from a feasible set of variables. There may be single or multiple objectives to be considered during optimization. The optimization process generally involves the minimization of a cost or maximization of a profit. Sliding mode controller design probl...

Lack of imprecise nonlinear model of real-time systems is inevitable due to several simplifications made, neglected frictions, dead-zones, and saturation. One of the most well-known nonlinear control design tools to deal with uncertainties is sliding-mode control approach. In this method the desired behavior is defined in terms of a sliding manifol...

Control of a system—for example, operation of a robot—through a communication link from a distant location is called teleoperationTeleoperation. Remote operations in hazardous and unreachable areas are crucial and inevitable; examples of such areas are complex and challenging tasks in disaster areas, space robotic applications, remotely operated ve...

This chapter deals with adaptive design of fuzzy controllers based on sliding-modeSliding mode control law. As it was mentioned earlier, a challenge to design a sliding- modeSliding mode controller is necessity to have the nominal dynamics of the system. This requires a series of modeling prior to the control of the system. Fuzzy logic systems as g...

Centralized direct digital control systemsDirect digital control systems have several drawbacks, for example, massive wiring requirements, difficult diagnosis, and difficult fault detection procedures. Most of these drawbacks may impose heavy costs on the maintenance of the control system. These disadvantages have given rise to the development of N...

The word “fuzzy” means imprecisely defined, confused, and vague. However, fuzzy systems benefit from mathematical formulation to determine their output. This chapter deals with type-1 fuzzy systems as well as interval type-2 fuzzy systems in which different parts of these fuzzy systems are explained.

In this work we showcase the design and assessment of the performance of a multi-camera UAV, when coupled with state-of-the-art planning and mapping algorithms for autonomous navigation. The system leverages state-of-the-art receding horizon exploration techniques for Next-Best-View (NBV) planning with 3D and semantic information, provided by a rec...

This paper addresses the trajectory tracking problem of an autonomous tractor-trailer system by using a fast distributed nonlinear model predictive control algorithm in combination with nonlinear moving horizon estimation for the state and parameter estimation in which constraints on the inputs and the states can be incorporated. The proposed contr...

More efficient agricultural machinery is needed as agricultural areas become more limited and energy and labor costs increase. To increase their efficiency, trajectory tracking problem of an autonomous tractor, as an agricultural production machine, has been investigated in this study. As a widely used model-based approach, model predictive control...

The detection of contextual anomalies is a challenging task for surveillance since an observation can be considered anomalous or normal in a specific environmental context. An unmanned aerial vehicle (UAV) can utilize its aerial monitoring capability and employ multiple sensors to gather contextual information about the environment and perform cont...

As a model is only an abstraction of the real system, unmodeled dynamics, parameter variations, and disturbances can result in poor performance of a conventional controller based on this model. In such cases, a conventional controller cannot remain well tuned. This paper presents the control of a spherical rolling robot by using an adaptive neuro-f...

Provision of some autonomous functions to an agricultural vehicle would lighten the job of the operator but in doing so, the accuracy should not be lost to still obtain an optimal yield. Autonomous navigation of an agricultural vehicle involves the control of different dynamic subsystems, such as the yaw angle dynamics and the longitudinal speed dy...

In order to achieve faster and more robust convergence (especially under noisy working environments), a sliding mode theory-based learning algorithm has been proposed to tune both the premise and consequent parts of type-2 fuzzy neural networks in this paper. Differently from recent studies, where sliding mode control theory-based rules are propose...

One of the most critical tasks in tractor operation is the accurate steering during field operations, e.g., accurate trajectory following during mechanical weeding or spraying, to avoid damaging the crop or planting when there is no crop yet. To automate the trajectory following problem of an autonomous tractor-trailer system and also increase its...

This paper addresses the trajectory tracking problem of an autonomous tractor-trailer system by using a decentralized control approach. A fully decentralized model predictive controller is designed in which interactions between subsystems are neglected and assumed to be perturbations to each other. In order to have a robust design, a tube-based app...

Nonlinear control system theory and various design techniques are widely used in the aerospace arena, especially in developing aircraft and spacecraft guidance, navigation, and control systems. Such systems have been critical to the success of many aerospace systems and will continue to be in the future. The design of these systems involves advance...

We present a deep autoencoder-based anomaly detection method (GridNet) for indoor surveillance. Unlike similar studies, GridNet is image-agnostic by taking a specific representation of a scene as inputs instead of the raw image itself. Its input is grid representations of scene images, which indicate spatial layouts of objects in a scene. This appr...

This book addresses some of the challenges suffered by the well-known and robust sliding-mode control paradigm. The authors show how the fusion of fuzzy systems with sliding-mode controllers can alleviate some of these problems and promote applicability. Fuzzy systems used as soft switches eliminate high-frequency signal oscillations and can subst...

Anomaly detection is a key goal of autonomous surveillance systems that should be able to alert unusual observations. In this paper, we propose a holistic anomaly detection system using deep neural networks for surveillance of critical infrastructures (e.g., airports, harbors, warehouses) using an unmanned aerial vehicle (UAV). First, we present a...

Drone racing is a recreational sport in which the goal is to pass through a sequence of gates in a minimum amount of time while avoiding collisions. In autonomous drone racing, one must accomplish this task by flying fully autonomously in an unknown environment by relying only on computer vision methods for detecting the target gates. Due to the ch...

To generate 3D virtual maps of outcrops in geo-science, manual flights of aerial robots are often employed which is challenging due to various reasons: 1) piloted flight over curved/uneven surfaces requires auto-focusing, 2) wind disturbances make it difficult to precisely maintain the desired overlap, and 3) hiring a skilled pilot is expensive as...

In this work, a learning model-free control method is proposed for accurate trajectory tracking and safe landing of unmanned aerial vehicles (UAVs). A realistic scenario is considered where the UAV commutes between stations at high-speeds, experiences a single motor failure while surveying an area, and thus requires to land safely at a designated s...

Unmanned aerial vehicles (UAVs) with mounted cameras have the advantage of capturing aerial (bird-view) images. The availability of aerial visual data and the recent advances in object detection algorithms led the computer vision community to focus on object detection tasks on aerial images. As a result of this, several aerial datasets have been in...

Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple expert human operators. Hence, there is a demand for an autonomous...

To facilitate accurate tracking in unknown/uncertain environments, this paper proposes a simple learning (SL) strategy for feedback linearization control (FLC) of aerial robots subject to uncertainties. The SL strategy minimizes a cost function defined based on the closed-loop error dynamics of the nominal system via the gradient descent technique...

Rather than utilizing a sophisticated robot which is trained—and tuned—for a scenario in a specific environment perfectly, most people are interested in seeing robots operating in various conditions where they have never been trained before. In accordance with the goal of utilizing aerial robots for daily operations in real application scenarios, a...

In this work, our goal is to use an online learning-based nonlinear model predictive control (NMPC) for systems with uncertain and/or time-varying parameters. We have deployed it for two robotic applications in real-time: an agricultural off-road ground vehicle and an aerial robotic system, namely a tilt-rotor tricopter unmanned aerial vehicle. Non...

Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple expert human operators. Hence, there is demand for an autonomous ci...

This work proposes a novel, learning-based method to leverage navigation time performance of unmanned aerial vehicles in dense environments by planning swift maneuvers using motion primitives. In the proposed planning framework, desirable motion primitives are explored by reinforcement learning. Two-stage training composed of learning in simulation...

In this work, a novel, end-to-end motion planning method is proposed for quadrotor navigation in cluttered environments. The proposed method circumvents the explicit sensing-reconstructing-planning in contrast to conventional navigation algorithms. It uses raw depth images obtained from a front-facing camera and directly generates local motion plan...

A nonlinear unsteady aerodynamics model is coupled with a three degree of freedom quadplane to control the forward and backward transition between hover and steady level flight. The unsteady lift and drag forces are modeled using a lumped vortex model for flat plates. Two variants for the quadplane are considered: (i) a pusher and (ii) a tilt-rotor...

This work presents an online learning method for improved control of nonlinear systems by combining deep learning and fuzzy logic. Given the ability of deep learning to generalise knowledge from training samples, the proposed method requires minimum amount of information about the system to be controlled. However, in robotics, particularly in aeria...

We present an artificial intelligence-based control approach, the fusion of artificial neural networks and type-2 fuzzy logic controllers, namely type-2 fuzzy-neural networks, for the outer adaptive position controller of unmanned aerial manipulators. The performance comparison of proportional-derivative (PD) controller working alone and the propos...

This paper presents a detailed aerodynamic modeling technique along with a fuzzy switching multi-model guidance and control strategy for a custom blended wing-body tilt-rotor unmanned aerial vehicle (UAV). The tilt-rotor configuration affects the overall aerodynamic characteristics significantly and hence, a comprehensive mathematical model is requ...

This work presents an online learning-based control method for improved trajectory tracking of unmanned aerial vehicles using both deep learning and expert knowledge. The proposed method does not require the exact model of the system to be controlled, and it is robust against variations in system dynamics as well as operational uncertainties. The l...

Although a considerable amount of eort has been put in to show that fuzzy logic controllers have exceptional capabilities of dealing with uncertainty, there are still noteworthy concerns, e.g., the design of fuzzy logic controllers is an arduous task due to the lack of closed-form input-output relationships which is a limitation to interpretability...

Aerial filming is becoming more and more popular thanks to the recent advances in drone technology. It invites many intriguing, unsolved problems at the intersection of aesthetical and scientific challenges. In this work, we propose an intelligent agent which supervises motion planning of a filming drone based on aesthetical values of video shots u...

In this paper, we propose an online learning approach that enables the inverse dynamics model learned for a source robot to be transferred to a target robot (e.g., from one quadrotor to another quadrotor with different mass or aerodynamic properties). The goal is to leverage knowledge from the source robot such that the target robot achieves high-a...

This paper presents an online adaptive tracker, which employs a novel weighted multiple instance learning (WMIL) approach.In the proposed tracker, both positive and negative sample importances are integrated into an online learning mechanism for improving tracking performance in challenging environments. The sample importance is computed based on a...

The principal objective of this study is to provide an insight into the simulation of fused deposition modeling (FDM) parts considering the influence of build and print orientations. The elastic modulus, strength and Poisson's ratio at different build and print orientations are obtained by performing uniaxial tensile tests. Based on the results, an...

Quality assessment during postconstruction of buildings is an indispensable procedure in construction industry. This paper describes the design and development of a quality inspection and assessment robot (QuicaBot) that can autonomously scan the entire room using cameras and laser scanners to pick up building defects, such as hollowness, crack, ev...

This paper presents a faster RRT-based path planning approach for regular 2-dimensional (2D) building environments. To minimize the planning time, we adopt the idea of biasing the RRT tree-growth in more focused ways. We propose to calculate the skeleton of the 2D environment first, then connect a geometrical path on the skeleton, and grow the RRT...

Input uncertainty, e.g., noise on the on-board camera and inertial measurement unit, in vision-based control of unmanned aerial vehicles (UAVs) is an inevitable problem. In order to handle input uncertainties as well as further analyze the interaction between the input and the antecedent fuzzy sets (FSs) of non-singleton fuzzy logic controllers (NS...

In a typical agricultural field operation, an agricultural vehicle must be accurately navigated to achieve an optimal result by covering with minimal overlap during tillage, fertilizing and spraying. To this end, a small scale tractor-trailer system is equipped by using off the shelf sensors and actuators to design a fully autonomous agricultural v...

Whereas type-1 and type-2 membership functions (MFs) are the core of any fuzzy logic system, there are no performance criteria available to evaluate the goodness or correctness of the fuzzy MFs. In this paper, we make extensive analysis in terms of the capability of type-2 elliptic fuzzy MFs in modeling uncertainty. Having decoupled parameters for...

This paper presents a novel application of a hybrid learning approach to the optimisation of membership and non-membership functions of a newly developed interval type-2 intuitionistic fuzzy logic system (IT2 IFLS) of a Takagi-Sugeno-Kang (TSK) fuzzy inference system with neural network learning capability. The hybrid algorithms consisting of decou...