Fig 1 - uploaded by Yaxiong Yuan
Content may be subject to copyright.
Context in source publication
Context 1
... number of users and the k-th user's demands in the n-th cluster. When all the demands in the current cluster are satisfied, the UAV will leave for the next cluster. After the service, the UAV flies back to the dock station and prepares for the next round. New users may arrive during the service. Their requests will be processed in the next round. Fig. 1 illustrates an example of the system model. The delivery process evolves over a sequence of frames whose structure is standardized. Since the data collected by the UAV have a certain life span, in each round, all the tasks must be completed within a limited time T L (in frames). We assume that a frame lasts T F (in seconds) and ...
Similar publications
Citations
... UAV-MECs are a promising technology due to flexible wireless connectivity and coverage even in the absence of network infrastructure. Despite the fact that UAV-MECs have numerous applications ranging from mobile relay BS to caching and MEC cloudlets, it is critical to thoroughly investigate UAV trajectory optimization [220] [221] [222], hovering altitude [223] [224], and speed control [225] [226]. The UAV-MEC mobility causes them to leave the coverage area of the serviced MDs, which may increase latency [227]; additionally, there is a large amount of data to be offloaded from MDs concerning the available bandwidth on both UAV-MECS and the backhaul network [228]. ...
The lack of resource constraints for edge servers makes it difficult to simultaneously perform many Mobile Devices' (MDs) requests. The Mobile Network Operator (MNO) must then select how to delegate MD queries to its Mobile Edge Computing (MEC) server to maximize the overall benefit of admitted requests with varying latency needs. Unmanned Aerial Vehicles (UAVs) and Artificial Intelligent (AI) can increase MNO performance because of their flexibility in deployment, high mobility of UAV, and efficiency of AI algorithms. There is a trade-off between the cost incurred by the MD and the profit received by the MNO. Intelligent computing offloading to UAV-enabled MEC, on the other hand, is a promising way to bridge the gap between MDs' limited processing resources, as well as the intelligent algorithms that are utilized for computation offloading in the UAV-MEC network and the high computing demands of upcoming applications. This study looks at some of the research on the benefits of computation offloading process in the UAV-MEC network, as well as the intelligent models that are utilized for computation offloading in the UAV-MEC network. In addition, this article examines several intelligent pricing techniques in different structures in the UAV-MEC network. Finally, this work highlights some important open research issues and future research directions of Artificial Intelligent (AI) in computation offloading and applying intelligent pricing strategies in the UAV-MEC network.
... UAV-MECs are a promising technology due to flexible wireless connectivity and coverage even in the absence of network infrastructure. Despite the fact that UAV-MECs have numerous applications ranging from mobile relay BS to caching and MEC cloudlets, it is critical to thoroughly investigate UAV trajectory optimization [220] [221] [222], hovering altitude [223] [224], and speed control [225] [226]. The UAV-MEC mobility causes them to leave the coverage area of the serviced MDs, which may increase latency [227]; additionally, there is a large amount of data to be offloaded from MDs concerning the available bandwidth on both UAV-MECS and the backhaul network [228]. ...
The Mobile Network Operator (MNO) must select how to delegate Mobile Device (MD) queries to its Mobile Edge Computing (MEC) server in order to maximize the overall benefit of admitted requests with varying latency needs. Unmanned Aerial Vehicles (UAVs) and Artificial Intelligent (AI) can increase MNO performance because of their flexibility in deployment, high mobility of UAV, and efficiency of AI algorithms. There is a trade-off between the cost incurred by the MD and the profit received by the MNO. Intelligent computing offloading to UAV-enabled MEC, on the other hand, is a promising way to bridge the gap between MDs' limited processing resources, as well as the intelligent algorithms that are utilized for computation offloading in the UAV-MEC network and the high computing demands of upcoming applications. This study looks at some of the research on the benefits of computation offloading process in the UAV-MEC network, as well as the intelligent models that are utilized for computation offloading. In addition, this article examines several intelligent pricing techniques in different structures in the UAV-MEC network. Finally, this work highlights some important open research issues and future research directions of Artificial Intelligent (AI) in computation offloading and applying intelligent pricing strategies in the UAV-MEC network.
... Year Category Specific Algorithm CA ML [232] 2018 Heuristic [233] 2021 Heuristic [29] 2020 Heuristic [234] 2020 Heuristic [235] 2020 Heuristic [236] 2016 Heuristic [237] 2018 Heuristic [238] 2020 Heuristic [30] 2020 DDPG [239], [240] 2020, 2021 Actor-critic as well as timely recharging of the UAV battery. A DRL algorithm based on DDPG was employed to solve the joint optimization of both the mobility and charging cycle of the UAV-BSs, leading to maximization of EE and achievement of fair user coverage. ...
... A DRL algorithm based on DDPG was employed to solve the joint optimization of both the mobility and charging cycle of the UAV-BSs, leading to maximization of EE and achievement of fair user coverage. The work in [239] and [240] considered the problem of user scheduling in UAV-based communication networks to minimize the energy consumption of the UAVs. An offline method, based on branch and bound method, was proposed to solve the problem, however, this approach involves a huge computational overhead. ...
Wireless communication networks have been witnessing an unprecedented demand due to the increasing number of connected devices and emerging bandwidth-hungry applications. Albeit many competent technologies for capacity enhancement purposes, such as millimeter wave communications and network densification, there is still room and need for further capacity enhancement in wireless communication networks, especially for the cases of unusual people gatherings, such as sport competitions, musical concerts, etc. Unmanned aerial vehicles (UAVs) have been identified as one of the promising options to enhance the capacity due to their easy implementation, pop up fashion operation, and cost-effective nature. The main idea is to deploy base stations on UAVs and operate them as flying base stations, thereby bringing additional capacity to where it is needed. However, because the UAVs mostly have limited energy storage, their energy consumption must be optimized to increase flight time. In this survey, we investigate different energy optimization techniques with a top-level classification in terms of the optimization algorithm employed; conventional and machine learning (ML). Such classification helps understand the state of the art and the current trend in terms of methodology. In this regard, various optimization techniques are identified from the related literature, and they are presented under the above mentioned classes of employed optimization methods. In addition, for the purpose of completeness, we include a brief tutorial on the optimization methods and power supply and charging mechanisms of UAVs. Moreover, novel concepts, such as reflective intelligent surfaces and landing spot optimization, are also covered to capture the latest trend in the literature.
... In [15], the authors propose a DRL-based method which is composed of two deep neural networks (DNNs) and deep deterministic policy gradient (DDPG) to maximize the energy efficiency for a group of UAVs by jointly considering communications coverage, energy consumption, and connectivity. In order to minimize the UAV's transmission and hovering energy, the authors in [16] formulate the energyefficient optimization problem as a Markov decision process. Then, they use two DNNs and the actor-critic-based RL algorithm to develop an online DRL algorithm that shows a good performance in terms of energy savings. ...
Unmanned aerial vehicles (UAVs) have emerged as a promising candidate solution for data collection of large-scale wireless sensor networks (WSNs). In this paper, we investigate a UAV-aided WSN, where cluster heads (CHs) receive data from their member nodes, and a UAV is dispatched to collect data from CHs along the planned trajectory. We aim to minimize the total energy consumption of the UAV-WSN system in a complete round of data collection. Toward this end, we formulate the energy consumption minimization problem as a constrained combinatorial optimization problem by jointly selecting CHs from nodes within clusters and planning the UAV's visiting order to the selected CHs. The formulated energy consumption minimization problem is NP-hard, and hence, hard to solve optimally. In order to tackle this challenge, we propose a novel deep reinforcement learning (DRL) technique, pointer network-A* (Ptr-A*), which can efficiently learn from experiences the UAV trajectory policy for minimizing the energy consumption. The UAV's start point and the WSN with a set of pre-determined clusters are fed into the Ptr-A*, and the Ptr-A* outputs a group of CHs and the visiting order to these CHs, i.e., the UAV's trajectory. The parameters of the Ptr-A* are trained on small-scale clusters problem instances for faster training by using the actor-critic algorithm in an unsupervised manner. At inference, three search strategies are also proposed to improve the quality of solutions. Simulation results show that the trained models based on 20-clusters and 40-clusters have a good generalization ability to solve the UAV's trajectory planning problem in WSNs with different numbers of clusters, without the need to retrain the models. Furthermore, the results show that our proposed DRL algorithm outperforms two baseline techniques.
... The conventional AC-DRL algorithms have limitations on tackling constrained combinatorial optimization problems, which may result in slow convergent, infeasible, and degraded solutions. The authors in [20], [21] developed AC-DRL algorithms for a combinatorial optimization problem in a UAV-aided system, where the performance is limited when the problem scale grows. In [22], [23], two AC-DRL algorithms based on deep deterministic policy gradient (DDPG) were developed to optimize UAV trajectory and resource allocation. ...
... Secondly, unlike the conventional DRL or AC-DRL methods, the proposed solution is suited to tackle constrained combinatorial optimization. Thirdly, compared to our previous work [21], we augment the algorithms by developing new theoretical results and tailored approaches to address two challenging issues in guaranteeing feasibility and controlling exponentially-increased action space. The major contributions are summarized as follows: ...
In unmanned aerial vehicle (UAV) applications, the UAV's limited energy supply and storage have triggered the development of intelligent energy-conserving scheduling solutions. In this paper, we investigate energy minimization for UAV-aided communication networks by jointly optimizing data-transmission scheduling and UAV hovering time. The formulated problem is combinatorial and non-convex with bilinear constraints. To tackle the problem, firstly, we provide an optimal relax-and-approximate solution and develop a near-optimal algorithm. Both the proposed solutions are served as offline performance benchmarks but might not be suitable for online operation. To this end, we develop a solution from a deep reinforcement learning (DRL) aspect. The conventional RL/DRL, e.g., deep Q-learning, however, is limited in dealing with two main issues in constrained combinatorial optimization, i.e., exponentially increasing action space and infeasible actions. The novelty of solution development lies in handling these two issues. To address the former, we propose an actor-critic-based deep stochastic online scheduling (AC-DSOS) algorithm and develop a set of approaches to confine the action space. For the latter, we design a tailored reward function to guarantee the solution feasibility. Numerical results show that, by consuming equal magnitude of time, AC-DSOS is able to provide feasible solutions and saves 29.94% energy compared with a conventional deep actor-critic method. Compared to the developed near-optimal algorithm, AC-DSOS consumes around 10% higher energy but reduces the computational time from minute-level to millisecond-level.
... Optimization-based solutions, e.g., successive convex approximation [5] or Lagrangian dual method [6], might not be able to make time-efficient decisions. First, the SDMAbased transmission mode enables the UAV to serve more than one GU simultaneously, resulting in exponential growth of decision variables as well as the complexity [1]. Moreover, diversified energy models in UAV systems may lead to non-convexity in problem formulation, which makes the problem difficult to be solved optimally. ...
In unmanned aerial vehicle (UAV)-assisted networks, UAV acts as an aerial base station which acquires the requested data via backhaul link and then serves ground users (GUs) through an access network. In this paper, we investigate an energy minimization problem with a limited power supply for both backhaul and access links. The difficulties for solving such a non-convex and combinatorial problem lie at the high computational complexity/time. In solution development, we consider the approaches from both actor-critic deep reinforcement learning (AC-DRL) and optimization perspectives. First, two offline non-learning algorithms, i.e., an optimal and a heuristic algorithms, based on piecewise linear approximation and relaxation are developed as benchmarks. Second, toward real-time decision-making, we improve the conventional AC-DRL and propose two learning schemes: AC-based user group scheduling and backhaul power allocation (ACGP), and joint AC-based user group scheduling and optimization-based backhaul power allocation (ACGOP). Numerical results show that the computation time of both ACGP and ACGOP is reduced tenfold to hundredfold compared to the offline approaches, and ACGOP is better than ACGP in energy savings. The results also verify the superiority of proposed learning solutions in terms of guaranteeing the feasibility and minimizing the system energy compared to the conventional AC-DRL.
... Optimization-based solutions, e.g., successive convex approximation [5] or Lagrangian dual method [6], might not be able to make time-efficient decisions. Firstly, the SDMA-based transmission mode enables the UAV to serve more than one GU simultaneously, resulting in an exponential growth of decision variables as well as the complexity [1]. Moreover, diversified energy models in UAV systems may lead to non-convexity in problem formulation, which makes the problem difficult to be solved optimally. ...
... The total demands is denoted by D = K k=1 q k . In each transmission task, all the GUs' demands need to be served within the time limitation T max (seconds), including the time used for acquiring data from ABS and delivering data to GUs [1] . As shown in Fig. 2, the system spectrum is reused in a TDMA fashion so that the time domain of a transmission task is divided into sequence of timeslots I = {1, ..., i, ..., I}, where I is the maximum number of timeslots, given by Tmax Φ , and Φ (seconds) refers to the duration of each timeslot. ...
... where U ∈ C Lt×Lt and V ∈ C Lr×Lr are unitary matrices, and Λ ∈ C Lt×Lr is a diagonal matrix whose elements are non-negative real numbers. The diagonal [1] The time and energy consumed on sending requests from GUs to UAV are not considered in this paper, since they are negligible compared to those on content delivery. elements λ 1 , ..., λ L in Λ are the ordered singular values (from large to small) for G. ...
In unmanned aerial vehicle ( UAV )-assisted networks, UAV acts as an aerial base station to serve ground users ( GUs ) through an access network, and meanwhile, the requested data traffic is acquired through the backhaul link. In this paper, we investigate an energy minimization problem with limited power supply for both backhaul and access links. The difficulties for solving such a non-convex and combinatorial problem lie at the high computational complexity/time. In solution development, we consider the approaches from both actor-critic deep reinforcement learning (AC- DRL ) and optimization perspectives. Firstly, two offline non-learning algorithms, i.e., an optimal and a heuristic algorithms, based on piecewise linear approximation and relaxation are developed as benchmarks. Secondly, towards real-time decision making, we improve the conventional AC- DRL and propose two learning schemes: AC-based user group scheduling and backhaul power allocation ( ACGP ), and joint AC-based user group scheduling and optimization-based backhaul power allocation ( ACGOP ). Numerical results show that the computation time of both ACGP and ACGOP are reduced tenfold to hundredfold compared to the offline approaches, where ACGOP is better than ACGP in energy savings. The results also verify the superiority of proposed learning solutions in terms of guaranteeing the feasibility and minimizing the system energy compared to the conventional AC- DRL .
... Part of this paper has been presented at IEEE EuCNC, June 2020 [18]. energy consumption comes from two aspects, propulsion energy for flying and hovering, and communication energy for data transmission. ...
... The authors in [17] developed an AC-DRL algorithm for a combinatorial optimization problem in a UAVaided system, but when the size of the action space grows exponentially, the convergence of the algorithm deteriorates. In [18], the authors applied a conventional AC-DRL approach to address an energy minimization problem in UAV networks, where the performance is limited by feasibility guarantee and rapidly-increasing action space. ...
In unmanned aerial vehicle (UAV) applications, the UAV's limited energy supply and storage have triggered the development of intelligent energy-conserving scheduling solutions. In this paper, we investigate energy minimization for UAV-aided communication networks by jointly optimizing data-transmission scheduling and UAV hovering time. The formulated problem is combinatorial and non-convex with bilinear constraints. To tackle the problem, firstly, we provide an optimal relax-and-approximate solution and develop a near-optimal algorithm. Both the proposed solutions are served as offline performance benchmarks but might not be suitable for online operation. To this end, we develop a solution from a deep reinforcement learning (DRL) aspect. The conventional RL/DRL, e.g., deep Q-learning, however, is limited in dealing with two main issues in constrained combinatorial optimization, i.e., exponentially increasing action space and infeasible actions. The novelty of solution development lies in handling these two issues. To address the former, we propose an actor-critic-based deep stochastic online scheduling (AC-DSOS) algorithm and develop a set of approaches to confine the action space. For the latter, we design a tailored reward function to guarantee the solution feasibility. Numerical results show that, by consuming equal magnitude of time, AC-DSOS is able to provide feasible solutions and saves 29.94% energy compared with a conventional deep actor-critic method. Compared to the developed near-optimal algorithm, AC-DSOS consumes around 10% higher energy but reduces the computational time from minute-level to millisecond-level.
Energy consumption is a critical constraint for Unmanned Aerial Vehicles (UAVs) delivery operations to achieve their full potential of providing fast delivery, reducing cost, and cutting emissions. In this article, we propose a synchronized delivery mechanism that employs trucks and UAVs to construct an energy efficient essential service delivery model using Multi-Swarm UAV-Truck (MSUT) framework in a sixth generation (6G) assisted environment. Firstly, we introduce an efficient Brain Storm Optimization (BSO) algorithm that determines the optimal placement location for the trucks and the number of UAV launch sites, given the delivery requirements for optimal delivery of essentials to the target destination. Further, a Multi-Agent Reinforcement Learning (MARL) model, namely Multi-Agent Advantage Actor Critic (MAAC), is employed on UAVs in a swarm for route optimization and efficient energy consumption while en route to the destination. We further investigate the reduced overall delivery time and energy metrics for the proposed UAV-truck network by comparing it with existing Deep Reinforcement Learning (DRL) delivery models.
