Article

Stochastic Network Optimization with Application to Communication and Queueing Systems

Authors:
To read the full-text of this research, you can request a copy directly from the author.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... • To ensure the practicality and scalability, we identify inherent properties of the JFCS problem and propose an intelligent resource management algorithm to solve it effectively by leveraging the stochastic optimization framework [14]. In particular, by exploiting the historical system information accumulated from the previous time-slots, an RL process is developed to build the smoothed best response while maximizing the long-term utility for each data-flow under arbitrary changes in traffic demands. ...
... , which is the quadratic Lyapunov function [14], [35]. For given (q[t s ],q[t s ]), the Lyapunov drift from time-slot t s to t s+1 is given as ...
... To guarantee joint network stability and penalty minimization (i.e., (5b) and (7c) hold true), we adopt the drift-pluspenalty procedure [14] to minimize the drift of a quadratic Lyapunov function and rewrite (7) as ...
Preprint
Full-text available
To enable an intelligent, programmable and multi-vendor radio access network (RAN) for 6G networks, considerable efforts have been made in standardization and development of open RAN (ORAN). So far, however, the applicability of ORAN in controlling and optimizing RAN functions has not been widely investigated. In this paper, we jointly optimize the flow-split distribution, congestion control and scheduling (JFCS) to enable an intelligent traffic steering application in ORAN. Combining tools from network utility maximization and stochastic optimization, we introduce a multi-layer optimization framework that provides fast convergence, long-term utility-optimality and significant delay reduction compared to the state-of-the-art and baseline RAN approaches. Our main contributions are three-fold: i) we propose the novel JFCS framework to efficiently and adaptively direct traffic to appropriate radio units; ii) we develop low-complexity algorithms based on the reinforcement learning, inner approximation and bisection search methods to effectively solve the JFCS problem in different time scales; and iii) the rigorous theoretical performance results are analyzed to show that there exists a scaling factor to improve the tradeoff between delay and utility-optimization. Collectively, the insights in this work will open the door towards fully automated networks with enhanced control and flexibility. Numerical results are provided to demonstrate the effectiveness of the proposed algorithms in terms of the convergence rate, long-term utility-optimality and delay reduction.
... over time slots. Under any control algorithms and V ≥ 0, and all possible G(t), the drift-minus-reward term has the following upper bound [26,39]: ...
... When the BA and RC decisions of the considered networks are given as c * (t) and µ * (t), respectively, in the networks, the admission and transmission rates of the link between the i-th FD SBS and k-th DL UE are denoted by A * D i,k (t) and R * D i,k (t), respectively; admission rate of the m-th UL UE and transmission rate of link between the m-th UL UE and n-th FD SBS are expressed by A * Um (t) and R * Um,n (t), respectively; the power consumption of n-th FD SBS is represented by P * Un . Proof : The proof of this lemma is intuitive and is discussed from pages 58 to 62 in [39]. ...
... We will prove Lemma 2 using the following Lemma [39]: ...
Article
Full-text available
This study investigates the resource management problem for millimeter-wave-based switched beam (SWB) full-duplex small cell networks with the consideration of user equipment’s (UE’s) quality of experience (QoE) requirement and time-varying wireless channels. An optimization problem is formulated to maximize the long-term QoE by implementing beam assignment (BA) and rate control (RC) under short-term beam and long-term energy efficiency constraints. By leveraging the Lyapunov optimization technique, the original problem is converted into a series of BA and RC problems in each time slot. To solve the converted problem with affordable complexity, novel closed-form solutions for BA and RC are first derived by considering beam constraints in SWB systems. A decomposition-based BA and RC (DBR) algorithm with only polynomial computational complexity is then proposed based on the derived closed-form solutions. The simulation results demonstrate that the proposed DBR method can effectively‘effectively’ appears to be a more suitable word in this case. balance the performance and complexity because the DBR scheme outperforms the benchmark scheme and achieves nearly optimal performance in terms of system delay and QoE.
... The major challenge in solving problem P 0 is to handle the long-term constraints. We leverage the Lyapunov technique [20], [21] to address this challenge. The core idea is to construct accuracy deficit queues to characterize the satisfaction status of the long-term accuracy constraints, thereby guiding the learning agent to meet the long-term accuracy constraints. ...
... Secondly, the Lyapunov function should be consistently pushed to a low value in order to guarantee the long-term accuracy constraints. Hence, we introduce a one-shot Lyapunov drift to capture the variation of the Lyapunov function across two subsequent time slots [20]. Given Z t m , the one-shot Lyapunov drift is defined as ∆ (Z t m ) = L Z t+1 m − L (Z t m ), which is upper bounded by ...
... • Delay myopic: Each industrial IoT device dynamically makes sampling rate selection and task offloading decisions by maximizing the one-step reward in (20) according to the network state. • Static configuration: Each industrial IoT device takes a static configuration on the sampling rate selection and task offloading decisions, which can guarantee services' accuracy requirements. ...
Preprint
Full-text available
Collaboration among industrial Internet of Things (IoT) devices and edge networks is essential to support computation-intensive deep neural network (DNN) inference services which require low delay and high accuracy. Sampling rate adaption which dynamically configures the sampling rates of industrial IoT devices according to network conditions, is the key in minimizing the service delay. In this paper, we investigate the collaborative DNN inference problem in industrial IoT networks. To capture the channel variation and task arrival randomness, we formulate the problem as a constrained Markov decision process (CMDP). Specifically, sampling rate adaption, inference task offloading and edge computing resource allocation are jointly considered to minimize the average service delay while guaranteeing the long-term accuracy requirements of different inference services. Since CMDP cannot be directly solved by general reinforcement learning (RL) algorithms due to the intractable long-term constraints, we first transform the CMDP into an MDP by leveraging the Lyapunov optimization technique. Then, a deep RL-based algorithm is proposed to solve the MDP. To expedite the training process, an optimization subroutine is embedded in the proposed algorithm to directly obtain the optimal edge computing resource allocation. Extensive simulation results are provided to demonstrate that the proposed RL-based algorithm can significantly reduce the average service delay while preserving long-term inference accuracy with a high probability.
... • For the multi-source age optimal scheduling problem, we also study the feasibility region of the averageage-optimal scheduler under age-violation-rate tolerance constraints to contrast its results with those of related earlier works that are developed for the single-channel multi-user scenario (see Section IV-C and Section VI). • Moreover, we develop (in Section V) an online scheduler using Lyapunov-drift-minimization methods (e.g., [28]) that does not require the knowledge of channel statistics, and compare its performance to the optimal and earlier designs to reveal how much the knowledge of channel statistics affects the feasibility region (see Section VI). ...
... Until this point, we have assumed that the channel success probabilities are known when solving the optimization problems. In this section, we use a Lyapunov-drift-plus-penalty approach(see [28]) to solve the multi-user online age related optimization problem in the scenario when only the current channel states are known, but the channel statistics are unknown. ...
... ] will be less than its output rate b i [28], so that the corresponding constraint can be satisfied. Define the state of both virtual queues and age at time t as ...
Preprint
We study the optimal scheduling problem where n source nodes attempt to transmit updates over L shared wireless on/off fading channels to optimize their age performance under energy and age-violation tolerance constraints. Specifically, we provide a generic formulation of age-optimization in the form of a constrained Markov Decision Processes (CMDP), and obtain the optimal scheduler as the solution of an associated Linear Programming problem. We investigate the characteristics of the optimal single-user multi-channel scheduler for the important special cases of average-age and violation-rate minimization. This leads to several key insights on the nature of the optimal allocation of the limited energy, where a usual threshold-based policy does not apply and will be useful in guiding scheduler designers. We then investigate the stability region of the optimal scheduler for the multi-user case. We also develop an online scheduler using Lyapunov-drift-minimization methods that do not require the knowledge of channel statistics. Our numerical studies compare the stability region of our online scheduler to the optimal scheduler to reveal that it performs closely with unknown channel statistics.
... 1) Methodology and Challenges: Our approach is based on the Lyapunov drift-plus-penalty framework [47], but with some notable differences. First, while infinite T is the primary focus in [47], we allow finite T in this paper, both in the problem formulation (see P1 and P2 above) and in our analysis later in Section IV-D. ...
... 1) Methodology and Challenges: Our approach is based on the Lyapunov drift-plus-penalty framework [47], but with some notable differences. First, while infinite T is the primary focus in [47], we allow finite T in this paper, both in the problem formulation (see P1 and P2 above) and in our analysis later in Section IV-D. This consideration is because, in practice, we usually train the model only for a finite number of iterations. ...
... This consideration is because, in practice, we usually train the model only for a finite number of iterations. Second, while P2.1 can depend on an underlying system state (i.e., ω(t) defined in [47]) that is independent across time t, we emphasize that the underlying states of P2.2 and P2.3 are both time-dependent and also dependent on previous decisions made by the control algorithm. To see this, note that the quantity b n t in P2.2 depends on the stochastic gradient computed on the model parameter x t , and the value of x t is related to the decisions on q n τ , v n τ , u τ made in previous iterations τ < t. ...
Preprint
Federated learning (FL) enables distributed model training from local data collected by users. In distributed systems with constrained resources and potentially high dynamics, e.g., mobile edge networks, the efficiency of FL is an important problem. Existing works have separately considered different configurations to make FL more efficient, such as infrequent transmission of model updates, client subsampling, and compression of update vectors. However, an important open problem is how to jointly apply and tune these control knobs in a single FL algorithm, to achieve the best performance by allowing a high degree of freedom in control decisions. In this paper, we address this problem and propose FlexFL - an FL algorithm with multiple options that can be adjusted flexibly. Our FlexFL algorithm allows both arbitrary rates of local computation at clients and arbitrary amounts of communication between clients and the server, making both the computation and communication resource consumption adjustable. We prove a convergence upper bound of this algorithm. Based on this result, we further propose a stochastic optimization formulation and algorithm to determine the control decisions that (approximately) minimize the convergence bound, while conforming to constraints related to resource consumption. The advantage of our approach is also verified using experiments.
... Since the optimal solution of the original problem probably has extremely high computational complexity, we decompose the original problem into two subproblems with different time scales where the solution of the beam activation runs on a slower time scale than that of the user scheduling and transmit power allocation problem. Besides, we apply Lyapunov DMU (Drift-Minus-Utility framework) [19] to the subproblems so that they are solved every time slot without loss of optimality. However, the transformed subproblems are still known as NP-hard; hence we introduce the intuitive idea, namely inner and outer reference users to optimize the transmit power with low computational complexity. ...
... We should note that there are three obstacles to solve the problem (P) as follows: (i) (P) is long-term average utility maximization problem, yet our practical problem is to find beam activation, user scheduling and power allocation every time slot, i.e., online-fashioned solution, (ii) finding joint beam activation, user scheduling and power allocation is NPhard problem, (iii) future information on the time-varying wireless channels cannot be known in advance. Hence, we exploit Lyapunov optimization framework [19] to derive practical problem since this well-known dynamic optimization framework does not require the future wireless channel information and the time-average constraints, i.e., average power constraints can be modeled as the operation of virtual queue; hence we can easily interpret the algorithm operation. ...
... where γ k,min and γ k,max are any constraints and the ancillary variable γ(t) follows r(t) smoothly over time. Leveraging Jensen's inequality [19], we can reformulate the problem (P1) to (P2) as follows: (4), (5), (6), where γ = {γ(t) : t ∈ T }. To capture constraints (6) and (11) in per-slot problem, we add two virtual queues Z c (t) for each cluster and W k (t) for each user. ...
Article
Full-text available
Beamforming, user scheduling and transmit power on existing interference management schemes in multi-cell mmWave networks have been independently controlled due to the high computational complexity of the problem. In this paper, we formulate a long-term utility maximization problem where beam activation, user scheduling and transmit power are incorporated in a single framework. To develop a low-complex algorithm, we first leverage the Lyapunov optimization framework to transform the original long-term problem into a series of slot-by-slot problems. Since the computational complexity to optimally solve the slot-by-slot problem is even significantly high like existing schemes, we decompose the problem into two different time scales: ( i ) a subproblem to find beam activation probability with a long time-scale and ( ii ) a subproblem to find user scheduling and power allocation with a short time-scale. Moreover, we introduce two additional gimmicks to more simplify the problem: ( i ) sequentially making decisions of beam activation, user scheduling, and power allocation, and ( ii ) considering a critical user for power allocation. Finally, via extensive simulations, we find that the proposed CRIM algorithm outperforms existing algorithms by up to 47.4% in terms of utility.
... where t n waiting indicates the waiting time at n-th in road queue. For queue stability [31], according to Lyapunov theorem, we should ensure the following equation holds, ...
... The trajectory optimization problem considering collision avoidance can be modeled as an unconstrained optimization problem as (31), which uses some tricks to achieve approximate hard constraints. ...
Preprint
Full-text available
Since the traffic administration at road intersections determines the capacity bottleneck of modern transportation systems, intelligent cooperative coordination for connected autonomous vehicles (CAVs) has shown to be an effective solution. In this paper, we try to formulate a Bi-Level CAV intersection coordination framework, where coordinators from High and Low levels are tightly coupled. In the High-Level coordinator where vehicles from multiple roads are involved, we take various metrics including throughput, safety, fairness and comfort into consideration. Motivated by the time consuming space-time resource allocation framework in [1], we try to give a low complexity solution by transforming the complicated original problem into a sequential linear programming one. Based on the "feasible tunnels" (FT) generated from the High-Level coordinator, we then propose a rapid gradient-based trajectory optimization strategy in the Low-Level planner, to effectively avoid collisions beyond High-level considerations, such as the pedestrian or bicycles. Simulation results and laboratory experiments show that our proposed method outperforms existing strategies. Moreover, the most impressive advantage is that the proposed strategy can plan vehicle trajectory in milliseconds, which is promising in real-world deployments. A detailed description include the coordination framework and experiment demo could be found at the supplement materials, or online at https://youtu.be/MuhjhKfNIOg.
... How to make the camera selection decisions without future information? We leverage the Lyapunov optimization technique [26] to make camera selection decisions without knowing far future information to balance long-term 3D estimation performance and energy consumption. Specifically, E 3 Pose converts the long-term optimization problem (5) into a sequence of per-slot optimization problems that can be easily solved using the predicted occlusion information and the current energy state. ...
... Thus indicates the deviation of the current power consumption from the long-term energy constraint. Following the drift-pluspenalty framework in Lyapunov optimization [26], in time slot , the camera scheduler makes the selection decision for the scene in time slot + by solving the following optimization problem ...
Preprint
Full-text available
Multi-human 3D pose estimation plays a key role in establishing a seamless connection between the real world and the virtual world. Recent efforts adopted a two-stage framework that first builds 2D pose estimations in multiple camera views from different perspectives and then synthesizes them into 3D poses. However, the focus has largely been on developing new computer vision algorithms on the offline video datasets without much consideration on the energy constraints in real-world systems with flexibly-deployed and battery-powered cameras. In this paper, we propose an energy-efficient edge-assisted multiple-camera system, dubbed E$^3$Pose, for real-time multi-human 3D pose estimation, based on the key idea of adaptive camera selection. Instead of always employing all available cameras to perform 2D pose estimations as in the existing works, E$^3$Pose selects only a subset of cameras depending on their camera view qualities in terms of occlusion and energy states in an adaptive manner, thereby reducing the energy consumption (which translates to extended battery lifetime) and improving the estimation accuracy. To achieve this goal, E$^3$Pose incorporates an attention-based LSTM to predict the occlusion information of each camera view and guide camera selection before cameras are selected to process the images of a scene, and runs a camera selection algorithm based on the Lyapunov optimization framework to make long-term adaptive selection decisions. We build a prototype of E$^3$Pose on a 5-camera testbed, demonstrate its feasibility and evaluate its performance. Our results show that a significant energy saving (up to 31.21%) can be achieved while maintaining a high 3D pose estimation accuracy comparable to state-of-the-art methods.
... From (34) in the proof of Lemma 15, we have ...
Preprint
This paper studies online nonstochastic control problems with adversarial and static constraints. We propose online nonstochastic control algorithms that achieve both sublinear regret and sublinear adversarial constraint violation while keeping static constraint violation minimal against the optimal constrained linear control policy in hindsight. To establish the results, we introduce an online convex optimization with memory framework under adversarial and static constraints, which serves as a subroutine for the constrained online nonstochastic control algorithms. This subroutine also achieves the state-of-the-art regret and constraint violation bounds for constrained online convex optimization problems, which is of independent interest. Our experiments demonstrate the proposed control algorithms are adaptive to adversarial constraints and achieve smaller cumulative costs and violations. Moreover, our algorithms are less conservative and achieve significantly smaller cumulative costs than the state-of-the-art algorithm.
... Based on problems P 2 and P 3 , we aim to design a scheduling mechanism which can decide the quantity of selected devices to reduce the impact of channel noise in each communication round effectively, and ensure that the quality of selected devices is as high as possible to take the full advantage of their training performances as well. Such a scheduling mechanism can be considered as a multi-objective joint optimization problem to obtain the linear tradeoff between the maximization of device quality and minimization of the objective function in FEEL systems, which can be solved by the drift-pluspenalty algorithm in Lyapunov optimization [45]. Therefore, we formulate the optimization problem of the device scheduling mechanism as P 4 : min an,t αU t − N n=1 a n,t I n,t , ...
Preprint
Full-text available
To satisfy the expected plethora of computation-heavy applications, federated edge learning (FEEL) is a new paradigm featuring distributed learning to carry the capacities of low-latency and privacy-preserving. To further improve the efficiency of wireless data aggregation and model learning, over-the-air computation (AirComp) is emerging as a promising solution by using the superposition characteristics of wireless channels. However, the fading and noise of wireless channels can cause aggregate distortions in AirComp enabled federated learning. In addition, the quality of collected data and energy consumption of edge devices may also impact the accuracy and efficiency of model aggregation as well as convergence. To solve these problems, this work proposes a dynamic device scheduling mechanism, which can select qualified edge devices to transmit their local models with a proper power control policy so as to participate the model training at the server in federated learning via AirComp. In this mechanism, the data importance is measured by the gradient of local model parameter, channel condition and energy consumption of the device jointly. In particular, to fully use distributed datasets and accelerate the convergence rate of federated learning, the local updates of unselected devices are also retained and accumulated for future potential transmission , instead of being discarded directly. Furthermore, the Lyapunov drift-plus-penalty optimization problem is formulated for searching the optimal device selection strategy. Simulation results validate that the proposed scheduling mechanism can achieve higher test accuracy and faster convergence rate, and is robust against different channel conditions.
Conference Paper
Wireless network computing has been suggested as a voracious development by isolating and exchanging network infrastructure across various software systems in future wireless networks to counter the contradiction amongst the rapid development of the demand for information services besides the small bandwidth of resources. In this article, the multi-service virtualized wireless system consists of three broad-based situations and suggests a two-dimensional hierarchic resource distribution system. The resource division and allotment dilemma are composed of two-time scales with a large time frame before the equivalent lateral assets and a limited time frame for the intra-slice resource preparation. In the long run, the problem of pre-allotting inter-slice services is conceived by modelling the packets that arrive and serve each slice as a queuing device. In short, the source and energy management in each share is described as a stochastic optimization problem because of variable time and complex traffic arrivals. This problem aims to maximize overall output relative to various service quality criteria, including network stability, latency, throughput, and control restrictions. By applying the Lyapunov objective function, the stochastic optimization problem can be turned into a delay-aware optimization problematic methodology comprising a probabilistic and concave algorithm. The simulation findings show that the suggested systems will minimize uncertainty with an output loss of up to 0.017 percent relative to the optimum solution, which also allows for a balance in terms of the regulation coagulation Factor quality.
Chapter
Mobile edge computing (MEC) is a technology that extends network capabilities from the core network to the edge network, which effectively solves the problems of devices in terms of computing capacity, service energy consumption, and service delay. In MEC networks, conventional models cannot meet the actual needs when computing-intensive services arrive dynamically. In addition, the large energy consumption of computing-intensive services makes User Equipment (UE) energy consumption a vital issue. Meanwhile, UE energy consumption is critical to the system economic benefits of the entire MEC network. Aiming at the above problems, a stochastic optimization model that can dynamically combine queue length, computing offloading, resource allocation, and energy consumption control is designed, and a Multi-Dimensional Resource Dynamic computation Offloading algorithm (MDRDO) based on Lyapunov optimization theory is proposed. The algorithm used Lyapunov optimization theory to solve the closed solution of computation offloading, bandwidth resource and computing resource allocation problem, and solved the above stochastic optimization model at the same time. Under the premise of ensuring the stability of the service queue, the algorithm improved the economic benefits of the overall MEC network system. The simulation results indicate that, compared with other algorithms, the proposed MDRDO could save the energy consumption of UE, and at the same time significantly improved the overall economic benefit of the system.KeywordsMobile edge computingLyapunov optimizationComputing offloadingSystem economic benefit
Article
This paper revisits the problem of sampling and transmitting status updates through a channel with random delay under a sampling frequency constraint [2]. We use the Age of Information (AoI) to characterize the status information freshness at the receiver. The goal is to design a sampling policy that can minimize the average AoI when the statistics of delay is unknown. We reformulate the problem as the optimization of a renewal-reward process, and propose an online sampling strategy based on the Robbins-Monro algorithm. We prove that the proposed algorithm satisfies the sampling frequency constraint. Moreover, when the transmission delay is bounded and its distribution is absolutely continuous, the average AoI obtained by the proposed algorithm converges to the minimum AoI when the number of samples K goes to infinity with probability 1. We show that the optimality gap decays with rate O (ln K / K ), and the proposed algorithm is minimax rate optimal. Simulation results validate the performance of our proposed algorithm.
Article
Full-text available
Goal-oriented communications represent an emerging paradigm for efficient and reliable learning at the wireless edge, where only the information relevant for the specific learning task is transmitted to perform inference and/or training. The aim of this paper is to introduce a novel system design and algorithmic framework to enable goal-oriented communications. Specifically, inspired by the information bottleneck principle and targeting an image classification task, we dynamically change the size of the data to be transmitted by exploiting banks of convolutional encoders at the device in order to extract meaningful and parsimonious data features in a totally adaptive and goal-oriented fashion. Exploiting knowledge of the system conditions, such as the channel state and the computation load, such features are dynamically transmitted to an edge server that takes the final decision, based on a proper convolutional classifier. Hinging on Lyapunov stochastic optimization, we devise a novel algorithmic framework that dynamically and jointly optimizes communication, computation, and the convolutional encoder classifier, in order to strike a desired trade-off between energy, latency, and accuracy of the edge learning task. Several simulation results illustrate the effectiveness of the proposed strategy for edge learning with goal-oriented communications.
Article
Full-text available
In this paper, we propose a novel algorithm for energy-efficient low-latency dynamic mobile edge computing (MEC), in the context of beyond 5G networks endowed with reconfigurable intelligent surfaces (RISs). We consider a scenario where new computing requests are continuously generated by a set of devices and are handled through a dynamic queueing system. Building on stochastic optimization tools, we devise a dynamic learning algorithm that jointly optimizes the allocation of radio resources (i.e., power, transmission rates, sleep mode and duty cycle), computation resources (i.e., CPU cycles), and RIS reflectivity parameters (i.e., phase shifts), while guaranteeing a target performance in terms of average end-to-end delay. The proposed strategy enables dynamic control of the system, performing a low-complexity optimization on a per-slot basis while dealing with time-varying radio channels and task arrivals, whose statistics are unknown. The presence and optimization of RISs helps boosting the performance of dynamic MEC, thanks to the capability to shape and adapt the wireless propagation environment. Numerical results assess the performance in terms of service delay, learning, and adaptation capabilities of the proposed strategy for RIS-empowered MEC.
Article
Full-text available
Mobile edge computing is a novel technique that can reduce computational burden of local terminal devices by tasks offloading, which emerges as a promising paradigm to provide computing capabilities near mobile users. In this paper, the task offloading decision to solve the queuing problem of tasks to be processed in the local terminals was being considered, with an application of Lyapunov theory to ensure the queue stability. Then, a trade-off model was formulated to minimize the delay and energy consumption to achieve a minimum execution cost. Moreover, an improved Dynamic Niche-based Self-organizing Learning Algorithm was presented to accelerate the speed of the search process to gain an optimal task offloading scheme. The simulation results provided supporting evidence that the proposed optimal scheme IDO could achieve a lower average energy consumption than LFO and a lower average execution delay than EFO under the scenarios of the waiting queue length varies within [10, 50], the number of users varies within [4, 20], and the number of tasks varies within [20, 100].
Article
Full-text available
Carbon footprint monitoring provides significant basis for computing facilities to improve the computing efficiency and reduce the energy cost, which can enable low/zero carbon computing and facilitate the construction of sustainable cites. Massive sensing devices access to 5G base station to transmit collected carbon emission data, which requires intelligent access management. Fast uplink grant possesses the advantages of reducing signalling overhead and access conflicts while facing the problems of incomplete information and difficulty in guaranteeing priority constraint. This paper examines, the maximum access queuing delay minimization problem is formulated under the long‐term service priority constraint and the short‐term access management constraint. First, Lyapunov optimization is leveraged to decouple the long‐term service priority constraint and short‐term access management optimization, and decompose the long‐term stochastic optimization problem into a series of short‐term deterministic problems. Then, a priority‐aware deep Q‐network (DQN)‐based fast uplink grant access management (PDAC) algorithm is proposed to achieve intelligent access management with differentiated service priority requirements. PDAC utilizes DQN to handle non‐convex high‐dimensional optimization problem with service priority constraint to achieve intelligent access management and priority awareness. Simulation results demonstrate that PDAC outperforms the existing algorithms in access queuing delay, buffer queue backlog, and priority deficit fluctuation.
Article
Full-text available
Networked microgrids (MGs) have a great potential to improve the efficiency, reliability, resilience, security, and sustainability of power supply services. Peer-to-peer (P2P) energy trading built on a smart information system in networked MGs is an emerging economic approach to facilitate energy sharing among networked MGs to achieve mutual cost-effective operation and improve the reliability and stability of energy supply service. Such a distributed and competitive energy trading market urges the need for an efficient energy trading strategy that incentivizes the self-interested MGs with various energy production and consumption profiles to participate in energy trading. In this paper, we propose a distributed real-time P2P energy trading strategy that integrates energy trading into energy management and enables the MGs with renewable energy sources (RESs) and energy storage systems (ESSs) to manage their storage scheduling, energy supply, and energy trading in a dynamic manner, jointly considering the randomness of renewable energy generation and load demand, operational constraints of ESSs and transmission losses associated with energy exchange. The proposed energy control and bidding algorithm allows each MG to dynamically and independently determine its energy control actions and price-quantity bids/offers, while the proposed trading pair matching algorithm matches the MGs on a many-to-many basis with respect to their individual payoffs, which couple price-quantity bids/offers of the MGs with distance-dependent energy transmission losses associated with energy exchange. Numerical simulation results demonstrate that the proposed distributed energy trading system yields significant improvements in terms of energy cost savings and renewable energy utilization efficiency, while reducing energy transmission losses within the system.
Article
The proliferation of distributed renewable energy triggers the peer-to-peer (P2P) energy market formations. To make profits, prosumers equipped with photovoltaic (PV) panels and even the energy storage system (ESS) can actively participate in the real-time P2P energy market and trade energy. However, in real situations, system states such as energy demands and renewable energy power generation are highly uncertain, making it difficult for prosumers to make optimal real-time decisions. Moreover, severe problems with the physical network can arise from the real-time P2P energy trading, such as bus voltage violations and line overload. To handle these problems, this work first formulates the real-time P2P energy trading problem as a spatio-temporally constrained stochastic optimization problem by considering ESS and the spatial physical network constraints. To deal with the uncertainties online, a modified Lyapunov optimization method is innovatively proposed to approximately reformulate the stochastic optimization problem into an online one by relaxing the time-coupling constraints. Compared with the state-of-the-art online methods, the proposed one renders more flexibility and better performance for the real-time P2P energy market operation. Additionally, to protect the prosumers’ privacy, an online distributed algorithm based on the consensus alternating direction method of multipliers (ADMM) is developed to solve the reformulated online problem by decoupling the spatial constraints. The theoretical near-optimal performance guarantee of the proposed online distributed algorithm is derived, and its performance can be further improved by minimizing the performance gap. Simulation results demonstrate that the proposed online distributed algorithm can guarantee the fast, stable, and safe long-term operation of the real-time P2P energy market. Allied with a performance gap minimization step, the proposed online distributed algorithm can obtain approximately a 53.10% cost reduction compared to the greedy algorithm for the intraday market operation.
ResearchGate has not been able to resolve any references for this publication.