No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
6G Deterministic Network Technology Based on Hierarchical Reinforcement Learning Framework
ResearchGate has not been able to resolve any citations for this publication.
In mixed-criticality Ethernet-based time-triggered networks, like TTEthernet, time-triggered communication (TT) coexists with rate-constrained (RC) and best-effort (BE) traffic. A global communication scheme, i.e., a schedule, establishes contention-free transmission times for TT flows ensuring guaranteed low latency and minimal jitter. Current approaches use Satisfiability Modulo Theories (SMT) to formulate the scheduling constraints and solve the resulting problem. However, these approaches do not take into consideration the impact of the TT schedule on RC traffic. Hence, the resulting TT schedule may cause the worst-case latency requirements of RC traffic not to be fulfilled anymore. In this paper, we present a novel method for including an RC analysis in state-of-the-art SMT-based schedule synthesis algorithms via a feedback loop in order to maintain the optimality properties of the SMT-based approaches while also being able to improve the RC traffic delays. Our method is designed in such a way that it can be readily integrated into existing SMT-or MiP-based solutions. We evaluate our approach using variants derived from a realistic use-case and present methods to further improve the efficiency of our feedback-based approach.
The growing number of Internet of Things (IoT) devices brings enormous time-sensitive applications, which require real-time transmission to effectuate communication services. The ultra-reliable and low latency communication (URLLC) scenario in the fifth generation (5G) has played a critical role in supporting services with delay-sensitive properties. Time-sensitive networking (TSN) has been widely considered as a promising paradigm for enabling the deterministic transmission guarantees for 5G. However, TSN is a hybrid traffic system with time-sensitive traffic and best-effort traffic, which require effective routing and scheduling to provide a deterministic and bounded delay. While joint optimization of time-sensitive and non-time-sensitive traffic greatly increases the solution space and brings a significant challenge to obtain solutions. Therefore, this paper proposes a graph convolutional network-based deep reinforcement learning (GCN-based DRL) solution for the joint optimization problem in practical communication scenarios. The GCN is integrated into DRL to obtain the network’s spatial dependence and elevate the generalization performance of the proposed method. Specifically, the GCN adopts the first order Chebyshev polynomial to approximate the graph convolution kernel, which reduces the complexity of the algorithm and improves the feasibility for the joint optimization task. Furthermore, priority experience replay is employed to accelerate the convergence speed of the model training process. Numerical simulations demonstrate that the proposed GCN-based DRL algorithm has good convergence, and outperforms the benchmark methods in terms of the average end-to-end delay.
Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications, where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on the existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this article, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art GNNs into four categories, namely, recurrent GNNs, convolutional GNNs, graph autoencoders, and spatial-temporal GNNs. We further discuss the applications of GNNs across various domains and summarize the open-source codes, benchmark data sets, and model evaluation of GNNs. Finally, we propose potential research directions in this rapidly growing field.
Dealing with sparse rewards is one of the biggest challenges in Reinforcement Learning (RL). We present a novel technique called Hindsight Experience Replay which allows sample-efficient learning from rewards which are sparse and binary and therefore avoid the need for complicated reward engineering. It can be combined with an arbitrary off-policy RL algorithm and may be seen as a form of implicit curriculum. We demonstrate our approach on the task of manipulating objects with a robotic arm. In particular, we run experiments on three different tasks: pushing, sliding, and pick-and-place, in each case using only binary rewards indicating whether or not the task is completed. Our ablation studies show that Hindsight Experience Replay is a crucial ingredient which makes training possible in these challenging environments. We show that our policies trained on a physics simulation can be deployed on a physical robot and successfully complete the task.
Key technologies to enable 5G and TSN coordination for industrial automation
- W Li
- L Sun
- J Wang
- Z Ma
A Survey of Potential Technologies in 6G Network
- Y Lin
- S Chen
- Y Dan
- Y Liu
- L Tang
Learning Multi-Level Hierarchies with Hindsight
- A Levy
- G Konidaris
- R Platt
- K Saenko
State-of-the-Art Survey of Deterministic Transmission Technologies in Time-Sensitive Networking
- Z Li
- S Yang
- J Yu
- Y Deng
- H Wan
A Survey of Potential Technologies in 6G Network
- lin
Learning Multi-Level Hierarchies with Hindsight
- levy
Key technologies to enable 5G and TSN coordination for industrial automation
- li
State-of-the-Art Survey of Deterministic Transmission Technologies in Time-Sensitive Networking
- li