ArticlePublisher preview available

Joint design of routing selection and user association in multi-hop mmWave IABN: a multi-agent and double DQN framework

Authors:
Hu Da
Hu Da
Hu Da
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Fei Xiao
Fei Xiao
Fei Xiao
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Zhongyu Ma at Lanzhou institute of Technology
Zhongyu Ma
  • Lanzhou institute of Technology
Ziqiang Zhang
Ziqiang Zhang
Ziqiang Zhang
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Show all 5 authorsHide
Read publisher preview
Download citation
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

The increasing demand for high-speed data services from users has promoted the exploration of high-frequency spectrum resources, and the introduction of millimeter wave (mmWave) technology provides enormous bandwidth resources for the development of 5 G and future networks. Although the densely deployed multi-hop mmWave integrated access and backhaul network (IABN) has reduced operation costs and improved spectrum utilization through efficient design, the intensification of network interference and the complexity of resource management have also been followed. Especially, how to maintain a balance of rate between the access and backhaul parts is crucial for improving the overall network throughput. Therefore, this paper investigates the joint optimization problem of routing selection and user association in multi-hop mmWave IABN. This formulated problem is essentially a mixed integer nonlinear programming (MINLP) problem under consideration of the dynamic changes in the IABN environment. Deep reinforcement learning has become a emerging method for handling complex decision problems. To this end, we propose an improved multi-agent double deep Q-network based joint optimization (MDDQJO) scheme of routing selection and user association, which aims to maximize the end-to-end throughput of the multi-hop mmWave IABN. The MDDQJO scheme adopts a load based spectrum allocation strategy to adaptively meet the traffic requirements of different nodes, and dynamically optimizes routing selection and user association decisions through parallel processing and distributed training mechanisms of all agents to respond to real-time changes in the network environment. Finally, through experimental verification, the scheme not only improves the system spectrum utilization and user service quality, but also significantly enhances the link transmission efficiency.
Figures - available from: Telecommunication Systems
This content is subject to copyright. Terms and conditions apply.
A preview of this full-text is provided by Springer Nature.
Learn more
Content available from Telecommunication Systems
This content is subject to copyright. Terms and conditions apply.
Telecommunication Systems (2025) 88:21
https://doi.org/10.1007/s11235-024-01256-w
Joint design of routing selection and user association in multi-hop
mmWave IABN: a multi-agent and double DQN framework
Hu Da1,2 ·Fei Xiao2·Zhongyu Ma2·Ziqiang Zhang2·Qun Guo3
Accepted: 30 December 2024 / Published online: 20 January 2025
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2025
Abstract
The increasing demand for high-speed data services from users has promoted the exploration of high-frequency spectrum
resources, and the introduction of millimeter wave (mmWave) technology provides enormous bandwidth resources for the
development of 5G and future networks. Although the densely deployed multi-hop mmWave integrated access and backhaul
network (IABN) has reduced operation costs and improved spectrum utilization through efficient design, the intensification
of network interference and the complexity of resource management have also been followed. Especially, how to maintain
a balance of rate between the access and backhaul parts is crucial for improving the overall network throughput. Therefore,
this paper investigates the joint optimization problem of routing selection and user association in multi-hop mmWave IABN.
This formulated problem is essentially a mixed integer nonlinear programming (MINLP) problem under consideration of
the dynamic changes in the IABN environment. Deep reinforcement learning has become a emerging method for handling
complex decision problems. To this end, we propose an improved multi-agent double deep Q-network based joint optimization
(MDDQJO) scheme of routing selection and user association, which aims to maximize the end-to-end throughput of the multi-
hop mmWave IABN. The MDDQJO scheme adopts a load based spectrum allocation strategy to adaptively meet the traffic
requirements of different nodes, and dynamically optimizes routing selection and user association decisions through parallel
processing and distributed training mechanisms of all agents to respond to real-time changes in the network environment.
Finally, through experimental verification, the scheme not only improves the system spectrum utilization and user service
quality, but also significantly enhances the link transmission efficiency.
Keywords MmWave IABN ·User association ·Routing selection ·DRL ·Multi-agent double deep Q-network
BZhongyu Ma
mazybg@nwnu.edu.cn
Hu Da
4942th@163.com
Fei Xiao
2022222223@nwnu.edu.cnm
Ziqiang Zhang
2023222074@nwnu.edu.cn
Qun Guo
guoqun@lut.edu.cn
1Innovation Development and Evaluation Department, Gansu
Computing Center, LanZhou 730030, China
2College of Computer Science and Engineering, Northwest
Normal University, LanZhou 730070, China
3College of Electrical and Information Engineering, Lanzhou
University of Technology, LanZhou 730050, China
1 Introduction
With the rapid development of mobile communication tech-
nology, the increasing demand for high-speed data services
from users has driven the exploration and application of high-
frequency spectrum resources. mmWave technology, with
its wide frequency band resources, is becoming the key to
the development of 5G and future networks [1–3]. mmWave
technology can achieve high-speed data transmission within
a narrow spectrum range, meeting users’ needs for fast
and high-capacity communication [4]. Although the high-
frequency characteristics of mmWave cause rapid attenuation
during signal propagation and high sensitivity to obstacles in
urban environments, the use of high gain directional antenna
technology [5] can effectively concentrate radiation energy,
improve signal transmission efficiency, reduce losses, and
lower interference, thereby improving communication qual-
ity and security.
123
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
ResearchGate has not been able to resolve any citations for this publication.
Analysis of Duplexing Patterns in Multi-Hop mmWave Integrated Access and Backhaul Systems
Article
Full-text available
  • Jan 2024
Integrated Access and Backhaul (IAB) technology promises to facilitate cost-effective deployments of 5G New Radio (NR) systems operating in both sub-6 GHz and millimeter-wave (mmWave) bands. As full-duplex wireless systems are in their infancy, initial deployments of IAB networks may need to rely on half-duplex operation to coordinate transmissions between access and backhaul links. However, the use of half-duplex operation not only makes the scheduling of links in the IAB networks interdependent, but also the number of their feasible combinations grows exponentially with the network size, thereby posing challenges to the optimal design of such systems. In this paper, by accounting for mmWave radio characteristics, we propose a joint resource allocation and link scheduling framework to enhance the user equipment (UE) throughput in multi-hop in-band IAB systems. We keep the problem in the form of linear programming type for the feasibility of the practical applications. We show that the increased number of uplink and downlink transmission time interval (TTI) configurations does not result in improved UE throughput as compared to two configurations. Further, we demonstrate that in-band IAB systems tend to be backhaul-limited, and the utilization of multi-beam functionality at the IAB-donor alleviates this limitation by doubling the average UE throughput. Finally, we show that the use of proportional-fair allocations allows the average UE throughput to be improved by around 10% as compared to the max-min allocations.
View
5G Network on Wings: A Deep Reinforcement Learning Approach to the UAV-Based Integrated Access and Backhaul
Article
Full-text available
  • Jan 2024
Fast and reliable wireless communication has become a critical demand in human life. In the case of mission-critical (MC) scenarios, for instance, when natural disasters strike, providing ubiquitous connectivity becomes challenging by using traditional wireless networks. In this context, unmanned aerial vehicle (UAV) based aerial networks offer a promising alternative for fast, flexible, and reliable wireless communications. Due to unique characteristics such as mobility, flexible deployment, and rapid reconfiguration, drones can readily change location dynamically to provide on-demand communications to users on the ground in emergency scenarios. As a result, the usage of UAV base stations (UAV-BSs) has been considered an appropriate approach for providing rapid connection in MC scenarios. In this paper, we study how to control multiple UAV-BSs in both static and dynamic environments. We use a system-level simulator to model an MC scenario in which a macro-BS of a cellular network is out of service and multiple UAV-BSs are deployed using integrated access and backhaul (IAB) technology to provide coverage for users in the disaster area. With the data collected from the system-level simulation, a deep reinforcement learning algorithm is developed to jointly optimize the three-dimensional placement of these multiple UAV-BSs, which adapt their 3-D locations to the on-ground user movement. The evaluation results show that the proposed algorithm can support the autonomous navigation of the UAV-BSs to meet the MC service requirements in terms of user throughput and drop rate.
View
Delay-Aware Link Scheduling in IAB Networks with Dynamic User Demands
Article
Full-text available
  • Oct 2024
Integrated Access and Backhaul (IAB) is a costefficient network densification technology for improving the coverage and capacity of the millimeter-wave (mmWave) cellular networks. In IAB systems, user traffic is forwarded to/from the wired base station by one or more relay stations, known as IAB nodes. Due to the multi-hop relaying, these systems may be subject to large packet delays and poor performance when the load is unevenly distributed among nodes. Addressing this limitation via delay-aware access and backhaul link scheduling in IAB networks is challenging due to potentially large network scale, complex topology, half-duplex, and interference constraints. In this paper, the topical link scheduling problem is formulated as a Markov decision problem (MDP) for a single-donor IAB system with a general topology that allows for users with different delay requirements and traffic dynamics. The proposed link scheduling strategy jointly optimizes (i) user traffic routing and (ii) multiplexing of access and backhaul links under half-duplex constraints and non-negligible interference that may arise in dense IAB systems even with high beam directionality. To address the complexity of our formulated MDP, we consider several approximation methods, namely, Q-learning, Monte Carlo Tree Search (MCTS), and genetic algorithms (GAs). Then, we propose a customized version of the GA, which provides the preferred optimality-complexity trade-off and offers a 15% packet delay reduction as compared to the state-of-the-art backpressure algorithm.
View
Mobility-Aware Resource Allocation for mmWave IAB Networks: A Multi-Agent Reinforcement Learning Approach
Article
Full-text available
  • Aug 2024
MmWaves have been envisioned as a promising direction to provide Gbps wireless access. However, they are susceptible to high path losses and blockages, which can only be partially mitigated by directional antennas. That makes mmWave networks coverage-limited, thus requiring dense deployments. Integrated access and backhaul (IAB) architectures have emerged as a cost-effective solution for network densification. Resource allocation in mmWave IAB networks must face big challenges originated by heavy temporal dynamics, such as intermittent links caused by user mobility and blockages from moving obstacles. This makes it extremely difficult to find optimal and adaptive solutions. In this article, exploiting the distributed structure of the problem, we propose a Multi-Agent Reinforcement Learning (MARL) framework to optimize user throughput via flow routing and link scheduling in mmWave IAB networks characterized by mobile users and obstacles. The proposed approach implicitly captures the environment dynamics, coordinates the interference, and manages the buffer levels of IAB relay nodes. We design different MARL components, respectively for full-duplex and half-duplex networks. In addition, we propose an online training algorithm, which addresses the feasibility issues of practical systems, especially the communication and coordination among RL agents. Numerical results show the effectiveness of the proposed approach.
View
Data Service Maximization in Space-Air-Ground Integrated 6G Networks
Article
  • Nov 2024
Integrating terrestrial and non-terrestrial networks has emerged as a promising paradigm to fulfill the constantly growing demand for connectivity, low transmission delay, and quality of services (QoS). This integration brings together the strengths of the reliability of terrestrial networks, broad coverage and service continuity of non-terrestrial networks like low earth orbit satellites (LEOSats), etc. In this work, we study a data service maximization problem in space-air-ground integrated network (SAGIN) where the ground base stations (GBSs) and LEOSats cooperatively serve the coexisting aerial users (AUs) and ground users (GUs). Then, by considering the spectrum scarcity, interference, and QoS requirements of the users, we jointly optimize the user association, AU’s trajectory, and power allocation. To address the formulated mixed-integer non-convex problem, we decompose it into two subproblems: 1) user association problem and 2) trajectory and power allocation problem. We formulate the user association problem as a binary integer programming problem and solve it by using the Gurobi optimizer. Meanwhile, the trajectory and power allocation problem is solved by the deep deterministic policy gradient (DDPG) method to cope with the problem’s non-convexity and dynamic network environments. Then, the two subproblems are alternately solved by the proposed block coordinate descent algorithm. By comparing with the baselines in the existing literature, extensive simulations are conducted to evaluate the performance of the proposed framework.
View
View
View
Access Point-User Association and Auction Algorithm-Based Pilot Assignment Schemes for Cell-Free Massive MIMO Systems
Article
  • Apr 2023
In this article, two effective schemes are proposed for association and pilot assignment to tackle the problem of spectral efficiency (SE) maximization and interference management in a user-centric cell-free (CF) massive multiple-input–multiple-output (mMIMO) system. First, a novel access point (AP)–user equipment (UE) association scheme is proposed, which ensures both scalability and association for all the UEs in a CF mMIMO system. After that, a pilot assignment scheme is also proposed which has an iterative structure that utilizes the auction algorithm to solve symmetrical assignment problems for pilot allocation. Numerical results demonstrate the superior performance of the proposed AP–UE association scheme compared to the existing AP preference-based association scheme. Moreover, the proposed pilot assignment scheme provides up to 30% average SE performance and up to 50% reduction in the average interference to signal power ratio (ISR) over that of several competing alternatives. Numerical simulated results further reveal that the proposed pilot assignment scheme improves the average SE and ISR reduction performance regardless of the number of UEs, APs, and pilot sequences in the system. Finally, the proposed pilot assignment scheme manifests the best balance in the tradeoff among the considered performance criteria relative to its comparatives.
View
View
View