No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
On the Computing and Communication Tradeoff in Reasoning-Based Multi-User Semantic Communications
... The remaining problem is how to translate correctly and effectively, which they achieve with a Bayesoptimal selection strategy. The authors in [49] also proposed a multi-user semantic communication system but focused on a different problem instead of the mismatch language. Specifically, they focus on the tradeoff between the computing and communication resources among semantic communication users, where each user has the reasoning ability supported by the massive computing resources, in which they can fill in the missing information when receiving a message. ...
Semantic Communication is becoming the next pillar in wireless communication technology due to its various capabilities. However, it still encounters various challenging obstacles that need to be solved before real-world deployment. The major challenge is the lack of standardization across different directions, leading to variations in interpretations and objectives. In the survey, we provide detailed explanations of three leading directions in semantic communications, namely Theory of Mind, Generative AI, Deep Joint Source-Channel Coding. These directions have been widely studied, developed, and verified by institutes worldwide, and their effectiveness has increased along with the advancement in technology. We first introduce the concepts and background of these directions. Firstly, we introduce the Theory of Mind, where the communication agents interact with each other, gaining understanding from observations and slowly forming a common language. Secondly, we present generative AI models, which can create new content and offer more freedom to interpret the data beyond the limitation of semantic meaning compression of raw data before transmitting it. The received signal is then decoded by another generative AI model to execute the oriented task. Thirdly, we review deep learning models to jointly optimize the source and channel coding modules. Then, we present a comprehensive survey of existing works in each direction, thereby offering readers an overview of past achievements and potential avenues for further contribution. Moreover, for each direction, we identify and discuss the existing challenges that must be addressed before these approaches can be effectively deployed in real-world scenarios.
Semantic communication is viewed as a revolutionary paradigm that can potentially transform how we design and operate wireless communication systems. However, despite a recent surge of research activities in this area, remarkably, the research landscape is still limited in at least three ways. First and foremost, the definition of a “semantic communication system" is ambiguous and varies widely between different studies. This lack of consensus makes it challenging to develop rigorous and scalable frameworks for building semantic communication networks. Secondly, current approaches to building semantic communication networks are limited by their reliance on data-driven and information-driven AI-augmented networks. These networks remain “tied" to the data, which limits their ability to perform versatile logic. In contrast, knowledge-driven and reasoning-driven AI-native networks would allow for more flexible and powerful communication capabilities. However, there is currently a lack of technical foundations to support such networks. Thirdly, the concept of “semantic representation" is not well understood yet, and its role in embedding meaning and structure in data transferred across wireless network is still a subject of active research. The development of semantic representations that are minimalist, generalizable, and efficient is critical to enabling the transmitter and receiver to generate content via a minimally semantic representation. To address these limitations, in this tutorial, we propose the first rigorous and holistic vision of an end-to-end semantic communication network that is founded on novel concepts from artificial intelligence (AI), causal reasoning, transfer learning, and minimum description length theory. We first discuss how the design of semantic communication networks requires a move from data-driven AI-augmented networks, in which wireless networks remain “tied" to data, towards reasoning-driven AI-native networks which can perform versatile logic and generalizable intelligence. We then distinguish the concept of semantic communications from several other approaches that have been conflated with it. We opine that building effective and efficient semantic communication systems necessitates surpassing the creation of new encoder and decoder types at the transmitter/receiver side, or developing an “AI for wireless" framework that only extracts application features or fine-tunes wireless protocols/algorithms. Then, we identify the main tenets that are needed to build an end-to-end semantic communication network. Among those building blocks of a semantic communication network, we highlight the necessity of creating semantic representations of data that satisfy the key properties of minimalism, generalizability, and efficiency so as to faithfully represent the data and enable the transmitter and receiver to do more with less. We then explain how those representations can form the basis of a so-called semantic language that will allow a transmitter and receiver to communicate at a semantic level. We then concretely define the concept of reasoning by investigating the fundamentals of causal representation learning and their role in designing reasoning-driven semantic communication networks. For such reasoning-driven networks, we propose novel and essential semantic communication key performance indicators (KPIs) and metrics, including new “reasoning capacity" measures that could surpass Shannon’s bound to capture the imminent convergence of computing and communication resources. Finally, we explain how semantic communications can be scaled to large-scale networks such as 6G and beyond cellular networks. In a nutshell, we expect this tutorial to provide a unified and self-contained reference on how to properly build, design, analyze, and deploy next-generation semantic communication networks.
In this paper, the problem of joint transmission and computation resource allocation for a multi-user probabilistic semantic communication (PSC) network is investigated. In the considered model, users employ semantic information extraction techniques to compress their large-sized data before transmitting them to a multi-antenna base station (BS). Our model represents large-sized data through substantial knowledge graphs, utilizing shared probability graphs between the users and the BS for efficient semantic compression. The resource allocation problem is formulated as an optimization problem with the objective of maximizing the sum of the equivalent rate of all users, considering the total power budget and semantic resource limit constraints. The computation load considered in the PSC network is formulated as a non-smooth piecewise function with respect to the semantic compression ratio. To tackle this non-convex non-smooth optimization challenge, a three-stage algorithm is proposed, where the solutions for the received beamforming matrix of the BS, the transmit power of each user, and the semantic compression ratio of each user are obtained stage by stage. The numerical results validate the effectiveness of our proposed scheme.
This paper studies the problem of performing a sequence of optimal interventions in a causal dynamical system where both the target variable of interest and the inputs evolve over time. This problem arises in a variety of domains e.g. system biology and operational research. Dynamic Causal Bayesian Optimization (DCBO) brings together ideas from sequential decision making, causal inference and Gaussian process (GP) emulation. DCBO is useful in scenarios where all causal effects in a graph are changing over time. At every time step DCBO identifies a local optimal intervention by integrating both observational and past interventional data collected from the system. We give theoretical results detailing how one can transfer interventional information across time steps and define a dynamic causal GP model which can be used to quantify uncertainty and find optimal interventions in practice. We demonstrate how DCBO identifies optimal interventions faster than competing approaches in multiple settings and applications.
Recently, deep learned enabled end-to-end (E2E) communication systems have been developed to merge all physical layer blocks in the traditional communication systems, which make joint transceiver optimization possible. Powered by deep learning, natural language processing (NLP) has achieved great success in analyzing and understanding a large amount of language texts. Inspired by research results in both areas, we aim to provide a new view on communication systems from the semantic level. Particularly, we propose a deep learning based semantic communication system, named DeepSC, for text transmission. Based on the Transformer, the DeepSC aims at maximizing the system capacity and minimizing the semantic errors by recovering the meaning of sentences, rather than bit-or symbol-errors in traditional communications. Moreover, transfer learning is used to ensure the DeepSC applicable to different communication environments and to accelerate the model training process. To justify the performance of semantic communications accurately, we also initialize a new metric, named sentence similarity. Compared with the traditional communication system without considering semantic information exchange, the proposed DeepSC is more robust to channel variation and is able to achieve better performance, especially in the low signal-to-noise (SNR) regime, as demonstrated by the extensive simulation results.
The rapid uptake of mobile devices and the rising popularity of mobile applications and services pose unprecedented demands on mobile and wireless networking infrastructure. Upcoming 5G systems are evolving to support exploding mobile traffic volumes, agile management of network resource to maximize user experience, and extraction of fine-grained real-time analytics. Fulfilling these tasks is challenging, as mobile environments are increasingly complex, heterogeneous, and evolving. One potential solution is to resort to advanced machine learning techniques to help managing the rise in data volumes and algorithm-driven applications. The recent success of deep learning underpins new and powerful tools that tackle problems in this space. In this paper we bridge the gap between deep learning and mobile and wireless networking research, by presenting a comprehensive survey of the crossovers between the two areas. We first briefly introduce essential background and state-of-the-art in deep learning techniques with potential applications to networking. We then discuss several techniques and platforms that facilitate the efficient deployment of deep learning onto mobile systems. Subsequently, we provide an encyclopedic review of mobile and wireless networking research based on deep learning, which we categorize by different domains. Drawing from our experience, we discuss how to tailor deep learning to mobile environments. We complete this survey by pinpointing current challenges and open future directions for research.
Building the next-generation wireless systems that could support services such as the metaverse, digital twins (DTs), and holographic teleportation is challenging to achieve exclusively through incremental advances to conventional wireless technologies like metasurfaces or holographic antennas. While the 6G concept of artificial intelligence (AI)-native networks promises to overcome some of the limitations of existing wireless technologies, current developments of AI-native wireless systems rely mostly on conventional AI tools such as auto-encoders and off-the-shelf artificial neural networks. However, those tools struggle to manage and cope with the complex, nontrivial scenarios faced in real-world wireless environments and the growing quality-of-experience (QoE) requirements of the aforementioned, emerging wireless use cases. In contrast, in this article, we propose to fundamentally revisit the concept of AI-native wireless systems, equipping them with the common sense necessary to transform them into artificial general intelligence (AGI)-native systems. Our envisioned AGI-native wireless systems acquire common sense by exploiting different cognitive abilities such as reasoning and analogy. These abilities in our proposed AGI-native wireless system are mainly founded on three fundamental components: a perception module, a world model, and an action-planning component. Collectively, these three fundamental components enable the four pillars of common sense that include dealing with unforeseen scenarios through horizontal generalizability, capturing intuitive physics, performing analogical reasoning, and filling in the blanks. Toward developing these components, we start by showing how the perception module can be built through abstracting real-world elements into generalizable representations. These representations are then used to create a world model, founded on principles of causality and hyperdimensional (HD) computing. Specifically, we propose a concrete definition of a world model, viewing it as an HD causal vector space that aligns with the intuitive physics of the real world—a cornerstone of common sense. In addition,we discuss how this proposed world model can enable analogical reasoning and manipulation of the abstract representations. Then, we show how the world model can drive an action-planning feature of the AGI-native network. In particular, we propose an intent-driven and objective-driven planning method that can maneuver the AGI-native network to plan its actions. These planning methods are based on brain-inspired frameworks such as integrated information theory and hierarchical abstractions that play a crucial role in enabling human-like decision-making. Next, we explain how an AGI-native network can be further exploited to enable three use cases related to human users and autonomous agent applications: 1) analogical reasoning for the next-generation DTs; 2) synchronized and resilient experiences for cognitive avatars; and 3) brain-level metaverse experiences exemplified by holographic teleportation. Finally, we conclude with a set of recommendations to ignite the quest for AGI-native systems. Ultimately, we envision this article as a roadmap for the next generation of wireless systems beyond 6G.
The wireless metaverse will create diverse user experiences at the intersection of the physical, digital, and virtual worlds. These experiences will enable novel interactions between the constituents (e.g., extended reality (XR) users and avatars) of the three worlds. However, remarkably, to date, there is no holistic vision that identifies the full set of these worlds, constituents, and experiences, and the implications of their associated interactions on next-generation communication and computing systems. In this article, we present a holistic vision of a limitless, wireless metaverse that distills the metaverse into an intersection of seven worlds and experiences that include the: physical, digital, and virtual worlds, along with the cyber, extended, live, and parallel experiences. We then articulate how these experiences bring forth interactions between diverse metaverse constituents, namely, humans and avatars and connected intelligence systems and digital twins (DTs). Then, we explore the wireless, computing, and artificial intelligence (AI) challenges that must be addressed to establish metaverseready networks that support these experiences and interactions. We particularly highlight the need for end-to-end synchronization of DTs, and the role of human-level AI and reasoning abilities for cognitive avatars. We conclude with a set of recommendations to deploy the limitless metaverse over future wireless systems.
The automation of factories and manufacturing processes has been accelerating over the past few years, leading to an ever-increasing number of scenarios with networked agents whose coordination requires reliable wireless communication. In this context, goal-oriented communication adapts transmissions to the control task, prioritizing the more relevant information to decide which action to take. Instead, networked control models follow the opposite pathway, optimizing physical actions to address communication impairments. In this work, we propose a joint design that combines goal-oriented communication and networked control into a single optimization model, an extension of a multi-agent Partially Observable Markov Decision Process (POMDP), which we call Cyber-Physical POMDP. The proposed model is flexible enough to represent a large variety of scenarios and we illustrate its potential in two simple use cases with a single agent and a set of supporting sensors. Our results assess that the joint optimization of communication and control tasks radically improves the performance of networked control systems, particularly in the case of constrained resources, leading to implicit coordination of communication actions.
Semantic communication has gained significant attention from researchers as a promising technique to replace conventional communication in the next generation of communication systems, primarily due to its ability to reduce communication costs. However, little literature has studied its effectiveness in multi-user scenarios, particularly when there are variations in the model architectures used by users and their computing capacities. To address this issue, we explore a semantic communication system that caters to multiple users with different model architectures by using a multi-purpose transmitter at the base station (BS). Specifically, the BS in the proposed framework employs semantic and channel encoders to encode the image for transmission, while the receiver utilizes its local channel and semantic decoder to reconstruct the original image. Our joint source-channel encoder at the BS can effectively extract and compress semantic features for specific users by considering the signal-to-noise ratio (SNR) and computing capacity of the user. Based on the network status, the joint source-channel encoder at the BS can adaptively adjust the length of the transmitted signal. A longer signal ensures more information for high-quality image reconstruction for the user, while a shorter signal helps avoid network congestion. In addition, we propose a hybrid loss function for training, which enhances the perceptual details of reconstructed images. Finally, we conduct a series of extensive evaluations and ablation studies to validate the effectiveness of the proposed system.
Task-oriented communication is a new paradigm that aims at providing efficient connectivity for accomplishing intelligent tasks rather than reception of every transmitted bit. This paper proposes task-oriented communication architecture for end-to-end semantics transmission, where extracted semantics is compressed by the proposed adaptable semantic compression (ASC) method. However, accommodating multiple users in a delay-intolerant system poses a challenge. Higher compression ratios conserve channel resources but cause semantic distortion, while lower ratios demand more resources and may lead to transmission failure due to delay constraints. To address this, we optimize both compression ratio and resource allocation to maximize task success probability. Specifically, we propose a compression ratio and resource allocation (CRRA) algorithm that separates the problem into two subproblems and solving them iteratively. Furthermore, for scenarios with varying service levels among users, a compression ratio, resource allocation, and user selection (CRRAUS) algorithm is proposed, adaptively selecting users through branch and bound method. Simulation results show that ASC approach can reduce the size of transmitted data by up to 80% without compromising task success probability. Furthermore, numerical results clearly demonstrate that both the proposed CRRA and CRRAUS algorithms lead to substantial improvements in terms of success gains when compared to the baselines.
Semantic communication (SC) is an effective approach to communicate reliably with minimal data transfer while simultaneously providing seamless connectivity. In this paper, a novel emergent SC (ESC) framework is proposed. This ESC system is composed of two key components: A signaling game for emergent language design and a neuro-symbolic (NeSy) artificial intelligence (AI) approach for causal reasoning. In order to design the language, the signaling game is solved using an alternating maximization between the transmit and receive nodes utilities. The generalized Nash equilibrium is characterized, and it is shown that the resulting transmit and receive signaling strategies lead to a local equilibrium solution. As such, the emergent language not only creates an efficient (in physical bits transmitted) transmit vocabulary dependent on communication contexts but it also aids the reasoning process (and enables generalization to unseen scenarios) by splitting complex received messages into simpler reasoning tasks for the receiver. The causal description (symbolic component) at the transmitter is then modeled using the emerging AI framework of generative flow networks (GFlowNets), whose parameters are optimized for higher semantic reliability. Using the reconstructed causal state, the receiver evaluates a set of logical formulas (symbolic part) to execute its task. This evaluation of logical formulas is done by combining GFlowNet, with the logical expressiveness of the symbolic structure, inspired from logical neural networks. The ESC system is also designed to enhance the novel semantic metrics of information, reliability, distortion and similarity that are designed using rigorous algebraic properties from category theory thereby generalizing the metrics beyond Shannon’s notion of uncertainty. Simulation results confirm that the ESC system effectively communicates with reduced bits and achieves superior semantic reliability compared to conventional wireless systems and state-of-the-art SC systems lacking causal reasoning capabilities. Additionally, the overhead involved in language creation diminishes over time, validating the system’s ability to generalize across multiple tasks.
Goal-oriented semantic communication will be a pillar of next-generation wireless networks. Despite significant recent efforts in this area, most prior works are focused on specific data types (e.g., image or audio), and they ignore the goal and effectiveness aspects of semantic transmissions. In contrast, in this paper, a holistic goal-oriented semantic communication framework is proposed to enable a speaker and a listener to cooperatively execute a set of sequential tasks in a dynamic environment. A common language based on a hierarchical belief set is proposed to enable semantic communications between speaker and listener. The speaker, acting as an observer of the environment, utilizes the beliefs to transmit an initial description of its observation (called event) to the listener. The listener is then able to infer on the transmitted description and complete it by adding related beliefs to the transmitted beliefs of the speaker. As such, the listener reconstructs the observed event based on the completed description, and it then takes appropriate action in the environment based on the reconstructed event. An optimization problem is defined to determine the perfect and abstract description of the events while minimizing the various communication costs with constraints on the task execution time and belief efficiency. Then, a novel bottom-up curriculum learning (CL) framework based on reinforcement learning is proposed to solve the optimization problem and enable the speaker and listener to gradually identify the structure of the belief set and the perfect and abstract description of the events. Simulation results show that the proposed CL method outperforms classical RL and CL without inference scheme in terms of convergence time, task execution cost and time, reliability, and belief efficiency.
While semantic communications have shown the potential in the case of single-modal single-users, its applications to the multi-user scenario remain limited. In this paper, we investigate deep learning (DL) based multi-user semantic communication systems for transmitting single-modal data and multimodal data, respectively. We adopt three intelligent tasks, including, image retrieval, machine translation, and visual question answering (VQA) as the transmission goal of semantic communication systems. We propose a Transformer based framework to unify the structure of transmitters for different tasks. For the single-modal multi-user system, we propose two Transformer based models, named, DeepSC-IR and DeepSC-MT, to perform image retrieval and machine translation, respectively. In this case, DeepSC-IR is trained to optimize the distance in embedding space between images and DeepSC-MT is trained to minimize the semantic errors by recovering the semantic meaning of sentences. For the multimodal multi-user system, we develop a Transformer enabled model, named, DeepSC-VQA, for the VQA task by extracting text-image information at the transmitters and fusing it at the receiver. In particular, a novel layer-wise Transformer is designed to help fuse multimodal data by adding connection between each of the encoder and decoder layers. Numerical results show that the proposed models are superior to traditional communications in terms of the robustness to channels, computational complexity, transmission delay, and the task-execution performance at various task-specific metrics.
The goal of this paper is to promote the idea that including semantic and goal-oriented aspects in future 6G networks can produce a significant leap forward in terms of system effectiveness and sustainability. Semantic communication goes beyond the common Shannon paradigm of guaranteeing the correct reception of each single transmitted bit, irrespective of the meaning conveyed by the transmitted bits. The idea is that, whenever communication occurs to convey meaning or to accomplish a goal, what really matters is the impact that the received bits have on the interpretation of the meaning intended by the transmitter or on the accomplishment of a common goal. Focusing on semantic and goal-oriented aspects, and possibly combining them, helps to identify the relevant information, i.e. the information strictly necessary to recover the meaning intended by the transmitter or to accomplish a goal. Combining knowledge representation and reasoning tools with machine learning algorithms paves the way to build semantic learning strategies enabling current machine learning algorithms to achieve better interpretation capabilities and contrast adversarial attacks. 6G semantic networks can bring semantic learning mechanisms at the edge of the network and, at the same time, semantic learning can help 6G networks to improve their efficiency and sustainability.
Cambridge Core - Communications and Signal Processing - Game Theory for Next Generation Wireless and Communication Networks - by Zhu Han
Causal Bayesian Optimization
- V Aglietti
- X Lu
- A Paleyes
- J Gonzalez