Recently, interest in Internet of Vehicles’ (IoV) technologies has significantly emerged due to the substantial development in the smart automobile industries. Internet of Vehicles’ technology enables vehicles to communicate with public networks and interact with the surrounding environment. It also allows vehicles to exchange and collect information about other vehicles and roads. IoV is introduced to enhance road users’ experience by reducing road congestion, improving traffic management, and ensuring the road safety. The promised applications of smart vehicles and IoV systems face many challenges, such as big data collection in IoV and distribution to attractive vehicles and humans. Another challenge is achieving fast and efficient communication between many different vehicles and smart devices called Vehicle-to-Everything (V2X). One of the vital questions that the researchers need to address is how to effectively handle the privacy of large groups of data and vehicles in IoV systems. Artificial Intelligence technology offers many smart solutions that may help IoV networks address all these questions and issues. Machine learning (ML) is one of the highest efficient AI tools that have been extensively used to resolve all mentioned problematic issues. For example, ML can be used to avoid road accidents by analyzing the driving behavior and environment by sensing data of the surrounding environment. Machine learning mechanisms are characterized by the time change and are critical to channel modeling in-vehicle network scenarios. This paper aims to provide theoretical foundations for machine learning and the leading models and algorithms to resolve IoV applications’ challenges. This paper has conducted a critical review with analytical modeling for offloading mobile edge-computing decisions based on machine learning and Deep Reinforcement Learning (DRL) approaches for the Internet of Vehicles (IoV). The paper has assumed a Secure IoV edge-computing offloading model with various data processing and traffic flow. The proposed analytical model considers the Markov decision process (MDP) and ML in offloading the decision process of different task flows of the IoV network control cycle. In the paper, we focused on buffer and energy aware in ML-enabled Quality of Experience (QoE) optimization, where many recent related research and methods were analyzed, compared, and discussed. The IoV edge computing and fog-based identity authentication and security mechanism were presented as well. Finally, future directions and potential solutions for secure ML IoV and V2X were highlighted.
1. Introduction
Intelligent Transportation Systems (ITS) and computational systems’ rapid development opened new scientific research in smart traffic safety with comfort and efficient solutions. Artificial Intelligence (AI) has been widely used to optimize traditional data-driven approaches in different research areas [1]. AI-based on the Vehicle-to-Everything (V2X) system obtains information from various sources, i.e., car, train, bus, etc., and enables to increase the realization of drivers and forecast to avoid accidents. This progression has directed to the opportunity to understand smart driving, which was built on the idea of copying real driving comportment, while avoiding human mistakes and bringing comfortable safety to drivers. Many services have been invented from crowd and light road traffic to adapting traffic, a legacy from self-based vehicle systems to the IoV [2]. IoV is addressed to change the interaction between the vehicles, roadside stations, on-board stations, and environments to communicate data and multimedia between various networks. The motivation of IoV is to be adopted and build the human-vehicle-roadside onboard IoT Connected services within the various vehicle and different networks.
Machine Learning (ML) is responsible for a wide range of AI applications. The ML techniques are unsupervised, supervised, and reinforcement learning. In the unsupervised ML scheme, training depends on untagged data. It tries to find an adequate representation of untagged data. While, in supervised learning, it learns from a group of labeled data. In supervised learning, regression and classification schemes train the discrete and continuous data for prediction and decision-making. Reinforcement learning (RL) studies from the learning agent’s activities from the consistent reward to capitalize on the notion of cumulative rewards. The Markov Decision Process (MDP) is a sample of RL [2]. This scheme is a perfect technique for taking many issues’ research problems in vehicular networks, such as in collaborative optimization of oil consumption for a specific area and optimum path forecasting of electric vehicles and minimizing traffic congestions.
Given the importance of the use of Artificial Intelligence (AI) in IoV, as it provides smart models in most of its applications, this paper contributes a brief concept on one of the AI methods known as machine learning and the possibility of its use in several specific aspects related to the IoV network. In IoV networks, edge computing and caching problems are the most considered challenges requiring an intelligent optimization method. Edge computing and caching challenges are related to many factors, i.e., channel condition, dynamic communication topology, and resource allocation management. In the IoV network architecture, artificial intelligence is in a separated layer responsible for virtual cloud infrastructure. The AI layer act as an information management brain. Deep neural networks are ML algorithms developed to make decisions according to learned IoV resource actions [3].
This paper has conducted a critical review with analytical simulation for offloading mobile edge computing decisions based on learning and Deep Reinforcement Learning (DRL) technologies for vehicular communication in (IoV). We have considered a typical IoV network architecture with one IoV Edge-Computing (IoVEC) and one mobile user. The tasks of the device arrive as a flow in time. Our analytical model performs the offloading decision process of the task flow as a Markov Decision Process (MDP). The optimization object minimizes the weighted sum of offloading latency and power consumption, which is decomposed into the reward of each time slot.
The rest of the paper is organized as follows; the study background and motivation are presented in Section 2, where systematic technical knowledge and motivation of secure ML in the IoV field were discussed. A brief concept of AI in IoV is reviewed in Section 3 by considering using AI in multimedia and IoV edge-based and Vehicle-to-Everything’s Internet communications. Section 4 provides a clear concept about the contribution of AI to enabling QoS and QoE optimization, where QoS manages and controls resources of the IoV network by setting various priorities for each data type, while QoE discusses the measurement of the overall system homogeneity and stability of service. Section 5 provides a detailed description of using machine learning algorithms with IoV in different aspects. The most common use cases of ML in IoV applications are presented in Section 6. Section 7 gives a brief review of the possible future research directions and potential ML solutions in IoV. Finally, the conclusion is presented in Section 8.
2. Background and Motivation
Due to the significant research and technology development in wireless communication, the traditional ITS has to care about the vehicular communication field. Recently, the numbers of vehicles have increased due to transporting huge numbers of people from region to region. This increment in the number of vehicles would create issues such as crowding and accidents on the roads. This issue could be considered as one of the main problems in daily life. Most of the general form of vehicular networking is known as the vehicular ad hoc network (VANET) [4]. VANET consists of Vehicle-to-Vehicle (V2V) and Vehicle-to-Roadside (V2R) communications to transfer the vehicles’ information. The VANETs’ communication depends on the Roadside Unit (RSU) to support Wireless Access in Vehicular Environments (WAVE).
The Roadside Unit (RSU) along the roadwork acts as wireless access points’ support communication to the vehicles inside its coverage area [5]. The hybrid vehicular network architecture, interacted with the cellular communication architectures, will operate the cellular communication services, i.e., voice, in collaborations. Due to the current trend to connect vehicular networks to information centers and the need to exchange data, IoV allows enabling Internet access among on-road vehicles. One of the essential IoV applications is to improve the features of VANETs to reduce various issues in urban traffic and accident environments [6]. IoV enables the vehicular road networks to interconnect with different wireless network technologies i.e., Wi-Fi and 4G/LTE for V2I, IEEE WAVE for V2V and V2R, MOST/Wi-Fi for V2S, and CarPlay NCF for V2P. It is useful to provide a comprehensive presentation to ML's concepts in IoV and explain the areas that could contribute to these networks' development [7].
In recent years, the arising need to introduce artificial intelligence technologies in IoV applications has been facing some challenges. These challenges are related to making particular decisions and forecasting different aspects of IoV, such as traffic monitoring and management, big data processing, energy and resource management, and intelligent interaction with users to provide high-quality services [6, 7]. Several studies have been conducted on using artificial intelligence techniques such as machine learning to develop solutions to most of these challenges [8]. Due to the current developments in the field of AI, especially in using machine learning techniques to make intelligent decisions in several IoV applications, it is useful to provide a comprehensive presentation to study some concepts of using ML in IoV and explain the areas that could contribute to the development of these networks.
3. Artificial Intelligence Methods in the IoV Network
AI technology is more related to the layer responsible for presentation and functionalities in the IoV-layered architecture. A term of virtual cloud infrastructure can describe this layer and be responsible for storing, processing, analyzing the information received from the IoV network, and decision-making based on the analyzed information. In IoV, the computation and analysis are provided by Big Data Analysis (BDA) and Vehicular Cloud Computing (VCC) systems which are used as an information management center [9]. According to the IoV applications, many services can be provided by the IoT cloud environment, requiring intelligent service management. The smart cloud-computing servers provide many smart services, i.e., safety, traffic administration, entertaining, and subscription, which are the foundation of elegance in IoV. The cloud servers based on AI enable the procedure and develop AI in Real-Time (RT) massive data traffic to provide a smart decision for intelligent customer services. The Vehicular Cyber-Physical System (VCPS) is considered a vehicular network model that concerns disseminating information using next-generation Internet [10]. VCPS depends on AI technology to provide smart processing in huge data traffic utilizing fog and cloud computing for civilian and safety applications.
In IoV networks, edge computing and caching problems are the most considered challenges requiring an intelligent optimization method. Edge computing and caching challenges are related to many factors, i.e., channel condition, dynamic communication topology, and resource allocation management. AI in IoV provides an intelligent approach to solve most of these challenges. The use of ML offers a means of interaction to the IoV environment and enables the creation of an agent that learns challenging factors to optimize the overall IoV network utilization [11]. Q-learning and deep neural networks are ML algorithms developed to make decisions according to learned IoV resource actions. In the IoV network architecture, the presentation of artificial intelligence in a separated layer is responsible for virtual cloud infrastructure. The AI layer acts as an information management brain [10, 11]. The AI layer in IoT architecture consists of big data analysis, cloud computing, and expert systems. It plays an essential task in storing, processing, and analyzing the information received from the coordination layer and takes decisions according to the network status.
3.1. Artificial Intelligence Methods for IoV Multimedia Communication
The deployment of IoV in multimedia communications requires a device that allows data exchange and communication with other surrounded devices. This can be achieved by any technology such as Personal Area Networks (PAN), the Internet of Things (IoT), and Wireless Sensor Network (WSN). Data exchange’s scalability and flexibility are quite important for IoV by integrating sensors, vehicles, humans, actuators, machines, etc. The sensor in intelligent IoV enhances vehicle and traffic systems’ safety, while harmonized traffic data transfer in the IoV system network enhances vehicular system efficiency. However, the amount of energy consumption, required capacity, green buffer-awareness, and message exchange through IoVs may compromise severe data transfer risk [12]. AI based on self-driven vehicles encourages several types of applications with many benefits of intelligence. Especially for the increase in the amount of data and complexity, which the algorithms will be processing, it is precise and effective for future directions. As growing, the high traffic information in IoVs required a smart utility, followed to efficiently monitor and manage the demand for intelligent IT technologies [13]. With the rapid evolution in digital technologies, the development of multimedia depends on the IoV system, and it needs a portable device to collect a voluminous amount of information for aiding and guiding the specific trend for analyzing the transportation industry by IoT-based platforms.
Figure 1 shows the structure of multimedia communication through sensor nodes in the IoV system. Its structure consists of three main parts for IoV data and information network techniques and models. The data and information network techniques and models are redeveloped with the central server. The inter and intravehicle network connections among various sections are executed by transferring urgent and sensitive data throughout the vehicle via adaptive and smart wireless communication. The vehicle’s client enables QoS monitoring [13, 14]. In this structure, the IoV traffic can be arranged based on the category containing sensitive/standard, prestored, real-time, or high-definition resolution, respectively. To accomplish the real-time and jitter-tolerant data and information exchange with low buffer storage and scarce power supply, it should be fortified to tolerate the raw unprocessed data and information into the regular and synchronized format with good and clean visibility.