No file available
To read the file of this research,
you can request a copy directly from the authors.
Augmented Reality-Based Advanced Driver-Assistance System for Connected Vehicles
Preprints and early-stage research may not have been peer reviewed yet.
Abstract
With the development of advanced communication technology, connected vehicles become increasingly popular in our transportation systems, which can conduct cooperative maneuvers with each other as well as road entities through vehicle-to-everything communication. A lot of research interests have been drawn to other building blocks of a connected vehicle system, such as communication, planning, and control. However, less research studies were focused on the human-machine cooperation and interface, namely how to visualize the guidance information to the driver as an advanced driver-assistance system (ADAS). In this study, we propose an augmented reality (AR)-based ADAS, which visualizes the guidance information calculated cooperatively by multiple connected vehicles. An unsignalized intersection scenario is adopted as the use case of this system, where the driver can drive the connected vehicle crossing the intersection under the AR guidance, without any full stop at the intersection. A simulation environment is built in Unity game engine based on the road network of San Francisco, and human-in-the-loop (HITL) simulation is conducted to validate the effectiveness of our proposed system regarding travel time and energy consumption.
ResearchGate has not been able to resolve any citations for this publication.
Testing autonomous driving algorithms on real autonomous vehicles is extremely costly and many researchers and developers in the field cannot afford a real car and the corresponding sensors. Although several free and open-source autonomous driving stacks, such as Autoware and Apollo are available, choices of open-source simulators to use with them are limited. In this paper, we introduce the LGSVL Simulator which is a high fidelity simulator for autonomous driving. The simulator engine provides end-to-end, full-stack simulation which is ready to be hooked up to Autoware and Apollo. In addition, simulator tools are provided with the core simulation engine which allow users to easily customize sensors, create new types of controllable objects, replace some modules in the core simulator, and create digital twins of particular environments.
Simulation offers the potential benefit of testing many miles in a variety of situations and environments. Not only can virtual testing environments be run quickly and in parallel, but a greater focus can be brought to bear on rare edge cases that need to be understood by ADS. They support simulation of sensor suites, environmental conditions, full control of all static and dynamic actors, maps generation and much more that enable automated vehicle simulations. They have large and growing communities that can contribute to the simulation ecosystem and develop use cases. This chapter presents ADS simulation research and capabilities discussed at the breakout session entitled “New Simulation tools for Training and Testing Automated Vehicles” at the 2019 Automated Vehicle Symposium in Orlando, FL. The section reviews key highlights and conclusions from three studies presented at the breakout session: 1) Responsibility Safety Sense (RSS) and software-in-the-loop (SiL) simulation; 2) augmented-reality-based testing with accelerated scenario design; and 3) human-in-the-loop testing for freeway cooperative merge.
With the rapid development of intelligent vehicles and Advanced Driving Assistance Systems (ADAS), a mixed level of human driver engagements is involved in the transportation system. Visual guidance for drivers is essential under this situation to prevent potential risks. To advance the development of visual guidance systems, we introduce a novel sensor fusion methodology, integrating camera image and Digital Twin knowledge from the cloud. Target vehicle bounding box is drawn and matched by combining results of object detector running on ego vehicle and position information from the cloud. The best matching result, with a 79.2% accuracy under 0.7 Intersection over Union (IoU) threshold, is obtained with depth image served as an additional feature source. Game engine-based simulation results also reveal that the visual guidance system could improve driving safety significantly cooperate with the cloud Digital Twin system.
As a good example of Advanced Driver-Assistance Systems (ADAS), Advisory Speed Assistance (ASA) helps improve driving safety and possibly energy efficiency by showing advisory speed to the driver of an intelligent vehicle. However, driver-based speed tracking errors often emerge, due to the perception and reaction delay, as well as imperfect vehicle control, degrading the effectiveness of ASA system. In this study, we propose a learning-based approach to modeling driver behavior, aiming to predict and compensate for the speed tracking errors in real time. Subject drivers are first classified into different types according to their driving behaviors using the k-nearest neighbors (k-NN) algorithm. A nonlinear autoregressive (NAR) neural network is then adopted to predict the speed tracking errors generated by each driver. A specific traffic scenario has been created in a Unity game engine-based driving simulator platform, where ASA system provides advisory driving speed to the driver via a head-up display (HUD). A human-in-the-loop simulation study is conducted by 17 volunteer drivers, revealing a 53% reduction in the speed error variance and a 3% reduction in the energy consumption with the compensation of the speed tracking errors. The results are further validated by a field implementation with a real passenger vehicle.
Digital twin, an emerging representation of cyber-physical systems, has attracted increasing attentions very recently. It opens the way to real-time monitoring and synchronization of real-world activities with the virtual counterparts. In this study, we develop a digital twin paradigm using an advanced driver assistance system (ADAS) for connected vehicles. By leveraging vehicle-to-cloud (V2C) communication, on-board devices can up-load the data to the server through cellular network. The server creates a virtual world based on the received data, processes them with the proposed models, and sends them back to the connected vehicles. Drivers can benefit from this V2C based ADAS, even if all computations are conducted on the cloud. The cooperative ramp merging case study is conducted, and the field implementation results show the proposed digital twin framework can benefit the transportation systems regarding mobility and environmental sustainability with acceptable communication delays and packet losses.
Connected and automated vehicles (CAVs) have the potential to address a number of safety, mobility, and sustainability issues of our current transportation systems. Cooperative longitudinal motion control is one of the key CAV technologies that allows vehicles to be driven in a cooperative manner to achieve system-wide benefits. In this paper, we provide a literature survey on the progress accomplished by researchers worldwide regarding cooperative longitudinal motion control systems of multiple CAVs. Specifically, the architecture of various cooperative CAV systems is reviewed to answer how cooperative longitudinal motion control can work with the help of multiple system modules. Next, different operational concepts of cooperative longitudinal motion control applications are reviewed to answer where they can be implemented in today's transportation systems. Different cooperative longitudinal motion control methodologies and their major characteristics are then described to answer what the critical design issues are. This paper concludes by describing an overall landscape of cooperative longitudinal motion control of CAVs, as well as pointing out opportunities and challenges in the future research and experimental implementations.
In recent years, the development of connected and automated vehicle (CAV) technology has inspired numerous advanced applications targeted at improving existing transportation systems. As one of the widely studied applications of CAV technology, connected eco-driving takes advantage of Signal Phase and Timing (SPaT) information from traffic signals to enable CAVs to approach and depart from sig-nalized intersections in an energy-efficient manner. However, the majority of the connected eco-driving studies have been numerical or microscopic traffic simulations. Only few studies have implemented the application on real vehicles, and even fewer have been focused on heavy-duty trucks. In this study, we developed a connected eco-driving system and equipped it on a heavy-duty diesel truck using cellular-based wireless communications. Field trials were conducted in the City of Carson, California, along two corridors with six connected signalized intersections capable of communicating their SPaT information. Early results showed the benefits of the system in smoothing the speed profiles of the equipped truck when approaching the connected signalized intersections.
[This paper won the SAE Best 2019 Paper Award] Agent-based modeling and simulation (ABMS) has been a popular approach for modeling autonomous and interacting agents in a multi-agent system. Specifically, ABMS can be applied to connected and automated vehicles (CAVs) since CAVs can operate autonomously with the help of onboard sensors, and cooperate with each other through vehicle-to-everything (V2X) communications. In order to improve energy efficiency and mobility of traffic, we have developed an online feedforward/feedback longitudinal controller for CAVs to cooperatively merge at ramps. Agent-based CAV models were built in the Unity3D environment, where vehicles are given connectivity and autonomy through C#-based scripting application programming interface (API). Agent-based infrastructure model is also built as a Unity3D simulation network based on the city of Mountain View, California. A simulation of cooperative on-ramp merging is carried out with a distributed consensus-based protocol, and then compared with the human-in-the-loop simulation where the on-ramp merging vehicle is driven by four different human drivers on a driving simulator. The benefits of introducing the proposed protocol are evaluated in terms of travel time, energy consumption, and pollutant emissions. The results show that the proposed cooperative on-ramp merging protocol can reduce average travel time, energy consumption, and pollutant emissions by 7%, 8%, and 58%, respectively, when compared to the human-in-the-loop scenario.
Connected vehicles will change the modes of future transportation management and organization, especially at intersections. In this paper, we propose a distributed conflict-free cooperation method for multiple connected vehicles at unsignalized intersections. We firstly project the approaching vehicles from different traffic movements into a virtual lane and introduce a conflict-free geometry topology considering the conflict relationship of involved vehicles, thus constructing a virtual platoon. Then we present the modeling of communication topology to describe two modes of information transmission between vehicles. Finally, a distributed controller is designed to stabilize the virtual platoon for conflict-free cooperation at intersections. Numerical simulations validate the effectiveness of this method.
Highway on-ramp merging is considered as one of the main factors that causes traffic congestion on highways. The drivers along the on-ramp need to adjust vehicle speeds and positions to enter the highway, while the drivers on the highway should also carefully accommodate vehicle speeds and positions to avoid collision with the merging vehicles from the on-ramp, which heavily affects upstream traffic flows. In congested traffic conditions, such maneuvers if inefficiently performed will lead to high risks of accidents and excessive energy consumption and pollutant emissions. In this work, we present an innovative approach to this scenario, where distributed consensus protocol is developed for Connected and Automated Vehicles (CAV) to cooperate with each other by using Vehicle-to-X (V2X) communications. A Road Side Unit (RSU)-equipped infrastructure installed in the merging area can receive vehicles’ information from both the highway and the on-ramp using Vehicle-to-Infrastructure (V2I) communications, and assign vehicles with sequence identifications based on their estimated arrival time at the merging area. Then vehicles apply distributed consensus protocol to adapt their speeds and positions to the preceding vehicles (either physical ones on the same lane or “ghost” ones projected from the other lane) with Vehicle-to-Vehicle (V2V) communications. After vehicles along the on-ramp merge into the highway, a new vehicle string (either tightly-coupled or loosely-coupled) is created. A comprehensive simulation study is conducted, and system-wide benefits in terms of traffic throughput and energy saving are also demonstrated in the work.
Recently there has been significant research on environment-focused Connected Vehicle (CV) applications that involve determining optimal speed profiles for vehicles traveling through signalized intersections and conveying this information to drivers via driver-vehicle interfaces (DVI's). However, findings from previous studies indicate that drivers may not be able to precisely follow the recommended speed profiles, resulting in degraded effectiveness of the applications. Moreover, the DVI could be distracting, which may compromise safety. As an alternative, partial automation can play an important role in ensuring that the benefits of these CV applications are fully realized. In this study, a partially automated vehicle system with an eco-approach and departure feature (called the GlidePath Prototype), which can receive dedicated short range communication (DSRC) message sets from the intersection and automatically follow recommended speed profiles, was developed, demonstrated, and evaluated. The results revealed that compared to manually following the recommended speed profiles, the GlidePath Prototype reduced fuel consumption by 17% on average. In some cases, the fuel savings are greater than 40% while the travel time is shortened by up to 64%. Furthermore, the system potentially improved the driving comfort since it would smooth out the speed profiles.
More and more modern vehicles are incorporating connected technologies that use data collected by sensors located in other vehicles or infrastructure (V2X) to assist the driver. Deployment of upcoming technologies based on cooperative systems will likely be the key step towards a significant reduction of accidents across the globe. Particularly in the case of conveying information to the driver in the form of dynamic warnings based on V2X communication technologies representing different levels of danger, it is crucial to investigate driver distraction levels, as well as the modality and dimension of the visual warnings and their appropriate in-vehicle location. This paper summarizes and analyzes previously published works in the field of human machine interaction (HMI) for use in a vehicular context, in particular those which address messages conveyed by cooperative systems, providing the designer and the general audience with a background, variety of approaches and example applications of the most current and important concepts in the field.
As wireless communication advances, multi-agent system (MAS) approaches to intersection traffic management have received increased attention. In this paper, we propose an improved multi-agent intersection management system where vehicle agents may form platoons using connected vehicles technologies. Traffic performance measures in terms of mobility and environmental sustainability were evaluated through a series of simulation studies, along with an analysis of the communication load of the system. Compared to the conventional traffic signal control system, the proposed platoon-based multi-agent intersection management system can shorten the average travel time by as much as 30% and reduce the fuel consumption and carbon dioxide emissions by around 23%, when the traffic volume is high. In comparison with a non-platoon-based multi-agent intersection management system, the proposed system can reduce the communication loads of an intersection management agent by up to 90% and exhibits strong robustness against the variation of traffic volumes.
Wireless vehicular communication has the potential to enable a host of new applications, the most important of which are a class of safety applications that can prevent collisions and save thousands of lives. The automotive industry is working to develop the dedicated short-range communication (DSRC) technology, for use in vehicle-to-vehicle and vehicle-to-roadside communication. The effectiveness of this technology is highly dependent on cooperative standards for interoperability. This paper explains the content and status of the DSRC standards being developed for deployment in the United States. Included in the discussion are the IEEE 802.11p amendment for wireless access in vehicular environments (WAVE), the IEEE 1609.2, 1609.3, and 1609.4 standards for Security, Network Services and Multi-Channel Operation, the SAE J2735 Message Set Dictionary, and the emerging SAE J2945.1 Communication Minimum Performance Requirements standard. The paper shows how these standards fit together to provide a comprehensive solution for DSRC. Most of the key standards are either recently published or expected to be completed in the coming year. A reader will gain a thorough understanding of DSRC technology for vehicular communication, including insights into why specific technical solutions are being adopted, and key challenges remaining for successful DSRC deployment. The U.S. Department of Transportation is planning to decide in 2013 whether to require DSRC equipment in new vehicles.
The use of wireless technology based on Vehicular Ad hoc Networks (VANETs) for information exchange can influence the drivers' behavior towards improving driving performance and reducing road accidents. This information can even be more relevant if it is presented as a video stream. In this paper we propose a system that relies on VANET and video-streaming technology: the See-Through System (STS). The system enhances driver's visibility and supports the driver's overtaking decision in challenging situations, such as overtaking a vision-obstructing vehicle. The use of the See-Through System provides the driver with an additional tool for determining if traffic conditions permit starting an overtaking maneuver thus reducing the risk of overtaking. We tested the See-Through System on an experimental vehicle on the road as well as in the context of a driving simulator for real world environment. Results are promising, since the use of the 802.11p standard wireless communication protocol allows a vehicle-to-vehicle connection without significant delay and the totality of the participants regarded the information provided by the STS as useful.
Articial intelligence research is ushering in a new era of sophisticated, mass-market transportation technology. While computers can already y a passenger jet better than a trained human pilot, people are still faced with the dangerous yet tedious task of driving au- tomobiles. Intelligent Transportation Systems (ITS) is the eld that focuses on integrating information technology with vehicles and transportation infrastructure to make transporta- tion safer, cheaper, and more ecien t. Recent advances in ITS point to a future in which vehicles themselves handle the vast majority of the driving task. Once autonomous vehicles become popular, autonomous interactions amongst multiple vehicles will be possible. Cur- rent methods of vehicle coordination, which are all designed to work with human drivers, will be outdated. The bottleneck for roadway eciency will no longer be the drivers, but rather the mechanism by which those drivers' actions are coordinated. While open-road driving is a well-studied and more-or-less-solved problem, urban trac scenarios, especially intersections, are much more challenging. We believe current methods for controlling trac, specically at intersections, will not be able to take advantage of the increased sensitivity and precision of autonomous vehicles as compared to human drivers. In this article, we suggest an alternative mechanism for coordinating the movement of autonomous vehicles through intersections. Drivers and intersections in this mechanism are treated as autonomous agents in a multiagent system. In this multiagent system, intersections use a new reservation-based approach built around a detailed communication protocol, which we also present. We demonstrate in simulation that our new mechanism has the potential to signican tly outperform current intersection control technology|trac lights and stop signs. Because our mechanism can emulate a trac light or stop sign, it subsumes the most popular current methods of intersection control. This article also presents two extensions to the mechanism. The rst extension allows the system to control human-driven vehicles in addition to autonomous vehicles. The second gives priority to emergency vehicles without signican t cost to civilian vehicles. The mechanism, including both extensions, is implemented and tested in simulation, and we present experimental results that strongly attest to the ecacy of this approach.
This paper presents findings of an ongoing empirical study focusing on identification and examination of several recurring freeway bottlenecks. It integrates the fusion of several traffic management and driver information data sources along an 18-km (11.2-mi) section of Autobahn 9 near Munich, Germany. These combined data sources further understanding of traffic dynamics and driver behavior before, during, and after bottleneck activation. The primary focus of this paper has been the investigation of variable speed limit and traveler information systems provided by means of overhead dynamic message signs so as to improve understanding of how these systems affect driver behavior and bottleneck formation and location. Toward that end, speed limit and information messages have been compared with actual traffic dynamics on the segment of Autobahn 9, and the analysis has found a strong correlation. It has been found that when drivers were warned of approaching congested conditions, the speed limit was reduced before bottleneck activation as a means of managing dense traffic. The system reduced the speed limit to control dense but still flowing traffic, and traffic continued to flow during congested periods at speeds between 30 and 40 km/h (19 and 25 mph). A comparison of sampled measurements of flow and speed with fundamental diagrams of speed–flow and flow–density indicated that speed limits were reduced. These findings will be connected to a before-and-after comparison of the system to be done to determine benefits and effects of such a system.
Vehicle-to-everything (V2X), including vehicle- to-vehicle (V2V), vehicle-to-pedestrian (V2P), vehicle-to-infrastructure (V2I), and vehicle-to-network (V2N) communications, improves road safety, traffic efficiency, and the availability of infotainment services. Standardization of Long Term Evolution (LTE)-based V2X has been actively conducted by the Third Generation Partnership Project (3GPP) to provide solutions for V2X communications, and has benefited from the global deployment and fast commercialization of LTE systems. LTE-based V2X was widely used as LTE-V in the Chinese vehicular communications industry, and LTE-based V2X was redefined as LTE V2X in 3GPP standardization progress. In this article, the overview of requirements and use cases in V2X services in 3GPP is presented. The up-to-date standardization of LTE V2X in 3GPP is surveyed. The challenges and detailed design aspects in LTE V2X are also discussed. Meanwhile, the enhanced V2X (eV2X) services and possible 5G solutions are analyzed. Finally, the implementations of LTE V2X are presented with the latest progress in industrial alliances, research, development of prototypes, and field tests.
Vehicular ad hoc networks (VANETs) are seen as an important enabling technology for improving both traffic safety and efficiency. Virtual Traffic Lights (VTLs) are a promising proposal for reducing travel time by efficiently controlling road intersections. VTLs use vehicle-to-vehicle communication to dynamically optimize traffic flow and they display traffic light information on the windshield. However, research so far has assumed that all vehicles are equipped with VTL support and it has ignored the incremental deployment phase, which could last decades. In this paper we present a solution for a VTL partial deployment scenario that is based on the idea of having VTL equipped cars display traffic light information on the outside of the vehicle. This allows drivers in non-equipped vehicles, or even pedestrians, to see the light color and respond accordingly. We show that the benefits of VTLs in terms of intersection throughput and average delay reduction grow as a function of the penetration rate of equipped vehicles.
Fully autonomous vehicles are technologically feasible with the current generation of hardware, as demonstrated by recent robot car competitions. Dresner and Stone proposed a new intersection control protocol called Autonomous Intersection Management (AIM) and showed that with autonomous vehicles it is possible to make intersection control much more efficient than the traditional control mechanisms such as traffic signals and stop signs. The protocol, however, has only been tested in simulation and has not been evaluated with real autonomous vehicles. To realistically test the protocol, we implemented a mixed reality platform on which an autonomous vehicle can interact with multiple virtual vehicles in a simulation at a real intersection in real time. From this platform we validated realistic parameters for our autonomous vehicle to safely traverse an intersection in AIM. We present several techniques to improve efficiency and show that the AIM protocol can still outperform traffic signals and stop signs even if the cars are not as precisely controllable as has been assumed in previous studies.
The vehicle platoon controller to be presented here plays an important part in the latest generation of the decentralised, autonomous intersection management. It is the executing instance in the intersection management and combines the results from different optimisations done by the vehicles on their way into the intersection in order to ensure optimum crossing of the intersection. To do so it must not only be platoon-stable but also have an enlarged functionality in comparison to a conventional platoon controller. In the following we present all relevant aspects for the design of the platoon controller and its use in simulations of the intersection management.
Human factors design guidance for driver-vehicle interfaces
- J Campbell
- J Brown
- J Graving
- C Richard
- M Lichty
- T Sanquist
- J Morgan
J. Campbell, J. Brown, J. Graving, C. Richard, M. Lichty, T. Sanquist, and
J. Morgan, "Human factors design guidance for driver-vehicle interfaces,"
U.S. Department of Transportation National Highway Traffic Safety
Administration, Tech. Rep., 2016.
VMS -Variable Message Signs: A literature review
- S Nygårdhs
- G Helmers
S. Nygårdhs and G. Helmers, "VMS -Variable Message Signs: A
literature review," Infrastructure maintenance, Tech. Rep. 570A, 2007.
CARLA: An open urban driving simulator
- A Dosovitskiy
- G Ros
- F Codevilla
- A Lopez
- V Koltun
A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, "CARLA:
An open urban driving simulator," arXiv preprint arXiv:1711.03938,
2017.
MOVESTAR: An open-source vehicle fuel and emission model based on usepa moves
- Z Wang
- G Wu
- G Scora
Z. Wang, G. Wu, and G. Scora, "MOVESTAR: An open-source vehicle
fuel and emission model based on usepa moves," 2020.