No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Spatio-temporal Analysis of Multi-agent Scheduling Behaviors on Fixed-track Networks
... However, this quantitative approach neglects an analysis of (a) which behavior changes have led to improvements in performance, (b) desirable behaviors (e.g., cooperation) that do not immediately reflect in the performance metric, and (c) the sequence in which behaviors are learned. Few visualization approaches (e.g., [AWB20,AWWB22]) enable a qualitative exploration of team behaviors, but they only focus on analyzing the fully trained agents. Addressing the challenge, we propose a visual analysis approach for evolving behaviors of AI agents in the bomblaying game environment Pommerman (Figure 1). ...
Analyzing the training evolution of AI agents in a multi-agent environment helps to understand changes in learned behaviors, as well as the sequence in which they are learned. We train an existing Pommerman team from scratch and, at regular intervals, let it battle against another top-performing team. We define thirteen game-specific behaviors and compute their occurrences in 600 matches. To investigate the evolution of these behaviors, we propose a visualization approach and showcase its usefulness in an application example.
Event sequence visualization aids analysts in many domains to better understand and infer new insights from event data. Analysing behaviour before or after a certain event of interest is a common task in many scenarios. In this paper, we introduce, formally define, and position double trees as a domain‐agnostic tree visualization approach for this task. The visualization shows the sequences that led to the event of interest as a tree on the left, and those that followed on the right. Moreover, our approach enables users to create selections based on event attributes to interactively compare the events and sequences along colour‐coded categories. We integrate the double tree and category‐based comparison into a user interface for event sequence analysis. In three application examples, we show a diverse set of scenarios, covering short and long time spans, non‐spatial and spatial events, human and artificial actors, to demonstrate the general applicability of the approach.
For autonomous vehicles to safely share the road with human drivers, autonomous vehicles must abide by specific "road rules" that human drivers have agreed to follow. "Road Rules" include rules that drivers are required to follow by law – such as the requirement that vehicles stop at red lights – as well as more subtle social rules – such as the implicit designation of fast lanes on the highway. In this paper, we provide empirical evidence that suggests that – instead of hard-coding road rulesinto self-driving algorithms – a scalable alternative may be to design multi-agent environments in which road rules emerge as optimal solutions to the problem of maximizing traffic flow. We analyze what ingredients in driving environments cause the emergence of these road rules and find that two crucial factors are noisy perception and agents’ spatial density. We provide qualitatively and quantitative evidence of the emergence of seven social driving behaviors, ranging from obeying traffic signals to following lanes, all of which emerge from training agents to drive quickly to destinations without colliding. Our results add empirical support for the social road rules that countries worldwide have agreed on for safe, efficient driving.
Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms, however, have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This article addresses an important aspect of deep RL related to situations that require multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multiagent deep RL (MADRL) is presented, including nonstationarity, partial observability, continuous state and action spaces, multiagent training schemes, and multiagent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to the future development of more robust and highly useful multiagent learning methods for solving real-world problems.
Intelligent Transportation Systems are an important enabler for the smart cities paradigm. Currently, such systems generate massive amounts of granular data that can be analyzed to better understand people’s dynamics. To address the multivariate nature of spatiotemporal urban mobility data, researchers and practitioners have developed an extensive body of research and interactive visualization tools. Data visualization provides multiple perspectives on data and supports the analytical tasks of domain experts. This article surveys related studies to analyze which topics of urban mobility were addressed and their related phenomena, and to identify the adopted visualization techniques and sensors data types. We highlight research opportunities based on our findings.
Multi-Agent Systems (MAS) have received tremendous attention from scholars in different disciplines, including computer science and civil engineering, as a means to solve complex problems by subdividing them into smaller tasks. The individual tasks are allocated to autonomous entities, known as agents. Each agent decides on a proper action to solve the task using multiple inputs, e.g. history of actions, interactions with its neighbouring agents, and its goal. MAS have found multiple applications including modeling complex systems, smart grids and computer networks. Despite their wide applicability, there are still a number of challenges faced by MAS including coordination between agents, security, and task allocation. This survey provides a comprehensive discussion of all aspects of MAS, starting from definitions, features, applications, challenges, and communications to evaluation. A classification on MAS applications and challenges is provided along with references for further studies. We expect this article to serve as an insightful and comprehensive resource on MAS for researchers and practitioners in the area.
This paper explores the problem of managing movements of aircraft along the surface of busy airports. Airport surface management is a complex logistics problem involving the coordination of humans and machines. The work described here arose from the idea that autonomous towing vehicles for taxiing aircraft could offer a solution to the 'capacity prob-lem' for busy airports, the problem of getting more efficient use of existing surface area to meet increasing demand. Supporting autonomous surface operations requires continuous planning, scheduling and monitoring of operations, as well as systems for optimizing complex human-machine interaction. We identify a set of computational subproblems of the surface management problem that would benefit from recent advances in multi-agent planning and scheduling and prob-abilistic predictive modeling, and discuss preliminary work at integrating these components into a prototype of a surface management system.
Origin-destination (OD) pattern is a highly useful means for transportation research since it summarizes urban dynamics and human mobility. However, existing visual analytics are insufficient for certain OD analytical tasks needed in transport research. For example, transport researchers are interested in path-related movements across congested roads, besides global patterns over the entire domain. Driven by this need, we propose waypoints-constrained OD visual analytics, a new approach for exploring path-related OD patterns in an urban transportation network. First, we use hashing-based query to support interactive filtering of trajectories through user-specified waypoints. Second, we elaborate a set of design principles and rules, and derive a novel unified visual representation called the waypoints-constrained OD view by carefully considering the OD flow presentation, the temporal variation, spatial layout and user interaction. Finally, we demonstrate the effectiveness of our interface with two case studies and expert interviews with five transportation experts.
The urban settlement development of the past centuries was characterized by the process of suburbanization. Currently, the process of (re-) urbanization in metropolitan areas leads to population growth. This increase results in even more traffic participants in highly condensed areas and thus challenging the urban mobility system. Capacity and frequency of service of public transportation, a city's layout and road capacities constrain urban traffic. Transitions in travel behavior in Germany (e.g. less young adults consider cars as status symbols, and thus car ownership decreases) as well as the introduction of new types of mobility, such as sharing systems and electric mobility exacerbate the challenges for the future urban mobility. Both an expansion and modernization of the transportation system and an intelligent shift of traffic streams will help to overcome those challenges.
Traffic simulation software is naturally used to derive results about public transportation in metropolitan areas. With the help of agent-based modeling, however, human behavior in a certain research area can be modeled according to pre-defined rules and variables. Not every single inhabitant of the metropolitan area is represented by traffic simulation software, whereas in agent-based modeling all inhabitants will be featured in the model. Agent-based modeling is used for scenarios of the future traffic behavior and the ability to easily adapt to new constraints in the general framework of the model. The metropolitan area of Stuttgart is used as case study and MATSim is used as agent modeling software. Following the model setup, different scenarios are evaluated. The main research question focuses on the alteration of the urban mobility system when newly-built residential or industrial areas or rehabilitated areas are connected to the transport system. First results regarding the demographic development of Stuttgart are presented as well as the current status of the model itself.
Visualizing trajectory attribute data is challenging because it involves showing the trajectories in their spatio-temporal context as well as the attribute values associated with the individual points of trajectories. Previous work on trajectory visualization addresses selected aspects of this problem, but not all of them. We present a novel approach to visualizing trajectory attribute data. Our solution covers space, time, and attribute values. Based on an analysis of relevant visualization tasks, we designed the visualization solution around the principle of stacking trajectory bands. The core of our approach is a hybrid 2D/3D display. A 2D map serves as a reference for the spatial context, and the trajectories are visualized as stacked 3D trajectory bands along which attribute values are encoded by color. Time is integrated through appropriate ordering of bands and through a dynamic query mechanism that feeds temporally aggregated information to a circular time display. An additional 2D time graph shows temporal information in full detail by stacking 2D trajectory bands. Our solution is equipped with analytical and interactive mechanisms for selecting and ordering of trajectories, and adjusting the color mapping, as well as coordinated highlighting and dedicated 3D navigation. We demonstrate the usefulness of our novel visualization by three examples related to radiation surveillance, traffic analysis, and maritime navigation. User feedback obtained in a small experiment indicates that our hybrid 2D/3D solution can be operated quite well.
One of the most difficult tasks in software development is
understanding the behavior of the final product. Making
sure that a system behaves as users expect is a challenging
endeavor. Understanding the behavior of a multi-agent system
is even more challenging given the additional complexities
of multi-agent problems. In this paper, we address the
problem of users creating visualizations to debug and understand
complex multi-agent systems. We introduce VizScript,
a generic framework that expedites the process of creating
such visualizations. VizScript combines a generic application
instrumentation, a knowledge-base, and simple scene
definitions primitives with a reasoning system, to produce
an easy to use visualization platform. Using VizScript, we
were able to recreate the visualizations for a complex multiagent
system with an order-of-magnitude less effort than was
required in a Java implementation
In this paper, we present an interactive visual analytics system, Triple Perspective Visual Trajectory Analytics (TripVista), for exploring and analyzing complex traffic trajectory data. The users are equipped with a carefully designed interface to inspect data interactively from three perspectives (spatial, temporal and multidimensional views). While most previous works, in both visualization and transportation research, focused on the macro aspects of traffic flows, we develop visualization methods to investigate and analyze microscopic traffic patterns and abnormal behaviors. In the spatial view of our system, traffic trajectories with various presentation styles are directly interactive with user brushing, together with convenient pattern exploration and selection through ring-style sliders. Improved ThemeRiver, embedded with glyphs indicating directional information, and multiple scatterplots with time as horizontal axes illustrate temporal information of the traffic flows. Our system also harnesses the power of parallel coordinates to visualize the multi-dimensional aspects of the traffic trajectory data. The above three view components are linked closely and interactively
Applications dealing with railway traffic management have been
usually modeled adopting classical technologies such as Operations
Research and Constraint Programming. These technologies are suitable to
model static situations where the information is complete, but they lack
to cope with the dynamics and uncertainty of freight trains traffic
management. The paper presents a new approach to the problem based on
the Multi-Agent System technology CaseLP, a logic programming based
environment for MAS prototyping, has been adopted to face a real
case-study: the management of freight trains traffic along the railway
line between the Italian stations of Milano and La Spezia. The research,
conducted within the framework of the EuROPE-TRIS Project, has
successfully demonstrated the advantages of the MAS approach to this
field of application
We consider the scheduling issues of the London Underground (LU), which spans the entire city and is a vast network of inter-related railway lines. By its very nature, it is a dynamic, complex and unpredictable environment operating or being maintained 24 hours-a-day. This paper is a report on an investigation into how a Multi-agent system (MAS) may be used for resolving scheduling issues of LU. It is a previously-unexplored domain. A prototype system, MASLU is developed through the use of Multi-agent system technology, in an innovative and unique manner, with a view of resolving the London Underground’s scheduling/logistics issues in real time.
Many cities and countries are now striving to create intelligent transportation systems that utilize the current abundance of multisource and multiform data related to the functionality and the use of transportation infrastructure to better support human mobility, interests, and lifestyles. Such intelligent transportation systems aim to provide novel services that can enable transportation consumers and managers to be better informed and make safer and more efficient use of the infrastructure. However, the transportation domain is characterized by both complex data and complex problems, which calls for visual analytics approaches. The science of visual analytics is continuing to develop principles, methods, and tools to enable synergistic work between humans and computers through interactive visual interfaces. Such interfaces support the unique capabilities of humans (such as the flexible application of prior knowledge and experiences, creative thinking, and insight) and couple these abilities with machines' computational strengths, enabling the generation of new knowledge from large and complex data. In this paper, we describe recent developments in visual analytics that are related to the study of movement and transportation systems and discuss how visual analytics can enable and improve the intelligent transportation systems of the future. We provide a survey of literature from the visual analytics domain and organize the survey with respect to the different types of transportation data, movement and its relationship to infrastructure and behavior, and modeling and planning. We conclude with lessons learned and future directions, including social transportation, recommender systems, and policy implications.
Origin-destination (OD) movement data describe moves or trips between spatial locations by specifying the origins, destinations, start, and end times, but not the routes travelled. For studying the spatio-temporal patterns and trends of mass mobility, individual OD moves of many people are aggregated into flows (collective moves) by time intervals. Time-variant flow data pose two difficult challenges for visualization and analysis. First, flows may connect arbitrary locations (not only neighbors), thus making a graph with numerous edge intersections, which is hard to visualize in a comprehensible way. Even a single spatial situation consisting of flows in one time step is hard to explore. The second challenge is the need to analyze long time series consisting of numerous spatial situations. We present an approach facilitating exploration of long-term flow data by means of spatial and temporal abstraction. It involves a special way of data aggregation, which allows representing spatial situations by diagram maps instead of flow maps, thus reducing the intersections and occlusions pertaining to flow maps. The aggregated data are used for clustering of time intervals by similarity of the spatial situations. Temporal and spatial displays of the clustering results facilitate the discovery of periodic patterns and longer-term trends in the mass mobility behavior.
Visual analytics plays a key role in the era of connected industry (or industry 4.0, industrial internet) as modern machines and assembly lines generate large amounts of data and effective visual exploration techniques are needed for troubleshooting, process optimization, and decision making. However, developing effective visual analytics solutions for this application domain is a challenging task due to the sheer volume and the complexity of the data collected in the manufacturing processes. We report the design and implementation of a comprehensive visual analytics system, ViDX. It supports both real-time tracking of assembly line performance and historical data exploration to identify inefficiencies, locate anomalies, and form hypotheses about their causes and effects. The system is designed based on a set of requirements gathered through discussions with the managers and operators from manufacturing sites. It features interlinked views displaying data at different levels of detail. In particular, we apply and extend the Marey's graph by introducing a time-aware outlier-preserving visual aggregation technique to support effective troubleshooting in manufacturing processes. We also introduce two novel interaction techniques, namely the quantiles brush and samples brush, for the users to interactively steer the outlier detection algorithms. We evaluate the system with example use cases and an in-depth user interview, both conducted together with the managers and operators from manufacturing plants. The result demonstrates its effectiveness and reports a successful pilot application of visual analytics for manufacturing in smart factories.
Public transportation schedules are designed by agencies to optimize service quality under multiple constraints. However, real service usually deviates from the plan. Therefore, transportation analysts need to identify, compare and explain both eventual and systemic performance issues that must be addressed so that better timetables can be created. The purely statistical tools commonly used by analysts pose many difficulties due to the large number of attributes at trip- and station-level for planned and real service. Also challenging is the need for models at multiple scales to search for patterns at different times and stations, since analysts do not know exactly where or when relevant patterns might emerge and need to compute statistical summaries for multiple attributes at different granularities. To aid in this analysis, we worked in close collaboration with a transportation expert to design TR-EX, a visual exploration tool developed to identify, inspect and compare spatio-temporal patterns for planned and real transportation service. TR-EX combines two new visual encodings inspired by Marey's Train Schedule: Trips Explorer for trip-level analysis of frequency, deviation and speed; and Stops Explorer for station-level study of delay, wait time, reliability and performance deficiencies such as bunching. To tackle overplotting and to provide a robust representation for a large numbers of trips and stops at multiple scales, the system supports variable kernel bandwidths to achieve the level of detail required by users for different tasks. We justify our design decisions based on specific analysis needs of transportation analysts. We provide anecdotal evidence of the efficacy of TR-EX through a series of case studies that explore NYC subway service, which illustrate how TR-EX can be used to confirm hypotheses and derive new insights through visual exploration.
Learning more about people mobility is an important task for official decision makers and urban planners. Mobility data sets characterize the variation of the presence of people in different places over time as well as movements (or flows) of people between the places. The analysis of mobility data is challenging due to the need to analyze and compare spatial situations (i.e., presence and flows of people at certain time moments) and to gain an understanding of the spatio-temporal changes (variations of situations over time). Traditional flow visualizations usually fail due to massive clutter. Modern approaches offer limited support for investigating the complex variation of the movements over longer time periods.
We propose a visual analytics methodology that solves these issues by combined spatial and temporal simplifications. We have developed a graph-based method, called MobilityGraphs, which reveals movement patterns that were occluded in flow maps. Our method enables the visual representation of the spatio-temporal variation of movements for long time series of spatial situations originally containing a large number of intersecting flows. The interactive system supports data exploration from various perspectives and at various levels of detail by interactive setting of clustering parameters. The feasibility our approach was tested on aggregated mobility data derived from a set of geolocated Twitter posts within the Greater London city area and mobile phone call data records in Abidjan, Ivory Coast. We could show that MobilityGraphs support the identification of regular daily and weekly movement patterns of resident population.
Network data is ubiquitous; e-mail traffic between persons, telecommunication, transport and financial networks are some examples. Often these networks are large and multivariate, besides the topological structure of the network, multivariate data on the nodes and links is available. Currently, exploration and analysis methods are focused on a single aspect; the network topology or the multivariate data. In addition, tools and techniques are highly domain specific and require expert knowledge. We focus on the non-expert user and propose a novel solution for multivariate network exploration and analysis that tightly couples structural and multivariate analysis. In short, we go from Detail to Overview via Selections and Aggregations (DOSA): users are enabled to gain insights through the creation of selections of interest (manually or automatically), and producing high-level, infographic-style overviews simultaneously. Finally, we present example explorations on real-world datasets that demonstrate the effectiveness of our method for the exploration and understanding of multivariate networks where presentation of findings comes for free.
There are a number of emergent traffic and transportation phenomena that cannot be analyzed successfully and explained using analytical models. The only way to analyze such phenomena is through the development of models that can simulate behavior of every agent. Agent-based modeling is an approach based on the idea that a system is composed of decentralized individual ‘agents’ and that each agent interacts with other agents according to localized knowledge. The agent-based approach is a ‘bottom-up’ approach to modeling where special kinds of artificial agents are created by analogy with social insects. Social insects (including bees, wasps, ants and termites) have lived on Earth for millions of years. Their behavior in nature is primarily characterized by autonomy, distributed functioning and self-organizing capacities. Social insect colonies teach us that very simple individual organisms can form systems capable of performing highly complex tasks by dynamically interacting with each other. On the other hand, a large number of traditional engineering models and algorithms are based on control and centralization. In this article, we try to obtain the answer to the following question: Can we use some principles of natural swarm intelligence in the development of artificial systems aimed at solving complex problems in traffic and transportation?
Network-centric operations, in which both automated and human entities are linked in order to leverage information superiority, will bring increases in available information sources, volume of information and operational tempo, placing higher cognitive demands on operators. In the future vision of allowing a single operator to control multiple unmanned vehicles (which could be on land, in the air, or under water), it is not well understood how operators will manage multiple vehicles, what kind of decision support can compliment operators, and how human cognitive limitations will impact overall system effectiveness. To this end, this paper presents the results of an experiment in which an operator simultaneously managed four highly autonomous UAVs executing an air tasking order, with the overall goal of destroying a predetermined set of targets within a limited time period. The primary factors under investigation were different levels of automation from manual to management-by-exception represented in a timeline, as well as different levels of replanning which are needed in time-sensitive targeting scenarios. Increasing levels of automation can reduce workload but they can also result in situation awareness degradation as well as automation bias. Results demonstrate that operators became fixated on the need to globally optimize their schedules and did not adequately weigh uncertainty in their decisions. This fixation significantly degraded operator performance to the point that operators without any decision support performed better than those with probabilistic prediction information and the ability to negotiate potential outcomes.
Flatland-RL: Multi-agent reinforcement learning on trains
- mohanty
Neural MMO: A massively multiagent game environment for training and evaluating intelligent agents
- suarez
Supplementary material: Spatio-temporal analysis of multi-agent scheduling behaviors on fixed-track networks
- S Agarwal
- G Wallner
- J Watson
- F Beck
Neural MMO: A massively multiagent game environment for training and evaluating intelligent agents
- J Suarez
- Y Du
- P Isola
- I Mordatch