A highperformance traffic flow microsimulation for large problems
ABSTRACT Traffic flow microsimulations are interesting for transport planning problems due to their high temporal and spatial resolution. Unfortunately, most of them involve high computational costs making them impractical for running large scale scenarios. In this paper, we present how we extend our previous eventdriven queuebased mircosimulation to run efficiently on parallel computers. Using appropriate load balancing and minimizing communication interfaces, we are able to simulate a test scenario involving 7 million simulated person days on a road net work with 28k links in 87 seconds on 64 CPUs. Furthermore, we add support for signaled intersections that makes the model well suited for application to urban street networks. Finally, we show that our resulting model reproduces a reasonable relation between traffic flow and density similar to fundamental diagrams extracted from real world counts data.
 Citations (11)
 Cited In (0)
 Transportation Research Record. 01/2007; 2003:3540.

 SourceAvailable from: matsim.org
Page 1
A HighPerformance Traffic Flow Microsimulation
for Large Problems
David Charypar
Michael Balmer
Kay W. Axhausen
Working paper 509
Transport and Spatial Planning2008
Page 2
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
Working paper509
A HighPerformance Traffic Flow Microsimulation for Large
Problems
David Charypar
Institute for Transport Plan
ning and Systems
ETH Zurich
CH8093 Zurich
phone: +4144633 35 62
fax: +4144633 10 57
charypar@ivt.baug.ethz.ch
Michael Balmer
Institute for Transport Plan
ning and Systems
ETH Zurich
CH8093 Zurich
phone: +4144633 27 80
fax: +4144633 10 57
balmer@ivt.baug.ethz.ch
Kay W. Axhausen
Institute for Transport Plan
ning and Systems
ETH Zurich
CH8093 Zurich
phone: +4144633 39 43
fax: +4144633 10 57
axhausen@ivt.baug.ethz.ch
2008
Abstract
Traffic flow microsimulations are interesting for transport planning problems due to their high
temporal and spatial resolution. Unfortunately, most of them involve high computational costs
making them impractical for running large scale scenarios. In this paper, we present how we
extend our previous eventdriven queuebased mircosimulation to run efficiently on parallel
computers. Using appropriate load balancing and minimizing communication interfaces, we
are able to simulate a test scenario involving 7 million simulated person days on a road net
work with 28k links in 87 seconds on 64 CPUs. Furthermore, we add support for signaled
intersections that makes the model well suited for application to urban street networks. Finally,
we show that our resulting model reproduces a reasonable relation between traffic flow and
density similar to fundamental diagrams extracted from real world counts data.
Keywords
Traffic flow microsimulation, Event driven simulation, Parallelization, Green time fractions,
Largescale scenario
Preferred citation style
Charypar, D., M. Balmer, and K. W. Axhausen (2008) A HighPerformance Traffic Flow Mi
crosimulation for Large Problems, Working paper, 509, Institute for transport planning and
system (IVT), ETH Zurich, Zurich.
1
Page 3
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
1 Introduction
In transport planning, traffic flow simulation plays an important role as it takes the demand
generated by an earlier process as input and produces derived quantities like traffic densities,
flow volumes, speeds, and travel times as output. These quantities are in turn important for
instance to analyze the state of the traffic system or to enable iterative demand generation
based on one or multiple of these values.
There are several choices when it comes to deciding on the type of flow simulation: macro
scopic aggregated models are available just as mesoscopic and microscopic approaches. For
resolution and accuracy, microsimulation would clearly be the method of choice here. Further
more, using a microscopic point of view, the advantages of advanced techniques in demand
generation based on the individual, like activity based (e.g. Bowman et al., 1999; Vovsha et al.,
2002) or agentbased(e.g. MATSimT, 2008) demand models, can be fully exploited by utiliz
ing the same individuals throughout.
The downside of microsimulation is the computational burden that comes with these methods.
One explanation for this is that data with very high temporal and spatial resolution must be
processed as well as produced. This inevitable property represents a major challenge when try
ing to use such a method for large scale scenarios. Another disadvantage is arguably the added
complexity in the demand generation process for creating a virtual population: zoneresolution
is no longer sufficient since microscopic flow simulations need the individual traffic objects
(i.e. the cars). On the other hand, tedious procedures known from macroscopic modeling, like
zone definition, placement of connector links, and generation of ODmatrices, can be avoided
this way, . Perhaps as a result of these properties, microsimulations are seldom used for large
scale scenarios today. Instead, mostly macroscopic models and sometimes mesoscopic models
are employed.
One way to reduce the computation time of microsimulations is by running them in parallel
on multiCPU computers and large computer clusters. Unfortunately, it turns out that this
approach cannot be generally applied to each microsimulation model, as often the resulting
parallel performance of such approaches is poor. With the advance of multicore CPUs in
desktop computers this fact is currently leading to the paradoxical situation that increases in
processing power often do not accelerate traffic flow microsimulations.
Inthiswork, wepresentamicrosimulationprogramthatrealizesdecentsingleCPUandparallel
performance making it suitable for very large scale scenarios. This speed is achieved by bearing
in mind implementation and parallelization issues already during model design. However,
looking at performance only is certainly not sufficient for traffic analysis. We need to make
sure that the model also shows reasonable flow properties. Analysis of such properties therefore
forms an important part of this work. The third focus of this paper is the functional extension
2
Page 4
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
of introducing traffic lights into the model. This extension allows the transport modeler to cope
with the intrinsic heterogeneity of simulating large scale scenarios with rural as well as urban
areas.
3
Page 5
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
2 Traffic Flow Simulation Approaches and Related Work
In this section, we give a brief overview of different classes of traffic flow simulation ap
proaches. Their level of abstraction has strong influence on the resulting accuracy, resolution
and simulation speed.
2.1 Physical Microsimulation Models
Physical microsimulation models represent the most accurate and expensive type. Their aim
is generally to mimic real traffic dynamics by imitating human car following, simulating lane
changing behavior, employing sophisticated intersection dynamics, etc. Space is often repre
sented continuously and simulated time is advanced in small timesteps. Examples are AIM
SUN (Barceló et al., 1998; AIMSUN, 2006), MITSSIM (Yang, 1997; MITSIM, 2006), and
VISSIM (VISSIM, 2006). Barceló et al. (1998) demonstrated a parallel implementation using
globally accessible data which ran 3.5 times faster on 8 processors.
For large scenarios however, physical simulation methods are still too expensive; to become
faster, one has to “trade in” some of the accuracy and resolution for speed.
2.2 Cellular Automata
In cellular automata (CAs, see e.g. Nagel and Schreckenberg, 1992; Chowdhury et al., 2000;
Brilon and Wu, 1998), roads are discretized in cells each of which can be either empty or
occupied by a car. Each car controlls its driving speed based on the available head space to
the front of it. CAs are faster than physical models while retaining a fair amount of spatial
resolution.
A CA is adopted for example in TRANSIMS (Nagel et al., 1998; TRANSIMS, 2006). Nagel
and Rickert (2001) and Rickert and Nagel (2001) showed a parallel version using message
passing between processors, running midsized scenarios on 32 CPUs. Although the running
times were about ten times faster than real time, ethernet latency problems and speed handicaps
due to the use of cellular automata were reported.
What limits CAs’ speed is the necessary update of each car’s position in every timestep. This
becomes impractical for large numbers (>1 million) of cars.
4
Page 6
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
2.3QueueBased Simulations
The model can be further simplified and thereby accelerated by representing links as queues.
This approach was followed in MATSimT (e.g. Cetin, 2005; MATSimT, 2006). The com
putational advantage comes from the shift from the cars being the main simulated elements to
links that “process” the cars. This reduces the number of simulated units (links vs. cars) and
thus simulation performance increases by a factor of 10 to 100, depending on network reso
lution. The links collaborate to move the cars through the network while adhering to various
constraints (capacity, free speed travel time, intersection precedence, and space availability).
Using message passing between cluster nodes, the queuebased model presented in (Cetin,
2005; Cetin et al., 2003) achieved a speedup of 32 using 64 CPUs when simulating a peak
period. Charypar et al. (2007) presented a single processor queuebased simulation based on
Cetin (2005), that replaces the constant timesteps by an eventdriven approach improving effi
ciency by more than a factor of 10. The idea of advancing time as necessary from event to event
can be found in Mahut (2000) as well, where the entry and exit times of vehicles are computed
directly.
2.4 Mesoscopic Models
In contrast to microsimulation, mesoscopic models (e.g. METROPOLIS (Marchal, 2001;
de Palma and Marchal, 2002), DynaMIT (BenAkiva et al., 1998; DynaMIT, 2006), DYNAS
MART (Chang et al., 1994; DYNASMART, 2006), DYNEMO (Schwerdtfeger, 1984; Nökel
and Schmidt, 2002), ORIENT/RV (Axhausen, 1988)) use aggregates to compute travel times
and speeds.
METROPOLISisabletosimulate largescenariosefficientlybyusingaparallelimplementation
based on threads. DynaMIT does not parallelize the traffic flow simulation itself but uses task
parallelization i.e. different modules are run in parallel. DYNEMO was run in parallel (Nökel
and Schmidt, 2002) by using a message passing technique on 19 CPUs for simulating small
scenarios. Larger numbers of CPUs were reported to be inefficient.
2.5 Macroscopic Models
The highest level of abstraction can be found in macroscopic models that compute all traf
fic quantities on an aggregated level. One example of a traffic simulation model as a one
dimensional incompressible fluid is NETCELL (Cayford et al., 1997).
5
Page 7
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
3 Traffic Flow Model
This section summarizes the traffic flow model we presented in Charypar et al. (2007), which
forms the base for the extensions presented in the follwing three sections: support for traffic
lights, analysis of the flow properties of the model, and the parallelization of the model to run
efficiently on many CPUs.
3.1EventDriven QueueBased Model
Themainoverallaimduringthedevelopmentofthepresentedmodelwascomputationalspeed.
As mentioned earlier, we are targeting very large scenarios with up to 10 million simulated
agents. It is important to realize that in this range of size we are competing mostly with macro
scopic models and not with high resolution microsimulations of the physical or cellular au
tomata type. For this reason, it was decided to develop a queuebased model to achieve the
maximal possible speed.
At the base of the simulation model there are the road segments which represent the built
infrastructure and a population of agents that represent the people inside cars that use the in
frastructure. Between agents there exist gaps which are represented explicitly:
• The road segments are directed links between a starting and an end point. They have a
length, an amount of available space, a minimum travel time and two flow capacities, for
in and outflow, respectively.
• Each agent has a complete 24hours activity plan and represents a person traveling by
car. The plan consists of an ordered list of activities, corresponding activity start times
and durations, and the route to travel between subsequent activity locations.
• The population of virtual agents together with their activity plans represent the travel
demand.
• In the opposite direction of agents gaps move upstream to produce the necessary delays
between a car leaving a congested link at the end and a new car entering that link at its
starting point.
During the course of the simulation agents and road segments work together by means of mes
sage passing. The basic idea is simple: In order to enter or leave a specific road segment, each
agent needs to ask the segment for approval first. Approval is not given until execution of the
corresponding action will not violate any of the following requirements:
6
Page 8
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
• Agents can only leave the road segment in the same order as they entered.
• The road segment must not hold more agents than a certain maximum.
• There is a minimum gap between two consecutive cars according to inflow capacity.
• Outflow is constrained analogously.
• To enter a road segment, there must be a gap available at the inlet. Gaps travel upstream
at a constant predefined speed.
• An agent cannot leave the road segment before the free speed travel time has elapsed.
If a road segment has available space, an agent’s request is accepted immediately. Only if the
road is full, approvals are delayed, thereby leading to a reduction in travel speed.
One crucial observation with queuebased models in general is that basically all information
about the course of simulation is contained in the entry and exit events. Between subsequent
entry and exit events for a given car, its exact position is unknown. As a result, simulating the
time between such a pair of events with timesteps only increases simulation costs.
Furthermore, on a heavily loaded link, one can even say that one event triggers the next. As
sume for example an agent trying to enter a road segment that is completely filled with cars.
The agent sends an entry request message to the road segment and waits until the next space
becomes available. At some point in time, the leaving event of the front car produces a gap
that travels upstream at constant speed. When it reaches the upstream end, it becomes avail
able for the new car to enter the link. This predetermination between subsequent events can
be exploited for simulation efficiency. It makes it possible to avoid constant timesteps and
instead use variable timesteps to advance time from one event to the next. This has a couple
of advantages:
• Computational events correspond to traffic events. That means, that computation time is
spentproportionallytothetrafficflow. Thisdenotesanadaptiveassignmentofprocessing
resources according to instantaneous traffic flow.
• Since the event processing rate is more or less constant on a given machine it is easy to
predict how long it will take to simulate a given traffic demand.
By using variable timesteps based on events, one can realize an overall speedup factor of at
least 10 over the timestepbased implementation.
To coordinate the necessary variable timesteps, we introduce a centralized clock object. All
agents and road segments use this clock to register timers as necessary for the event handling
7
Page 9
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
Figure 1: The traffic flow protocol: the vertical lines indicate how time runs during the course
of the simulation; the arrows between these lines represent messages sent between the
objects of the simulation; the numbers in circles denote the corresponding protocol
step of the message.
????
???????
???? ????????
??????????
?????????????
??????????????????
??????????
??????????
?????????
??????????????????????????????
???????????????????
?????????????????
??????????
??????????
?????????
????????????
????????????????????????
??????????????????
??????????
??????????
?????????
????????????
?????????????
??????????????????
?
?
!∀
#
∃
%&
?∋
??
??
?
?!
Source: Charypar et al. (2007)
process described above. When such a timer expires, the corresponding object gets informed
and is then able to take the corresponding action. The rules that define these objects’ interac
tions form the socalled flowprotocol, which is illustrated in Figure 1 for the case of an agent
traveling along a line of road segments.
8
Page 10
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
4 Green Time Fractions
The simulation of 24 hours scenarios on urban networks with many signaled intersections often
involves a special problem: The observable link flow capacities on crossing roads do not always
keep a fixed ratio as the signal control parameters may be time of day dependent. This type
of intersections cannot be handled correctly with constant flow capacities. To alleviate this
problem, we introduce green time fractions that modulate the links outflow capacity. These
greentimefractionscanbevaryingovertime, enablingthesimulationofdifferenttrafficcontrol
schemes over the day.
4.1Model Extension
The general idea is to introduce on each link an explicit representation of the outgoing capacity
reduction based on the fraction of time the associated traffic light is switched to green. We
simply assume the average capacity realized on such a link is proportional to the relative green
time. For example on a link with its traffic light switched to green during 30% of the time on
average, we expect to observe an average realized capacity of 30% of the free flow capacity.
Admittedly, there probably is no linear relation between green time fractions and realized ca
pacity. Still we choose this model to make the simulation easy to understand. After all, we can
switch to an interpretation of the data where we replace relative green time by relative useable
green time, rendering our linear model correct.
In our model, the green time fractions per link are not constant but are merely modulated, each
individually, by a piecewise linear function of time. Other models would have been possible
(including piecewise constant modulation functions), but we think it is a beneficial property to
have a continuously changing capacity on the links to avoid undesired shocks in the system.
After all, if desired, very quick changes in capacity can still be realized.
The question might arise why we model averaged green time fractions instead of individual
green phases. There are multiple reasons to mention:
• The necessary data is difficult to obtain and process, especially for very large scenarios
with hundreds or even thousands of traffic lights. These scenarios often cross adminis
trative boundaries resulting in a lot of heterogeneity in the input data.
• The data cannot be easily made up and small errors in these numbers may produce totally
different realized flows on crossing links.
• For similar reasons, it is very difficult to test new policies using individual green phases
compared to the relative ease of adapting the average green time fractions on a road
9
Page 11
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
Figure 2: Plot of example green time fractions for one link.
0
0.2
0.4
0.6
0.8
1
0 5 10 15 20
green time fraction
time [h]
during some time of day.
Note, that if there is, for some reason, the need to simulate individual green phases on certain
intersections, our model is flexible enough to mimic such a behavior by changing the green
times fraction periodically. For example, a link’s green time fraction can be set to alternate from
100% green for 73 seconds to 0% green (i.e. red) for 128 second by defining a corresponding
modulation function in the input data.
This input data is provided as a separate file (the green time fraction file) in a human readable
XML format. In this file, for each link incorporating traffic lights, there is a list of data points,
specifying the green time fraction on that link at a given point in time. Furthermore, the simu
lation assumes that the values repeat periodically according to a period defined in the same file.
In Figure 2 you can see a plot of example green time fractions for one link.
4.2Technical Description
Internally, for each of the road segments holding green time fractions data an instance of a
special modulator object is created. Apart from holding the data, this object is also responsible
for computing the car waiting times on the associated road segment.
The data is stored in a sorted data structure allowing quick access to any data point needed
during computations. Each data point consists of a time stamp and a green time fraction.
10
Page 12
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
When a car wants to leave a signaled road segment, just as before without signalling, it requests
the next available time slot. This value depends on the time when the last car left the link and
on the out flow capacity of the road segment, which is now time dependent. Technically, the
car has to wait until the integral of the time dependent capacity reaches 1[car]. This integral of
the piecewise linear capacity function q(t) is the piecewise quadratic volume function V (t). If
tlis the time when the last car left the road segment, the requested next leaving time tncan be
computed by solving the equation
V (tn) − V (tl) = 1
(1)
for tn. This involves inverting the volume function which is not trivial, since it is a piecewise
quadratic function, and it is not clear which piece will contain the tnto be computed. Our im
plementation solves this problem by starting at tland integrating each piece of q(t) sequentially
until the integral exceeds the required 1[car]. At this point, it is clear that the last covered piece
contains the tnin question and this can be easily found by solving a quadratic equation.
4.3Demonstration of New Simulation Possibilities
If inclusion of new links into existing road networks is considered, for instance the creation
of a city bypass, accompanying measures are often investigated. Changing the timing and
prioritization of traffic lights is often one of these measures tested and the presented extension
of our traffic flow model provides a way of rendering such ideas in the simulation.
Our model was used at our institute in a recent project where the effects of opening a city
bypass tunnel around the city of Zurich, Switzerland, were investigated (see Balmer et al.,
forthcoming). Thescenarioinvolvedaroadnetworkwith60klinks, usedbyaround670kagents
each of which executing its 24hours activity plan. One of the questions under investigation was
how changes in traffic flow patterns could be influenced by modifying the traffic lights policies
aroundrelevantpointsinthenetwork. Figure3visualizeshowpeakhourtrafficvolumeschange
if traffic flow into and out of the city center is restrained by using traffic lights. The section
shows an area around the freeway exit “Wollishofen” near the east entrance of the tunnel under
investigation. One can see clearly that traffic starts to avoid the restrained road and starts to use
alternative routes.
11
Page 13
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
Figure 3: Effect of changing traffic lights policy near the freeway exit Wollishofen. The num
bers indicate the change in hourly traffic volumes of the peak hour. The circle in the
center indicates where the traffic lights were changed, thickness of the roads indicates
the number of lanes.
10
240
80
30
20
190
560
50
90
210
60
170
360
220
110
130
200
100
30
40
110
120
60
70
100
20
90
200
170
80
130
70
120
±
12
Page 14
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
Figure 4: In our queuebased microsimulation model with gaps the average density and flow
ratio show a clear trapezoidal relation. This property is independent of network reso
lution.
5 Flow Properties
To be able to use our model in a way that it produces the best possible simulation results, it is
important to know its inherent properties. One especially important property is the relation be
tween traffic density and traffic flow and how much this relation depends on external properties
like the network resolution and the travel demand. With this information available, it is pos
sible to tell if and how the model should be applied to certain classes of simulation problems.
Can the model be used for the simulation of freeway traffic? Does the accuracy of the result
change if we split long links into pieces? Such questions are ultimately under investigation in
this section.
5.1Averaged Characteristics
To find the relation between average flow and average density, we perform a virtual experiment
on a circular test network. This circle is loaded accordingly to produce all relevant average
traffic densities. The average traffic flow is then measured and the corresponding values plot
ted against each other. In Figure 4, we show the resulting flowdensity plot. The graph has
a trapezoidal shape, the left, upper, and right edges of which can be clearly associated with
independent parameters of our queuebased model with gaps: The slope of the left edge is con
13
Page 15
A HighPerformance Traffic Flow Microsimulation for Large Problems2008
trolled by the free speed defined on a road segment, the height of the top flat part corresponds
to the flow capacity on the link, and the cutoff point together with the slope of the right edge
are a result of the maximum density (fully congested density) and the backward traveling gap
speed. Interestingly, increasing the network resolution by a factor of 500 does not change the
average flowdensity relation at all. Another interesting observation: all observed states on
the network lie on the border of the graph. No state of medium average density together with
medium average flow was observed in any of our tests. It seems that our model is not only able
to produce the limit values, but it also will produce exactly these limit values if used on a ring
topology.
5.2 Instantaneous Flow Characteristics
We are not only interested in the relation between average flow and average density: We also
want to know, how these values relate locally and on a short time scale. To extract the desired
values, the test setup was slightly modified by introducing a short measurement strip. Sub
sequently, density and flow at any time were measured on this sensor strip and no temporal
smoothing was applied. For this test setup, the demand was left the same as in the previous
case. Interestingly, the resulting instantaneous flowdensity plot looked very similar to the av
erage flowdensity plot. The only noticeable difference was a certain clustering at high an low
densities, so it seems medium densities are rare local observations. Maybe this can be explained
through transitions from congested flow conditions (high density) to free flow conditions (low
density) back again to congestion. This maybe can be interpreted as stopandgo traffic.
5.3 Stochastic Flow characteristics
With the test scenario describe above we have found that our traffic flow model basically re
produces the envelope of what could be termed a stylized fundamental diagram.
Interestingly, the “inner” states, as they are often seen in real wold fundamental diagram are
never observed. To figure out if this is a property of our model we have created a new test
case, basically an isolated short road segment that is fed by two stochastic and independent
processes: On the upstream end, one such process is randomly producing cars trying to enter
the road segment and at the downstream end another one is randomly producing outflow slots
for cars arriving. This setup corresponds to the (admittedly unrealistic) situation where cars
enter and leave a road segment with the highest possible fluctuation.
The resulting flow and density patterns show the same trapezoidal envelope as before, but now,
the inner states are realized regularly.
14
View other sources
Hide other sources
 Available from Kay W. Axhausen · May 15, 2014
 Available from ethz.ch
 Available from ethz.ch