Content uploaded by Franziska Henze
Author content
All content in this area was uploaded by Franziska Henze on Oct 19, 2020
Content may be subject to copyright.
Sensitivity Analysis of a Planning Algorithm Considering Uncertainties*
Franziska Henze1, Dennis Faßbender2and Christoph Stiller1
Abstract— Recent trajectory or maneuver planning ap-
proaches in automated driving show the tendency to get more
complex or even become a black box. Thus, an algorithm‘s
decisions become less transparent, especially when uncertain
input parameters have a large influence. To identify those
input parameters whose uncertainty is more relevant than
others’, Morris’ method of elementary effects is used here [1].
It is a quantitative sensitivity analysis that classifies the inputs
into relevant and irrelevant, depending on how sensitive the
algorithm reacts to changes in the input. The method is adapted
to analyze the behavior of a car-following and lane-changing
model during an overtaking maneuver with two vehicles. The
results show that Morris’ method is capable of determining
important parameters for each situation. It is even possible to
identify boundaries for the necessary accuracy of each input.
With this, we are able to determine input parameter ranges
for which the planning algorithm is able to produce reliable
output.
I. INTRODUCTION
The process of automated driving can be decomposed into
the stages sense – plan – act [2], [3]. On the one hand the
planning algorithms need to handle uncertain information
such as other vehicles’ state estimates or behavior predictions
and on the other hand they have to produce reliable output
to the acting components, cf. Fig. 1. Thus, it is particularly
important to understand their ability to cope with uncertain-
ties. To this end, we examine the applicability of sensitivity
analysis to a simple planning approach. The aim is to identify
the most influential input parameters for a specific situation.
We apply the method of elementary effects to a slightly
modified Intelligent Driver Model (IDM) [4] combined with
a lane-changing model (Minimizing Overall Braking Induced
by Lane Change, MOBIL) [5]. We propose a method to
automatically quantify the sensitivity of the planning output
to the input. Furthermore, bounds on the input variations are
derived that guarantee stable planning. This information is
crucial for the safe operation of automated vehicles, as a
selected plan should be followed only if the input accuracy
can be guaranteed to stay within those bounds.
The remainder of this paper is structured as follows: In
section II we give a brief overview of currently used planning
approaches and the developments in the field of sensitivity
analysis. We introduce our extensions to the method of
*The German Federal Ministry of Economics and Energy funded this
research within the project @City: Automated Cars and Intelligent Traffic
in the City. This research was also supported by AUDI AG.
1Authors are with Institute for Measurement and Control Systems,
Karlsruhe Institute of Technology (KIT), 76131 Karlsruhe, Germany,
{franziska.henze, stiller}@kit.edu
2Author is with Pre-Development of Automated Driving, AUDI AG,
85045 Ingolstadt, Germany, dennis.fassbender@audi.de
ego cp
Fig. 1. Uncertain input parameters, e.g., uncertain ego position, incomplete
knowledge of other vehicles’ states or uncertain prediction of behavior,
might lead to uncertain decisions: Each slightly different input combina-
tion produces an altered trajectory or maneuver (red lines). The selected
trajectory is marked with red dots.
elementary effects to general planning algorithms in section
III. Afterwards, we shortly explain a simple driver model and
the scenario used throughout the simulation (section IV) and
present the results in section V.
II. RELATED WORK
A. Planning algorithms
In order to plan suitable maneuvers or trajectories on
highways, different methods are applicable. One possibility
is to optimize a cost function including, e.g., dynamical
costs such as deceleration, or risks arising from uncertain
perception [6], [7]. To incorporate uncertainties directly into
the planning process, Partially Observable Markov Decision
Processes are used (POMDPs). Several approaches exist to
solve POMDPs online [8], [9], e.g., with Monte Carlo Tree
Search [10], [11]. An extension is to combine this idea
with reinforcement learning [12], [13]. Another approach is
to use reinforcement learning to determine feasible driving
actions from sensor data [14]. You et al. [15] imitated the
driving style of an expert and derived a reward function using
inverse reinforcement learning. Subsequently, they compared
this function to a reward function posed by reinforcement
learning.
The advantage of using machine learning in planning as
opposed to rule-based algorithms with an elaborate model
formulation is the possibility to react to a variety of scenarios
instead of only those that were pre-defined. The drawback is
that most of the calculations occur in a black box obstructing
insight into the decision making process. Especially deriving
requirements on the accuracy of input parameters is ham-
pered. Sensitivity analysis provides a way of examining the
effect uncertain input parameters have on model output.
B. Sensitivity analysis
Morris [1] proposes a sensitivity method to determine
only a few important input parameters among many. He
introduces a global approach to calculate difference quotients
describing the sensitivity of the model to changes in one
2020 IEEE Intelligent Vehicles Symposium (IV)
October 20-23, 2020. Las Vegas, USA
978-1-7281-6672-8/20/$31.00 ©2020 IEEE 857
input (one-at-a-time approach). The distribution of those so-
called elementary effects (EEs) is used to categorize all input
parameters into relevant or irrelevant. Campolongo et al. [16]
extend this method and study the effect of a group of inputs
instead of only one factor at a time: By means of educated
guesses matching parameters are grouped together such that
not every combination is tested. This way, the computational
effort is reduced significantly, making the method applicable
even to large systems. Both approaches are qualitative, which
means they only distinguish input parameters into more and
less important, but do not rank inputs. To be able to state
such rankings, quantitative methods as the one proposed by
Saltelli et al. [17] are used. Their main goal is to formulate
simpler substitute models rather than gaining information
about the accuracy of input data, thus we use the quantitative
method of elementary effects.
While Sumner et al. [18] extend a qualitative method to
a time-dependent model describing the insulin signaling
pathway, Specka et al. [19] extend the method of elementary
effects to a time-dependent ecological model. They define
time-dependent thresholds to classify all input data and
identify the important parameters at each time.
In automated driving, sensitivity analysis has so far mostly
been used to calibrate parameters in models. Fraikin et al.
[20] use Morris’ method for the calibration of automated
driving functions. With this approach they are able to narrow
the important factors and reduce the search space for a
subsequent optimization procedure. Ge et al. [21] develop an
improved sampling strategy to be able to apply the method
of elementary effects to a complex microscopic traffic simu-
lation. Ciuffo et al. [22] use a quantitative sensitivity analysis
to study the IDM and calibrate the model-inherent unknown
parameters. In this paper, we analyze the sensitivity of a
modified IDM to uncertain inputs in an overtaking scenario.
With this, we extend the work of Ciuffo et al. by studying the
effect of aleatoric rather than epistemic uncertainty, allowing
for statements on the model’s robustness.
III. METHOD OF ELEMENTARY EFFECTS
With the method of elementary effects we characterize
the expected decision change if a trajectory planning al-
gorithm gets varying input data. For now, we assume a
model f= (fa,fl, . . .)T:Rk→Rmmapping kinputs to
both continuous, e.g., acceleration a, and discrete outputs,
e.g., lane change decision l: either keep lane (l=0) or
change to the left/right (l=1>0or l = −1<0)1. The
algorithm depends only implicitly on time, as its kinputs
might be time-dependent. In Fig. 2 an exemplary planned
trajectory without any uncertain parameter is displayed by
the red line with dots mimicking the discrete decisions at
times tn∈R+,n∈N. At each time tnthe ego has an
exact representation of the situation qtn∈Rk(cf. Fig. 2
red square with dot). Two steps are necessary to calculate
1Notation: All non-italic mathematical symbols refer to quantities with
units, all italic symbols refer to quantities without. Bold symbols correspond
to vector-valued quantities.
the EEs for every output ◦ ∈ {a,l, . . .}: Using a Monte-
Carlo-like approach we first generate random samples to
represent the ego’s uncertain situation description. To this
end, we sample MGaussian white noise vectors {˜qtn,j}M
j=1,
˜qtn,j = (˜q1,tn,j ,...,˜qk,tn,j )T∈Rk, i.e. vectors with
pairwise uncorrelated components, each having mean µi= 0
and standard deviation σi>0(∀i≤k). With this, we
shift each representation qi,tnof input i≤kto a noisy
sample qi,tn+ ˜qi,tn,j (cf. Fig. 2, violet squares). As we
do this Mtimes, we obtain Mof the ego’s possible rep-
resentations of the situation. Secondly, we determine now
the EEs of each input sample qi,tn+ ˜qi,tn,j separately. An
EE d◦
i,n ∈ {da
i,n,dl
i,n, . . .}of input ion the calculation of
output ◦∈{a,l, . . .}at time tnis the directional difference
quotient
d◦
i,n(˜qtn,j ) := f◦(qtn+˜qtn,j + ∆iei)−f◦(qtn+˜qtn,j)
∆i
,
(1)
with unit vector ei= (0,...,0,1,0,...,0)T∈Rkand
offset ∆i= 0.5¯qip/(p−1) >0for some p∈N. The
constants ¯qi∈R+account for different scales, since, e.g.,
a larger variation in longitudinal velocity estimation might
be reasonable, while in lateral direction a smaller offset
is realistic. In case, e.g., parameter irepresents the ego’s
velocity v, the EE describes how much the decision changes
at time tnif we consider a velocity that is increased by ∆i
(cf. Fig. 2, orange filled square).
With (1) we calculate approximations to the directional
derivative along eiat several random points around qtn. To
determine the expected change in decision we estimate the
mean µ◦,i,n and variance σ2
◦,i,n of sample {d◦
i,n(˜qtn,j )}M
j=1
(◦∈{a,l, . . .}). If µ◦,i,n ≈σ2
◦,i,n ≈0holds true, shifting
parameter idoes not bias the calculation of ◦∈{a,l, . . .}at
time tn. For µ◦,i,n 6= 0 and σ2
◦,i,n ≈0, parameter ishows
linear effects for an increase of input iby ∆i: If µ◦,i,n >0,
we expect, e.g., the acceleration to increase (◦= a), or a
decision for a lane change to the left (◦= l). Conversely, if
µ◦,i,n <0, the expected change in acceleration is negative,
ego cp
qtn
˜
qtn,j
qtn+˜
qtn,j
∆iei
qtn+˜
qtn,j + ∆iei
Fig. 2. Exemplary calculation of EEs. Blue: Preceding vehicle cp; Red:
The ego, planned trajectory (line) and exact description of situation qtn
(dots); Violet squares: Mnoisy observations qtn+˜qtn,j of the situation
qtnat time tn(j≤M); Orange filled square: noisy observation with offset
qtn+˜qtn,j + ∆ieito calculate the difference quotient in ei-direction.
858
or a lane change to the right is expectable, respectively. If
additionally σ2
◦,i,n >0, some non-linear effects possibly
including other parameters ˜
i6=iare visible. In this case,
the magnitude of σ2
◦,i,n indicates how much the results are
influenced by the choice of samples {˜qtn,j}M
j=1.
This analysis is performed for every input parameter i≤k,
so eventually we are able to classify all kparameters into
relevant and irrelevant at time tn: For each output ◦two
constants ◦,µ,n, ◦,σ,n >0are defined. A parameter iwhose
uncertainty can be modelled with distribution N(µi, σ2
i)is
said to be irrelevant for the calculation of ◦∈{a,l, . . .}, if
|µ◦,i,n| ≤ ◦,µ,n and σ2
◦,i,n ≤2
◦,σ,n and relevant otherwise.
So far, we only considered some fixed standard deviations
σito generate the Msamples. To determine how uncertain
input imay be for the planning module to produce reliable
outputs we determine the EEs for several values of σiwhile
keeping all other σ˜
i,˜
i6=i, constant: In general, we choose
σ˜
i= 1 for ˜
i6=i(global analysis), but it is also possible
not to sample the other inputs ˜
iat all (σ˜
i= 0 ∀˜
i6=i,
local analysis). In this case, we only study the influence of
changing one input iat a time and make statements about the
linearity of the effects (σ2
◦,i,n ≈0vs. σ2
◦,i,n 0). In both
cases, parameter imay be classified as relevant or irrelevant
depending on the uncertainties’ magnitude: If one can assure
that it is possible to determine parameter iprecisely enough
and there is no influence in this parameter range, we label
it as irrelevant for this accuracy. Selecting the correct range
for each input is crucial, as invalid or unrealistic samples can
distort the analysis’ results [22].
To determine the influence of all inputs at each time tn
the trajectory planning algorithm calculates the next decision
depending on the current non-perturbed state qtn. The model
we use in this paper is presented in the following section.
IV. PROBLEM STATEMENT
We use a simple car-following model combined with a
lane-changing model to study the applicability of Morris’
method of elementary effects to a planning module. The
models describe the behavior of an automated car while
overtaking a slower vehicle ahead.
A. Driver Model
The driver model used to simulate the behavior of all
vehicles in the simulation is composed of the Intelligent
Driver Model (IDM) being responsible for the longitudinal
control and the Minimizing Overall Braking Induced by Lane
Change Model (MOBIL) accounting for lateral movements.
The IDM was proposed by Treiber et al. [4]. It is a car-
following model calculating the acceleration depending on
the preceding vehicle. During the simulation we determine
the acceleration a : R+→Rat time t∈R+as [23]
at= ¯a 2−vt
vdδ
−sd(vt,∆vt)
min (st,sd(vt,∆vt))2!,(2)
with maximum comfortable acceleration ¯a >0, longitudinal
velocity v : R+→R+, desired velocity vd>0, free accel-
eration exponent δ > 0, velocity difference ∆v : R+→R
between the current vehicle and the preceding vehicle and
current distance s : R+→R+to the preceding vehicle. The
desired safety distance sd:R+×R→R+to the preceding
vehicle depends on the current velocity vand the velocity
difference ∆v between the ego and the preceding vehicle cp:
sd(vt,∆vt)=sj+ max 0,vt∆T + vt∆vt
2√¯ab .
Additionally, the jam distance sj>0, a desired time gap
∆T >0and the desired deceleration b>0are considered.
It is worth noting that our formulation is a slight modification
of the model proposed in [4], as for vt= vdthe original
model states
at= ¯a
|{z}
>0
1−vd
vdδ
|{z }
=1
−sd(vd,∆vt)
st2
|{z }
>0
<0
and thus decelerates. Additionally, substituting
min (st,sd(vt,∆vt)) for stlets the vehicle ignore every
preceding vehicle further away than the desired safety
distance. The slightly modified model (2) reduces to
at= 0 m/s2for vt= vdand st= sd(vd,∆vt), i.e. keeps
the desired velocity vd. Another model proposed to account
for this issue is the Improved Intelligent Driver Model
(IIDM) presented in [24]. It was not used here, since the
aim of this work is not to reflect a good car-following model
but rather to study the potential of the method applied
to a simple understandable planning algorithm including
maneuvers like a lane change.
The MOBIL is a lane-changing model introduced by Kesting
et al. [5]. Throughout this paper we consider an asymmetric
lane-changing criterion applied in most European countries
requiring rightmost lane driving. It determines the lane
change decision from left to right or vice versa (to simplify
the notation, the explicit time-dependence is suppressed):
L→R: ˜aeur
c−ac+p(˜ao−ao)>∆ath −∆abias ,
R→L: ˜ac−aeur
c+p(˜an−an)>∆ath + ∆abias ,
with aeur
c:= (min(ac,˜ac),vc>˜vlead >vcrit ,
acotherwise.
All quantities marked with ˜
·refer to the acceleration on
the target lane (after the lane change), while all quantities
without tilde refer to the current lane. Quantities with ·c,·n
or ·orefer to the vehicle under consideration, the following
vehicle in the new (target) or old (current) lane, respectively.
The factor p∈[0,1] weighs the advantages and disadvan-
tages of the other traffic participants, p= 0 leads to egoistic,
p= 1 to cooperative behavior. The velocities ˜vlead,vcrit
refer to the preceding vehicle on the left lane and a constant
speed limit below which overtaking on the right is allowed.
Calculating aeur
creflects the fact that overtaking on the right
is not permitted if there is no traffic jam (˜vlead >vcrit )
and the preceding vehicle on the left lane is slower than
ego (vc>vlead). The threshold ∆ath prevents the vehicle
from changing lanes too frequently and the constant ∆abias
859
encourages lane changing to the right and impedes lane
changing to the left implementing the obligation to ride on
right lanes. Thus, the lane change decision is described by
a discontinuous scalar-valued function l : R+→ {−1,0,1}
defined by
lt=
−1,if L →R at time t,
1,if R →L at time t,
0,otherwise.
(3)
B. Scenario
We consider a two-lane scenario with two vehicles. The
decisions of both vehicles are simulated with the IDM-
MOBIL model, both drive initially on the right lane, the ego
vehicle approaches the slower vehicle cpin front of itself
and overtakes it, cf. Fig. 1. For the parameters in [25], the
IDM-MOBIL maneuver can be divided into five stages, cf.
Fig. 3: At first, we encounter a deceleration phase, where
the ego vehicle reduces its velocity while closing the gap to
the leading vehicle (braking, stage one). Once the distance to
the slower vehicle is reduced sufficiently, ego starts the first
lane change to overtake (lane change 1, stage two). This is
possible as no other vehicle is present. At the same time, the
ego starts applying a gentle acceleration until it reaches its
desired velocity. As soon as the lane change is completed,
the ego enters the overtaking stage. When the overtaking is
finished, the ego executes a second lane-change (lane change
2). The fifth stage starts as soon as ego has returned to the
right lane (free driving).
Ego’s description of the situation is displayed by a set of
time-dependent variables qt= (q1,t,...,qk,t)T∈Rk. It
describes its own state consisting of a position x:R+→R2,
the velocity v:R+→R2and the longitudinal and lateral
accelerations a,alat :R+→Ras well as the observed
longitudinal position, velocity and acceleration of preceding
and following vehicles on the left, same and right lane of
the ego (i.e. k= 24). To simulate the behavior of every
vehicle over a time interval [t0,tend]⊂R+, it is discretized
into equidistant time points t0<t1< . . . < tend and for
each tna new longitudinal acceleration atnand lane change
0 5 10 15 20
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
-1
0
1
Fig. 3. IDM-MOBIL decisions for the ego during scenario: Lane change
(right axis, •) and acceleration profile (left axis): ◦braking, ∗lane change
1 (R →L), 4overtaking, ×lane change 2 (L →R), free driving (the
color gradient reflects the course of time).
decision ltnare calculated as defined in (2)-(3). All other
states are propagated according to the kinematic equations
˙
vt= [at,alat,t]T,˙
xt=vt.
V. RESULTS
To analyze the IDM-MOBIL model in the scenario de-
scribed above we simulate the trajectories of both vehicles
for 0s≤t≤22 sat discrete time points tn:= n∆t with
step size ∆t = 0.1s. To begin with, a local sensitivity analy-
sis with M= 50 samples is performed at each discretization
point for each decision, i.e. acceleration and lane change,
the ego makes. In this scenario, six inputs show influence
on the calculation of the acceleration, while nine parameters
are influential for the lane change decision: Every parameter
which is important for the acceleration is also important
for the lane change decision whenever a lane change is
considered. This is a direct reflection of the fact that the
MOBIL-model decides when to change lanes depending
on the acceleration of ego and chooses the corresponding
acceleration accordingly. Thus, all parameters influencing the
acceleration also affect the lane change decision.
0
0
0.1
20
0.2
-0.2
0.3
10
-0.4 0
(a) Acceleration a.
0 5 10 15 20
-0.4
-0.3
-0.2
-0.1
0
0.1
(b) Acceleration a(σ2
a,v is dismissed).
Fig. 4. Distribution of local EEs of longitudinal ego velocity von
acceleration a.σv= 0.5m/s,σ˜
i= 0 for ˜
i6= v. For explanation of
markers, cf. Fig. 3.
860
0
1
0.2
0.4
20
0.6
0.5
0.8
1
10
00
(a) Lane change decision l.
0 5 10 15 20
0
0.2
0.4
0.6
0.8
(b) Lane change decision l(σ2
l,v is dismissed).
Fig. 5. Distribution of local EEs of longitudinal ego velocity von lane
change decision l.σv= 0.5m/s,σ˜
i= 0 for ˜
i6= v. For explanation of
markers, cf. Fig.3.
In Figs. 4, 5 the local effects of an uncertain higher longi-
tudinal ego velocity vare displayed for σv= 0.5m/s and
σ˜
i= 0 for ˜
i6= v, i.e. we study the results of a local analysis
with respect to input parameter i= v. The ego’s decisions
for both acceleration (cf. Fig. 4) and lane change (cf. Fig. 5)
show non-linear effects during the first stage of deceleration
and during the beginning of the first lane change. Due to
a higher velocity the ego is expected to decelerate stronger,
i.e. have a smaller acceleration, cf. Fig. 4, since it is going
to approach the vehicle ahead faster and thus has to reduce
the velocity quicker to keep a safe distance. Simultaneously
several samples consider an early lane change when the
velocity is higher, cf. Fig. 5. As the decision for the lane
change coincides with the start of the acceleration phase (cf.
Fig. 3), more samples consider an early lane change and
the expected change in acceleration increases. If µl,v ≈1,
many samples execute a lane change if ego was faster. At
this point also the expected acceleration change is the largest.
We observe that σ2
a,v, σ 2
l,v are at their maximum then, too (cf.
Fig. 4 (a), 5(a)). This mirrors the fact that there are non-linear
effects influencing the lane change and acceleration decision
as well. After the peak the means µa, µlstart decreasing, as
many samples ˜qtn,j would change lanes now even without a
higher velocity and thus dl
i,n(˜qtn,j ) = 0 does not contribute
to the expected lane change decision. Even after the first
lane change is executed (transition from stage 1 to stage
2), there exist some samples for which the lane change is
only executed if there is a larger velocity, so µl,v >0for
several other time steps. Without a preceding vehicle the
expected change in acceleration tends to µa,v ≈ −0.188, cf.
Fig. 4 (b). As in the simulations, we consider stn=∞m,
vd= 120/3.6m/s,δ= 4,¯a = 1.5m/s2, cf. [25], and
∆v= 5/9m/s (p= 10,¯qv= 1 m/s) [1]. For the EE of a
sample ˜qv,tn,j = 0 m/s, i.e. vtn= vtn+ 0 = vd, holds
da
v,n(0) = ¯a
∆v 2−vd+ ∆v
vdδ
−sd
min(stn,sd)2
−2 + vd
vdδ
+sd
min(stn,sd)2!
=¯a
∆vv4
dv4
d−(vd+ ∆v)4
≈ −0.1846 s−1.
Since σa∈ O(10−5)is very small, this implies a mainly
linear relation between a change in ego velocity vand
the expected change in acceleration neglecting all higher
order, i.e. non-linear terms. Physically speaking, the larger
the velocity is, the less ego needs to accelerate to reach
the desired velocity, when no preceding vehicle exists. In
general, the expected change in acceleration is quite small
compared to accelerations in real world scenarios. This is
due to the rather moderate choice of maximum desired
acceleration, cf. the overall acceleration profile in Fig. 3.
So far, we only considered the local sensitivity to changing
one input at a time. This might be of theoretical interest,
but is very often infeasible in practice, since it is usually
not possible to fix all inputs but one. So in general, it is
more appropriate to analyze situations in which all inputs
are uncertain. To this end, we perform a global sensitivity
analysis with M= 50 samples, σv= 0.5m/s and σ˜
i= 1
for all other inputs ˜
i6=v, cf. Fig 6. Comparing Figs. 6 (a)
and 4 (a) as well as 6 (b) and 5 (a) we notice that for both
acceleration and lane change the mean and variance roughly
remain unchanged. Thus, there exist only negligible non-
linear effects depending on other input factors, while the
main, partially non-linear effects stem from the change in
the ego’s velocity.
For an uncertain input i= vp,r (longitudinal velocity of the
preceding vehicle on the ego’s right lane) the characteristic
numbers of the distribution describing the global EEs of the
lane change decision are displayed in Fig. 7 for standard de-
viations σvp,r ∈ {1.5m/s, 5m/s}(i.e. σ˜
i= 1 for ˜
i6= vp,r).
For σvp,r = 1.5m/s only very few samples consider a lane
change back to the right lane if the velocity vp,r is larger, cf.
Fig. 7 (a): To abort the overtaking seems reasonable, when
the velocity of the preceding vehicle is large enough for ego
to follow with decent distance and without having to break
861
0
0.05
0
0.1
20
0.15
-0.1
0.2
0.25
-0.2 10
-0.3 0
(a) Acceleration a.
0
0.2
0.6
0.4
20
0.4
0.6
0.8
10
0.2
00
(b) Lane change decision l.
Fig. 6. Distribution of global EEs of longitudinal ego velocity v.σv= 0.5m/s,σ˜
i= 1 for ˜
i6= v. For explanation of markers, cf. Fig. 3.
0
0
0.05
0.1
20
-0.05
0.15
0.2
10
-0.1 0
(a) Lane change decision l,σvp,r = 1.5m/s,σ˜
i= 1 for ˜
i6= vp,r.
0
0
0.1
20
0.2
-0.1
0.3
10
-0.2 0
(b) Lane change decision l,σvp,r = 5 m/s,σ˜
i= 1 for ˜
i6= vp,r.
Fig. 7. Distribution of global EEs of velocity vp,r of preceding vehicle cpon the right lane. For explanation of markers, cf. Fig. 3.
too harshly. We observe µl,vp,r ∈ O(10−2)shortly after ego
enters the left lane. In combination with σ2
l,vp,r ∈ O(10−1)
this implies only a small relevance of the velocity vp,r for
small uncertainties with standard deviation σvp,r = 1.5m/s.
In Fig. 7 (b) the results for σvp,r = 5 m/s show higher
influence: There are more samples considering a lane change
to the right for increasing vp,r. This uncertainty is visible
for a larger time interval even after the first lane change is
completed. As µl,vp,r ∈ O(10−1)also for σvp,r = 5 m/s,
vp,r seems to be a minor factor compared to the influence
of the ego velocity (cf. Fig. 5). Nevertheless, the difference
between σvp,r = 1.5m/s and σvp,r = 5 m/s shows the
potential of the method: Depending on how reliable the
output of the planning method needs to be, statements about
admissible input uncertainties are possible. In this case it
is reasonable to require the measurement of the preceding
vehicle’s velocity to have a better standard deviation than
σvp,r = 5 m/s.
In Fig. 8 the influence of the preceding vehicle’s velocity
on the acceleration is shown. For both σvp,r = 1.5m/s and
σvp,r = 5 m/s mean and variance µa,vp,r , σ2
a,vp,r ∈ O(10−3)
are very small. Thus, an uncertain velocity vp,r seems to be
irrelevant for the calculation of a reliable acceleration at the
beginning of an overtaking maneuver.
VI. CONCLUSIONS
In this paper we propose a sensitivity analysis that can be
applied to any given planning algorithm. It determines the
influence of aleatoric uncertainties and identifies parameters
that are relevant in a specific situation, while others are
classified as irrelevant. Additionally, we formulate some
accuracy requirements for the input data such that the
planning algorithm can still compute reliable decisions. The
value of such statements is twofold: On the one hand, they
indicate if waiting for better information may yield better
output. Additionally, they allow defining clear uncertainty
boundaries, which other modules have to adhere to. On the
other hand, if the prerequisites can’t be met, it is easy to
862
0
2
0.2
0.4
20
10-3
0.6
0
10-3
0.8
1
10
-2
0
(a) Acceleration a,σvp,r = 1.5m/s,σ˜
i= 1 for ˜
i6= vp,r.
0
0.2
5
0.4
20
0.6
10-3
0.8
0
10-3
1
10
-5
0
(b) Acceleration a,σvp,r = 5 m/s,σ˜
i= 1 for ˜
i6= vp,r.
Fig. 8. Distribution of global EEs of velocity vp,r of preceding vehicle cpon the right lane. For explanation of markers, cf. Fig. 3.
derive with respect to which input the algorithm needs to
become more robust to handle the uncertain inputs.
Further research will focus on enhancing this method to study
scenarios where maneuver decisions can change fundamen-
tally (e.g., entering a roundabout before or after another ve-
hicle passed by). Ideally, a better understanding of influential
inputs improves the transparency of difficult decisions.
REFERENCES
[1] M. D. Morris, “Factorial sampling plans for preliminary computational
experiments,” Technometrics, vol. 33, no. 2, pp. 161–174, 1991.
[2] R. Brooks, “A robust layered control system for a mobile robot,” IEEE
Journal on Robotics and Automation, vol. 2, no. 1, pp. 14–23, 1986.
[3] D. Kortenkamp, R. Simmons, and D. Brugall, “Robotic systems
architectures and programming,” in Springer Handbook of Robotics,
B. Siciliano and O. Khatib, Eds. Cham: Springer International
Publishing, 2016, pp. 283–305.
[4] M. Treiber, A. Hennecke, and D. Helbing, “Congested traffic states in
empirical observations and microscopic simulations,” Physical Review
E, vol. 62, no. 2, pp. 1805–1824, 2000.
[5] A. Kesting, M. Treiber, and D. Helbing, “General lane-changing model
mobil for car-following models,” Transportation Research Record:
Journal of the Transportation Research Board, vol. 1999, no. 1, pp.
86–94, 2007.
[6] H. Bey, F. Dierkes, S. Bayerl, A. Lange, D. Fassbender, and
J. Thielecke, “Optimization-based tactical behavior planning for au-
tonomous freeway driving in favor of the traffic flow,” IEEE Intelligent
Vehicles Symposium, pp. 912–919, 2019.
[7] W. Xu, J. Pan, J. Wei, and J. M. Dolan, “Motion planning under
uncertainty for on-road autonomous driving,” IEEE International
Conference of Robotics and Automation, pp. 2507–2512, 2014.
[8] C. Hubmann, J. Schulz, M. Becker, D. Althoff, and C. Stiller, “Au-
tomated driving in uncertain environments: Planning with interaction
and uncertain maneuver prediction,” IEEE Transactions on Intelligent
Vehicles, vol. 3, no. 1, pp. 5–17, 2018.
[9] P. Sch ¨
orner, L. T¨
ottel, J. Doll, and J. M. Z¨
ollner, “Predictive trajectory
planning in situations with hidden road users using partially observable
markov decision processes,” IEEE Intelligent Vehicles Symposium, pp.
2042–2049, 2019.
[10] C. Hubmann, J. Schulz, G. Xu, D. Althoff, and C. Stiller, “A belief
state planner for interactive merge maneuvers in congested traffic,”
IEEE International Conference on Intelligent Transportation Systems,
pp. 1617–1624, 2018.
[11] D. Sierra Gonz ´
alez, M. Garz´
on, J. Dibangoje, and C. Laugier,
“Human-like decision-making for automated driving in highways,”
IEEE International Conference on Intelligent Transportation Systems,
pp. 2087–2094, 2019.
[12] C. Paxton, V. Raman, G. Hager, and M. Kobilarov, “Combining
neural networks and tree search for task and motion planning in
challenging environments,” IEEE/RSJ International Conference on
Intelligent Robots and Systems, pp. 6059–6066, 2017.
[13] C.-J. Hoel, K. Driggs-Campbell, K. Wolff, L. Laine, and M. J.
Kochenderfer, “Combining planning and deep reinforcement learning
in tactical decision making for autonomous driving,” ArXiv, vol.
abs/1905.02680, 2019.
[14] A. El Sallab, M. Abdou, E. Perot, and S. Yogamani, “Deep rein-
forcement learning framework for autonomous driving,” IS&T Inter-
national Symposium on Electronic Imaging, Autonomous Vehicles and
Machines, no. 19, pp. 70–76, 2017.
[15] C. You, J. Lu, D. Filev, and P. Tsiotras, “Advanced planning for
autonomous vehicles using reinforcement learning and deep inverse
reinforcement learning,” Robotics and Autonomous Systems, vol. 114,
pp. 1–18, 2019.
[16] F. Campolongo, J. Cariboni, and A. Saltelli, “An effective screening
design for sensitivity analysis of large models,” Environmental Mod-
elling & Software, vol. 22, no. 10, pp. 1509–1518, 2007.
[17] A. Saltelli, P. Annoni, I. Azzini, F. Campolongo, M. Ratto, and
S. Tarantola, “Variance based sensitivity analysis of model output.
design and estimator for the total sensitivity index,” Computer Physics
Communications, vol. 181, no. 2, pp. 259–270, 2010.
[18] T. Sumner, E. Shephard, and D. Bogle, “A methodology for global-
sensitivity analysis of time-dependent outputs in systems biology
modelling,” Journal of the Royal Society, Interface, vol. 9, no. 74,
pp. 2156–2166, 2012.
[19] X. Specka, C. Nendel, and R. Wieland, “Temporal sensitivity analysis
of the MONICA model: Application of two global approaches to
analyze the dynamics of parameter sensitivity,” Agriculture, vol. 9,
pp. 1–29, 2019.
[20] N. Fraikin, K. Funk, M. Frey, and F. Gauterin, “Dimensionality
reduction and identification of valid parameter bounds for the efficient
calibration of automated driving functions,” Automotive and Engine
Technology, vol. 4, pp. 75–91, 2019.
[21] Q. Ge and M. Menendez, “An efficient sensitivity analysis approach
for computationally expensive microscopic traffic simulation models,”
International Journal of Transportation, vol. 2, no. 2, pp. 49–64, 2014.
[22] V. Punzo, M. Montanino, and B. Ciuffo, “Global sensitivity analysis
techniques to simplify the calibration of traffic simulation models.
methodology and application to the IDM car-following model,” IET
Intelligent Transport Systems, vol. 8, no. 5, pp. 479–489, 2014.
[23] H. Bey, “Modified IDM-model: personal communication,” 11.11.2019.
[24] M. Treiber and A. Kesting, Traffic Flow Dynamics. Data, Models and
Simulation, 1st ed. Berlin, Heidelberg: Springer, 2013.
[25] A. Kesting, M. Treiber, and D. Helbing, “Enhanced intelligent driver
model to access the impact of driving strategies on traffic capacity,”
Philosophical Transactions of the Royal Society A: Mathematical,
Physical and Engineering Sciences, vol. 368, no. 1928, pp. 4585–
4605, 2010.
863
