ArticlePDF Available

Abstract

In this paper, we study formal synthesis of control policies for partially observed jump-diffusion systems against complex logic specifications. Given a state estimator, we utilize a discretization-free approach for formal synthesis of control policies by using a notation of control barrier functions without requiring any knowledge of the estimation accuracy. Our goal is to synthesize an offline control policy providing (potentially maximizing) a lower bound on the probability that the trajectories of the partially observed jump-diffusion system satisfy some complex specifications expressed by deterministic finite automata. Finally, we illustrate the effectiveness of the proposed results by synthesizing a policy for a jet engine example.
Synthesis of Partially Observed Jump-Diffusion
Systems via Control Barrier Functions
Niloofar Jahanshahi, Pushpak Jagtap, and Majid Zamani
Abstract—In this paper, we study formal synthesis of control
policies for partially observed jump-diffusion systems against
complex logic specifications. Given a state estimator, we utilize
a discretization-free approach for formal synthesis of control
policies by using a notation of control barrier functions without
requiring any knowledge of the estimation accuracy. Our goal
is to synthesize an offline control policy providing (potentially
maximizing) a lower bound on the probability that the trajecto-
ries of the partially observed jump-diffusion system satisfy some
complex specifications expressed by deterministic finite automata.
Finally, we illustrate the effectiveness of the proposed results by
synthesizing a policy for a jet engine example.
Index Terms—Stochastic control systems, Control barrier func-
tions, Controller synthesis, Output feedback control.
I. INTRODUCTION
RECENT years have witnessed a growing interest in
formal synthesis of controllers for complex systems
against complex logic specifications [1]. These specifications
are usually expressed using temporal logic formulae or as
(in)finite strings over finite automata. Several approaches
based on finite abstraction have been widely used to solve
such synthesis problems. Existing techniques include policy
synthesis enforcing linear temporal logic specifications for
non-stochastic systems [2], [3] and for stochastic ones [4],
[5], [6]. When dealing with large systems, these approaches
suffer severely from the curse of dimensionality (i.e., compu-
tational complexity grows exponentially with the dimension
of the state set). In order to overcome the large computational
burden, a discretization-free approach, based on control barrier
functions has shown potential to solve the formal synthesis
problems (See [7], [8], [9], [10] and references therein). The
aforementioned works assume the availability of complete
state information. However, in many real applications we do
not have access to complete state information. Motivated by
this limitation, the recent result in [11] provides the synthesis
of controllers enforcing invariance properties for stochastic
control systems with incomplete information by assuming a
prior knowledge of the control barrier functions. In our recent
The authors contributed equally to this work.
This work was supported in part by the H2020 ERC Starting Grant
AutoCPS (grant agreement No. 804639), the German Research Foundation
(DFG) through the grants ZA 873/1-1 and the Research Training Group
2428, and the TUM International Graduate School of Science and Engineering
(IGSSE).
N. Jahanshahi is with the Computer Science Department, Ludwig
Maximilian University of Munich, Germany. P. Jagtap is with the
Department of Electrical and Computer Engineering, Technical University
of Munich, Germany. M. Zamani is with the Computer Science
Department, University of Colorado Boulder, USA. M. Zamani is with
the Computer Science Department, Ludwig Maximilian University of
Munich, Germany. Emails: niloofar.jahanshahi@lmu.de,
pushpak.jagtap@tum.de,majid.zamani@colorado.edu.
result [12], we consider the problem of synthesizing controllers
for partially observed stochastic control systems. In particular,
we search for a control barrier function that provides a
controller along with a lower bound on the probability that
the system satisfies invariance specifications over a finite-time
horizon. Similar to [11], this work also assumes the existence
of an estimator with a given probabilistic accuracy. Then we
provide the overall probability threshold using the probability
bound on the estimator accuracy and that of the trajectories of
the estimator satisfying the invariance specifications, obtained
via control barrier functions.
The contributions of this paper in comparison with those of
[11], [12] are twofold. First, we provide an offline controller
synthesis approach enforcing complex logic specifications
expressed by (non)deterministic finite automata for partially
observed jump-diffusion systems. As a special case, those
properties include invariance ones. Second, we provide an
approach for computing lower bound on the probability that
the system satisfies given specifications over a finite-time
horizon without requiring any knowledge of the estimator’s
accuracy. Finally, we demonstrate the effectiveness of the
proposed results on a nonlinear jet engine example.
II. PRELIMINARIES AND PROBLEM DEFINITION
Notations: We denote the set of natural, real, and non-
negative real numbers by N,R, and R+
0, respectively. We use
Rnto denote the n-dimensional Euclidean space and Rn×rto
denote the space of real matrices with nrows and rcolumns.
We denote by eiRnthe vector whose all elements are
zero, except the ith element, which is one. Given a matrix
ARn×n, Tr(A)represents trace of Awhich is the sum
of all diagonal elements of A. The zero matrix in Rn×mis
denoted by 0n×m. Given sets Xand Y, we donate f:XY
an ordinary map from Xto Yand the notation |X|denotes
the cardinality of set X.
A. Partially Observed Jump-Diffusion Systems
Let the triplet (Ω,F,P)denote a probability space with
a sample space , filtration F, and the probability measure
P. The filtration F= (Fs)s0satisfies the usual conditions
of right continuity and completeness [13]. Let (Wks)s0be
¯rk-dimensional F-Brownian motions, k= 1,2. Let (Pks)s0
be a ¯qk-dimensional F-Poisson processes, with k= 1,2.
We assume that the Poisson processes and Brownian motions
are independent of each other. The Poisson process Pks :=
[P1
ks;· · · ;P¯qk
ks ]models ¯qkkinds of events, k= 1,2, whose
occurrences are assumed to be independent of each other. We
consider the partially observed jump-diffusion system (po-
JDS), denoted by S, which is described by the following
stochastic differential equations (SDE)
S:(dξ=f(ξ, υ) d t+g1(ξ) d W1t+r1(ξ) d P1t,
dy=h(ξ) d t+g2(ξ) d W2t+r2(ξ) d P2t,(II.1)
where ξ(t)XRnis the value of solution process ξof
S,υ(t)URmis the input vector, and y(t)Rpis the
output vector representing the noisy partial observation at time
tR+
0P-almost surely (P-a.s.). Functions f:X×URn,
g1:XRnׯr1,g2:XRpׯr2,r1:XRnׯq1,r2:
XRpׯq2, and h:XRpare assumed to be Lipschitz
continuous to ensure existence and uniqueness of the solution
of S[13]. Throughout the paper, we use the notation ξ(t)to
denote the value of the solution process of Sat time tR+
0
under the input signal υstarting from the initial state ξ(0) =
aP-a.s., in which ais a random variable that is measurable
in F0. Here, we assume that the Poisson processes Pi
ks for
any i∈ {1,...,¯qk},k= 1,2, have the rates of λki. In order
to provide the results in this paper, we raise the following
assumption on the existence of the estimator that estimates
the state of the po-JDS (II.1).
Assumption 2.1: The states of the po-JDS Sin (II.1) can be
estimated by a proper estimator ˆ
Srepresented in the form of
an SDE as:
ˆ
S: d ˆ
ξ=f(ˆ
ξ, υ) d t+Kdyh(ˆ
ξ) d t,(II.2)
where KRn×pis the estimator gain.
There are plenty of results in the literature on the computation
of estimator gain Kfor various classes of stochastic systems;
see the results in [14], [11], [15], and [16]. We define the
augmented process [ξ, ˆ
ξ]T, where ξand ˆ
ξare the solution pro-
cesses of Sand ˆ
S, respectively. The corresponding augmented
jump-diffusion system ˜
Scan be defined as:
dξ
dˆ
ξ=f(ξ, υ)
f(ˆ
ξ, υ)+0n×p0n×p
KKh(ξ)
h(ˆ
ξ)dt
+g1(ξ) 0nׯr2
0nׯr1Kg2(ξ)dW1t
dW2t+r1(ξ)
0nׯq1dP1t+0nׯq2
Kr2(ξ)dP2t.
(II.3)
For later use, we provide the definition of the infinitesimal
generator (denoted by operator D) for ˜
Susing Ito’s differen-
tiation [13]. Let B:X×XRbe a twice differentiable
function. The infinitesimal generator of Bassociated with the
system ˜
Sfor all (x, ˆx)X×Xand for all uUis given
by
DB(x, ˆx,u) =xB ∂ˆxB(f(x, u)
fx, u)+0n×p0n×p
KKh(x)
hx))
+1
2Tr(g1(x) 0nׯr2
0nׯr1Kg2(x)g1(x) 0nׯr2
0nׯr1Kg2(x)TxxB ∂xˆxB
ˆxxB ∂ˆxˆxB)
+
¯q1
X
i=1
λ1i(B(x+r1(x)ei,ˆx)B(x, ˆx))
+
¯q2
X
i=1
λ2i(B(x+Kr2(x)ei,ˆx)B(x, ˆx)).(II.4)
The symbols xand x,ˆxin (II.4) represent first and second-
order partial derivatives with respect to x(1st argument) and
ˆx(2nd argument), respectively. Note that we dropped the
arguments of xB,ˆxB,x,xB,x, ˆxB,ˆx,xB, and ˆx,ˆxB
in (II.4) for the sake of simplicity.
Given a po-JDS Sin (II.1), we aim at synthesizing a control
policy that guarantees a potentially tight lower bound on the
probability that system Ssatisfies a complex specification over
a finite time horizon. The class of specifications considered in
this paper are provided in the next subsection.
Remark 2.2: The use of the augmented system ˜
Swill
allow us to provide the main result of the paper without
any correctness requirement on the observer. In particualr, our
augmented system formulation provides the user the flexibility
to design any observer by means of any technique. The
probabilistic distance between the values of state and their
estimator is natively considered in our formulation and one
does not need to quantify this distance a-priori which is needed
in the results proposed in [12], [11].
B. Specifications
In this subsection, we consider the class of specifications
expressed by nondeterministic finite automata (NFA)as de-
fined below.
Definition 2.3: [17] A nondeterministic finite automaton
(NFA)is a tuple A= (Q, Q0,Σ, δ, F ), where Qis a finite
set of states, Q0Qis a set of initial states, Σis a finite set
(a.k.a. alphabet),δ:Q×ΣP(Q)is a transition function,
where P(Q)denotes the power set of Q, and FQis a set
of accepting (or final) states.
NFA Ais called deterministic if the transition function is
defined as δ:Q×ΣQ, and we refer to it as deterministic
finite automata (DFA). Since every NFA can be converted to
its equivalent DFA using the powerset construction [18], in
the rest of the paper, we only deal with DFA. Moreover, it
is well known that the complement of a DFA A, denoted
by Ac, is again a DFA [19]. We use the notation qσ
q0
to denote transition relation (q, σ, q0)δ. A finite word
σ= (σ0, σ1, . . . , σk1)Σkis accepted by DFA Aif there
exists a finite state run q= (q0, q1, . . . , qk)Qk+1 such that
q0Q0,qi
σi
qi+1 for all 0i<kand qkF. The
accepted language of A, denoted by L(A), is the set of all
words accepted by A.
In this work, we consider those specifications given by the
accepting languages of DFA Adefined over a set of atomic
propositions Π, i.e., the alphabet Σ=Π. We should highlight
that all linear temporal logic specifications defined over finite
traces, referred to as LTLF, are recognized by DFA [20].
C. Satisfaction of Specification by po-JDS
A given po-JDS Sin (II.1) is connected to the specification
given by the accepting language of a DFA Adefined over a
set of atomic propositions Π, with the help of a measurable
labeling function L:XΠas described in the next
definition which is similar to [21, Definition 2].
Definition 2.4: For a po-JDS Sas in (II.1) and the la-
beling function L:XΠ, a finite sequence σ(ξ) =
(σ0, σ1, . . . , σk1)Πk,kN, is a finite trace of the
solution process ξover a finite time horizon [0, T )R+
0if
there exists an associated time sequence t0, t1, . . . , tk1such
that t0= 0,tk=T, and for all j∈ {0,1, . . . , k 1},tjR+
0
following conditions hold
tj< tj+1;
ξ(tj)L1(σj);
If σj6=σj+1, then for some t0
j[tj, tj+1],ξ (t)
L1(σj)for all t(tj, t0
j);ξ(t)L1(σj+1 )for all
t(t0
j, tj+1); and either ξ (t0
j)L1(σj)or ξ(t0
j)
L1(σj+1).
Next, we define the probability that the solution process ξof
the po-JDS Sstarting from some initial state ξ(0) = aX0
under control policy υsatisfies the specification given by DFA
A.
Definition 2.5: The finite trace corresponding to the solution
process of a po-JDS Sstarting from aXand under the
control policy υover a finite-time horizon [0, T )R+
0, i.e.
σ(ξ)=(σ0, σ1, . . . , σj, . . . , σk1)Πkas in Definition
2.4, satisfies a specification given by the language of a DFA A,
denoted by σ(ξ)|=A, if there exists j∈ {0, . . . , k1}such
that (σ0, σ1, . . . , σj)∈ L(A). The probability of satisfaction
of the specification given by Ais denoted by P{σ(ξ)|=A}.
Remark 2.6: The set of atomic propositions Π =
{p0, p1, . . . , pM}and the labeling function L:XΠ
provide a measurable partition of the state set X=N
i=1Xi
as Xi:= L1(pi). Without loss of generality, we assume that
Xi6=for any i.
D. Problem Definition
Now, we formally define the main synthesis problem con-
sidered in this work.
Problem 2.7: Given a po-JDS Sas in (II.1), a specification
given by the accepting language of DFA A= (Q, Q0,Π, δ, F )
over a set of atomic propositions Π = {p0, p1, . . . , pM}, a
labeling function L:XΠ, and a real value ϑ(0,1),
compute an offline control policy υ(if existing)such that
P{σ(ξ)|=A} ≥ ϑ, for all aL1(pi)and some i
{0,1, . . . , M }.
Finding a solution to Problem 2.7 (if existing) is difficult
in general. We should highlight that the proposed approach
here is sound in solving the considered synthesis problem.
This means that if the proposed method provides a solution to
a synthesis problem, then we can formally conclude that the
proposed controller renders the given specification with the
corresponding lower bound on the probability of satisfaction.
However, if the method fails to provide any solution, then
there may or may not exist a solution to the original synthesis
problem). Our approach is to compute a policy υtogether
with a lower bound ϑ. Our aim is to find the potentially
largest lower bound, which can be compared with ϑand gives
policy, i.e., a solution for Problem 2.7 if ϑϑ. Instead of
computing a control policy that guarantees the lower bound ϑ,
we compute a policy that guarantees P{σ(ξ)|=Ac} ≤ ¯
ϑ,
for any aL1(pi)and some i∈ {0,1, . . . , M }. Then
for the same control policy the lower bound can be easily
obtained as ϑ= 1 ¯
ϑ. This is done by constructing a DFA Ac
whose language is the complement of the language of DFA
A. To synthesize a controller, we utilize the notion of control
barrier functions defined for augmented jump-diffusion system
˜
Sintroduced in the next section.
III. CON TRO L BARRIER FUNCTIONS
In this section, we provide sufficient conditions using so-
called control barrier functions under which we can provide
the upper bound on the probability that the trajectories of
system Sstarting from any initial state in X0Xreach
X1X. To provide a result giving an upper bound on the
reachability probability for the trajectory of S, we provide
conditions on barrier functions constructed over the augmented
system ˜
S.
Theorem 3.1: Consider a po-JDS Sas in (II.1), its estimator
ˆ
Sas in (II.2), the resulting augmented system ˜
Sas in (II.3) and
sets X0, X1X. Suppose there exists a twice differentiable
function B:X×XR+
0, constants c0and γ[0,1)
such that
(x, ˆx)X0×X0, B(x, ˆx)γ, (III.1)
(x, ˆx)X1×X, B(x, ˆx)1,(III.2)
ˆxX, uU, xX, DB(x, ˆx, u)c. (III.3)
Then the probability that the solution process ξof the system
Sstarts from any initial state aX0and reaches region X1
under the control policy υwithin time horizon [0, T )R+
0
is upper bounded by γ+cT .
Proof: By using (III.1) and the fact that X1×
X(x, ˆx)X×X|B(x, ˆx)1, we have
Pξ(t)X1ˆ
ξˆ (t)Xt[0, T )|a, ˆa
Psup0tTB(ξ(t),ˆ
ξˆ (t)) 1|a, ˆaB(a, ˆa) + cT
γ+cT . The second inequality is obtained by utilizing the result
of [22, Theorem 1]. This implies that the probability of the
augmented trajectory of ˜
Sstaring from any (a, ˆa)X0×X0
and reaching X1×Xis upper bounded by γ+cT .
Now we get Pξ(t)X1ˆ
ξˆ (t)Xt[0, T )|
a, ˆaPξ (t)X1t[0, T )|a+Pˆ
ξˆ (t)Xt
[0, T )|ˆaPξ (t)X1ˆ
ξˆ (t)Xt[0, T )|a, ˆa.
Since, the second and last terms trivially hold with probability
1, one has Pξ(t)X1ˆ
ξˆ (t)Xt[0, T )|a, ˆa
Pξ(t)X1t[0, T )|a. Now, since the right term
of the and (i.e. ) is held for all time, the inequality above
becomes an equality and one gets Pξ(t)X1t[0, T )|
aγ+T c which concludes the proof.
The function Bin Theorem 3.1 satisfying (III.1)-(III.3) is
usually referred to as the control barrier function.
Remark 3.2: Condition (III.3) implicitly associates a sta-
tionary controller u:XUaccording to the existential
quantifier on ufor any ˆxXand is independent of choice of
xX. The stationary control policy υdriving the system is
readily given by υ(t) = u(ˆ
ξ(t)), where ˆ
ξis the solution
process of the estimator.
IV. FORMAL SYN TH ES IS O F CON TROLLERS
To synthesize control policies using control barrier functions
enforcing specifications expressed by DFA A, we first provide
the decomposition of specifications into sequential reachability
tasks which will later be solved using control barrier functions.
A. Decomposition into Sequential Reachability
Consider a DFA Aexpressing the properties of interest for
the system S. Consider DFA Ac= (Q, Q0,Π, δ, F )whose
language is the complement of the language of DFA A. The
sequence q= (q0, q1, . . . , qk)Qk+1,kNis called an
accepting state run if q0Q0,qkF, and there exists a finite
word σ= (σ0, σ1, . . . , σk1)Πksuch that qi
σi
qi+1
for all i∈ {0,1, . . . , k 1}. We denote the finite word
corresponding to accepting state run qby σ(q). We also
indicate the length of qQk+1 by |q|, which is k+ 1. Let
Rbe the set of all finite accepting state runs starting from
q0Q0excluding self-loops, where
R:={q=(q0, q1, . . . , qk)Qk+1 |qkF, qi6=qi+1,i<k}.
Computation of Rcan be done algorithmically by viewing
Acas a directed graph G= (V,E)with vertices V=Qand
edges E V × V such that (q, q 0)∈ E if and only if q06=q
and there exist pΠsuch that qp
q0. For any (q, q0)∈ E,
we donate the atomic proposition associated with the edge
(q, q0)by σ(q , q0). From the construction of the graph, it is
obvious that the finite path in the graph starting from vertices
q0Q0and ending at qFFis an accepting state run q
of Acwithout any self-loop and therefore belongs to R. One
can easily compute Rusing depth first search algorithm [23].
For each pΠ, we define a set Rpas
Rp:= {q= (q0, q1, . . . , qk)∈ R | σ(q0, q1) = p}.(IV.1)
Decomposition into sequential reachability is performed as
follows. For any q= (q0, q1, . . . , qk)∈ RppΠ, we
define Pp(q)as a set of all state runs of length 3,
Pp(q) := {(qi, qi+1, qi+2 )|0ik2}.(IV.2)
Now, we define P(Ac) := SpΠSq∈RpPp(q).
Remark 4.1: Note that Pp(q) = for |q|= 2. In fact, any
accepting state run of length 2specifies a subset of the state
set such that the system satisfies Acwhenever it starts from
that subset. This gives trivial zero probability for satisfying
the specification, thus neglected in the sequel.
For the illustration of the above sets, we kindly refer the
interested reader to Example 1 in [8]. Having Pp(q)in (IV.2)
as the set of state runs of length 3, in this subsection, we
provide a systematic approach to compute a policy together
with a (potentially tight) lower bound on the probability that
the solution process of Ssatisfies the specifications given by
DFA A. Given a DFA Ac, our approach relies on performing
a reachability computation over each element of P(Ac)(i.e.,
SpΠSq∈RpPp(q)), where reachability probability is upper
bounded using control barrier functions along with appropri-
ate choices of control inputs as mentioned in Theorem 3.1.
However, computation of control barrier functions and the
policies for each element ν∈ P(Ac), can cause ambiguity
while utilizing controllers in closed-loop whenever there are
more than one outgoing edges from a state of the automaton.
To resolve this ambiguity, we simply merge such reachability
problems into one reachability problem by replacing the
reachable set X1×Xin Theorem 3.1 with the union of regions
corresponding to the alphabets of all outgoing edges. Thus we
get a common control barrier function and a corresponding
controller. This enables us to partition P(Ac)and put the
elements sharing a common control barrier function and a
corresponding controller in the same partition set. These sets
can be formally defined as
µ(q,q0,∆(q0)) := {(q, q0,q00 )∈ P(Ac)
|q, q0, q 00 Qand q00 ∆(q0)}.
The control barrier function and the controller (as discussed
in Remark 3.2) corresponding to the partition set µ(q,q0,∆(q0))
are denoted by Bµ(q,q0,∆(q0)) (x, ˆx)and uµ(q,q0,∆(q0)) (ˆx), respec-
tively. Thus, for all ν∈ P(Ac), we have
Bν(x, ˆx) = Bµ(q,q0,∆(q0)) (x, ˆx)and uνx) = uµ(q,q 0,∆(q0)) x),
if νµ(q,q0,∆(q0)) .
(IV.3)
B. Control Policy
From the above discussion, one can readily observe that
we have different control policies at different locations of
the automaton which can be interpreted as a switching con-
trol policy. Next, we define the automaton representing the
switching mechanism for control policies. Consider the DFA
Ac= (Q, Q0,Π, δ, F )corresponding to the complement of
DFA Aas discussed in Section IV-A, where ∆(q)denotes
the set of all successor states of qQ. Now, the switching
mechanism is given by a DFA Am= (Qm, Qm0,Πm, δm, Fm),
where Qm:= Qm0∪ {(q, q0,∆(q0)) |q , q0Q\F} ∪ Fmis
the set of states, Qm0:= {(q0,∆(q0)) |q0Q0}is the set
of initial states, Πm= Π,Fm=F, and the transition relation
(qm, σ, q0
m)δmis defined as
for all qm= (q0,∆(q0)) Qm0,
(q0,∆(q0))σ(q0,q00)
(q0,q00 ,∆(q00)), where q0
σ(q0,q00)
q00;
for all qm= (q, q0,∆(q0)) Qm\(Qm0Fm),
(q, q0,∆(q0)) σ(q0,q 00)
(q0, q00,∆(q00 )), such that
q, q0, q 00 Q,q0σ(q0,q00)
q00, and q00 /F; and
(q, q0,∆(q0)) σ(q0,q 00)
q00, such that q, q0, q00 Q,
q0σ(q0,q00)
q00, and q00 F.
The hybrid controller defined over augmented state-space X×
Qmthat is a candidate for solving Problem 2.7 is given by
˜
ux, qm) = uµ(q0
m)x),(qm, L(ˆx), q0
m)δm.(IV.4)
The corresponding hybrid control policy υis given by υ(t) =
˜
u(ˆ
ξ(t), qm). For the illustration of the switching mechanism,
see Example 1 in [8, Section 5]. In the next subsection,
we discuss the computation of bound on the probability of
satisfying the specification under such a policy, which then
can be used for checking if this policy is indeed a solution for
Problem 2.7.
C. Computation of Probability
The next theorem provides an upper bound on the probabil-
ity that the solution process satisfies the specifications given
by A.
Theorem 4.2: For a specification given by the accepting
language of DFA A, let Acbe the DFA corresponding to the
complement of A,Rpbe the set defined in (IV.1), and Pp
be the set of runs of length 3defined in (IV.2). Then the
probability that the solution process of the system Sstarting
from any initial state aL1(p)under the hybrid control
policy υassociated with the hybrid controller (IV.4) satisfies
Acwithin time horizon [0, T )is upper bounded by
P{σ(ξ)|=Ac}≤ X
q∈Rp
Y{(γν+cνT)|ν=(q,q0,q00 )Pp(q)},
(IV.5)
where γν+cνTis the upper bound on the probability that the
solution process of Sstarts from X0:= L1(σ(q, q0)) and
reaches X1:= L1(σ(q0, q00 )) under control policy υwithin
time horizon [0, T )which is computed via Theorem 3.1.
Proof: The proof is similar to that of [8, Theorem 5.2]
and is omitted here due to the lack of space.
Theorem 4.2 enables us to decompose the specification into
a collection of sequential reachabilities, compute bounds on
the reachability probabilities using Theorem 3.1, and then
combine the bounds in a sum-product expression.
Remark 4.3: In case we are unable to find control barrier
functions for some of the elements ν∈ Pp(q)in (IV.5), we
replace the related term (γν+cνT)by the pessimistic bound
1and apply random control input. In order to get a non-trivial
bound in (IV.5), at least one control barrier function must be
found for each q∈ Rp.
Corollary 4.4: Given the result of Theorem 4.2, the proba-
bility that the solution process of Sstarts from any aL1(p)
under control policy υand satisfies specifications given by
DFA Aover time horizon [0, T )R+
0is lower-bounded by
P{σ(ξ)|=A} ≥ 1P{σ(ξ )|=Ac}.
D. Computation of Control Barrier Functions
Proving the existence of a control barrier function and find-
ing one are in general hard problems. However, if functions f,
h,g1,g2,r1, and r2are polynomial with respect to their argu-
ments and partition sets Xi=L1(pi), i ∈ {0,1,2, . . . , M },
are bounded semi-algebraic sets (i.e., they can be represented
by polynomial (in)equalities), one can formulate conditions
in Theorem 3.1 as a sum-of-squares (SOS) optimization
problem. See [8, Section 5.3.1.] for a detailed discussion on
a similar approach. Having an SOS optimization problem,
one can efficiently search for a polynomial control barrier
function Bν(x, ˆx)and controller uνx), for any ν∈ P(A¬ϕ)
as in (IV.3) using SOSTOOLS [24] in conjunction with a
semidefinite programming solver such as SeDuMi [25] while
minimizing constants γνand cν. Having values of γνand
cνfor all ν∈ P(A¬ϕ), one can simply utilize results of
Theorem 4.2 and Corollary 4.4 to compute a lower bound on
the probability of satisfying the given specification. Note that
it may not be possible in advance to obtain a probability bound
that is meaningful, in such cases the order of a control barrier
function needs to increase to achieve the desired probability
bound.
Remark 4.5: Under the assumption that sets X, X0, and X1
in Theorem 3.1 are compact and input set Uis finite, one can
utilize counterexample guided inductive synthesis (CEGIS)
approach to search for barrier control functions for more
general nonlinear functions f, h, g1, g2, r1, and r2in (II.1).
For more detailed discussion on CEGIS approach, we kindly
refer interested readers to the algorithm in [8, Section 5.3.2.].
Computational Complexity: The number of triplets and
hence the number of control barrier functions needed to be
computed are bounded by |Q|3, where |Q|is the number of
states in DFA A. However, this is the worst-case bound and
in practice, the number of control barrier functions is much
smaller. In the case of sum-of-squares optimization approach,
the computational complexity of finding polynomial control
barrier functions depends on both the degree of polynomials
and the number of state variables. One can easily see that
for fixed polynomial degrees, the required computations grow
polynomially with respect to the dimension of the augmented
system. For the CEGIS approach, due to its iterative nature
and lack of guarantee on termination, it is difficult to provide
any analysis on the computational complexity.
V. CAS E STU DY
We consider a nonlinear Moore-Greitzer jet engine model
in no-stall mode [26] as a partially observed jump-diffusion
systems by adding noise and jump terms which is given by:
dξ1= (ξ23
2ξ2
11
2ξ3
1) d t+ 0.2 d W11t+ 0.9 d Pt,
dξ2= (ξ1υ) d t+ 0.06 d W12t,
dy=ξ2dt+ 0.06 d W2t,
where ξ= [ξ1, ξ2]T,ξ1= Φ 1,ξ2= Ψ ψ2,Φis the
mass flow, Ψis the pressure rise, and ψis a constant. Terms
W11t, W12t, and W2tdenote the standard Brownian motions
and Ptdenotes the Poisson process with rate λ= 5. We
consider a compact state set X= [1,3]×[4,4] and regions
of interest X0= [0,1]×[1,1],X1= [1,0.2]×[4,2.5],
X2= [1,3] ×[2,4], and X3=X\(X0X1X2). The set
of atomic propositions is given by Π = {p0, p1, p2, p3}with
labeling function L(xj) = pjfor all xjXj,j∈ {0,1,2,3}.
The objective here is to compute a control policy that provides
a lower bound on the probability that the trajectories of
the system satisfy the specification given by the accepting
language of the DFA Agiven in Figure 1 over finite time-
horizon [0, T = 10). Language of Aentails that if we start in
X0then the system will always stay away from X1or X2.
The corresponding DFA Acaccepting complement of L(A)is
shown in Figure 1. Following Subsection IV-A, we only need
to compute a control barrier function corresponding to triplet
(q0, q1, q2).
Now with an estimator gain in (II.2) as K=
[6.1394,7.8927]T, we use SOSTOOLS and SeDuMi to com-
pute a sum-of-squares polynomial control barrier function
B(x, ˆx)of order 4, sum-of-square polynomials ψ0(x, ˆx),
ψ1(x, ˆx),ψ(x, ˆx)of order 4, with total 1125 coefficients
resulting in a computation time of about 15 minutes. The
corresponding controller of order 2 is obtained as follows:
ux)=0.7321ˆx11.8612ˆx1ˆx21.4356ˆx2.(V.1)
The values of γ= 0.099 and c= 1 ×105are obtained
using bisection method resulting in P{σ(ξ)|=A} ≥ 0.89
Fig. 1. The DFA Arepresenting specification (left) and the DFA Ac
representing complement of A(right).
Fig. 2. A few closed loop trajectories starting from different initial conditions
in X0under controller (V.1).
for all x0L1(p0), as discussed in Subsection IV-D. One
can see that only one controller is enough for enforcing the
specification, thus we do not need any switching mechanism.
Figure 2 shows a few trajectories starting from different initial
conditions under the control policy (V.1).
VI. CONCLUSIONS
In this paper, we proposed a discretization-free approach
for the formal controller synthesis of partially observed jump-
diffusion systems. The proposed method computes a hybrid
control policy together with a lower bound on the probability
of satisfying complex temporal logic specifications given by
the accepting language of DFA Aover a finite-time horizon.
This is achieved by constructing control barrier functions over
an augmented system consisting of both the system and the es-
timator. As a result, the probability bound is computed without
requiring any prior information of estimation accuracy.
REFERENCES
[1] C. Belta, B. Yordanov, and E. A. Gol, Formal methods for discrete-time
dynamical systems. Springer, 2017, vol. 89.
[2] P. Tabuada, Verification and control of hybrid systems: a symbolic
approach. Springer Science & Business Media, 2009.
[3] C. Belta, B. Yordanov, and E. A. Gol, “Discrete-time dynamical
systems,” in Formal Methods for Discrete-Time Dynamical Systems.
Springer, 2017, pp. 111–118.
[4] M. Zamani, P. M. Esfahani, R. Majumdar, A. Abate, and J. Lygeros,
“Symbolic control of stochastic systems via approximately bisimilar
finite abstractions,” IEEE Transactions on Automatic Control, vol. 59,
no. 12, pp. 3135–3150, 2014.
[5] M. Zamani, I. Tkachev, and A. Abate, “Towards scalable synthesis of
stochastic control systems,” Discrete Event Dynamic Systems, vol. 27,
no. 2, pp. 341–369, 2017.
[6] A. Lavaei, S. Soudjani, and M. Zamani, “Compositional (in) finite
abstractions for large-scale interconnected stochastic systems,” IEEE
Transactions on Automatic Control, 2020.
[7] A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control barrier
function based quadratic programs for safety critical systems,” IEEE
Transactions on Automatic Control, vol. 62, no. 8, pp. 3861–3876, 2016.
[8] P. Jagtap, S. Soudjani, and M. Zamani, “Formal synthesis of
stochastic systems via control barrier certificates,” arXiv preprint
arXiv:1905.04585, 2019.
[9] P. Jagtap, A. Swikir, and M. Zamani, “Compositional construction
of control barrier functions for interconnected control systems,” in
Proceedings of the 23rd International Conference on Hybrid Systems:
Computation and Control, 2020, pp. 1–11.
[10] C. Huang, X. Chen, W. Lin, Z. Yang, and X. Li, “Probabilistic safety
verification of stochastic hybrid systems using barrier certificates,” ACM
Transactions on Embedded Computing Systems (TECS), vol. 16, no. 5s,
p. 186, 2017.
[11] A. Clark, “Control barrier functions for complete and incomplete in-
formation stochastic systems,” in 2019 American Control Conference
(ACC). IEEE, 2019, pp. 2928–2935.
[12] N. Jahanshahi, P. Jagtap, and M. Zamani, “Synthesis of stochastic
systems with partial information via control barrier functions,” 21st IFAC
World Congress, 2020.
[13] B. Øksendal and A. Sulem, Applied stochastic control of jump diffusions.
Springer Science & Business Media, 2007.
[14] X. Kai, C. Wei, and L. Liu, “Robust extended kalman filtering for
nonlinear systems with stochastic uncertainties,” IEEE Transactions on
Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 40,
no. 2, pp. 399–405, 2009.
[15] B.-S. Chen, W.-H. Chen, and H.-L. Wu, “Robust h2/hglobal
linearization filter design for nonlinear stochastic systems,” IEEE trans-
actions on circuits and systems I: Regular Papers, vol. 56, no. 7, pp.
1441–1454, 2008.
[16] C.-S. Tseng, “Robust fuzzy filter design for a class of nonlinear
stochastic systems,” IEEE Transactions on Fuzzy Systems, vol. 15, no. 2,
pp. 261–274, 2007.
[17] C. Baier and J.-P. Katoen, Principles of model checking. MIT press,
2008.
[18] F. Bonchi and D. Pous, “Checking nfa equivalence with bisimulations
up to congruence,” ACM SIGPLAN Notices, vol. 48, no. 1, pp. 457–468,
2013.
[19] J. E. Hopcroft, R. Motwani, and J. D. Ullman, “Introduction to automata
theory, languages, and computation,Acm Sigact News, vol. 32, no. 1,
pp. 60–65, 2001.
[20] G. De Giacomo and M. Vardi, “Synthesis for ltl and ldl on finite
traces,” in Twenty-Fourth International Joint Conference on Artificial
Intelligence, 2015.
[21] T. Wongpiromsarn, U. Topcu, and A. Lamperski, “Automata theory
meets barrier certificates: Temporal logic verification of nonlinear sys-
tems,” IEEE Transactions on Automatic Control, vol. 61, no. 11, pp.
3344–3355, 2015.
[22] H. Kushner, “Stochastic stability and control, ser,Mathematics in
Science and Engineering. New York: Academic Press, 1967.
[23] S. J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach,
2nd ed. Pearson Education, 2003.
[24] S. Prajna, A. Papachristodoulou, and P. A. Parrilo, “Introducing sostools:
A general purpose sum of squares programming solver,” in Proceedings
of the 41st IEEE Conference on Decision and Control, 2002., vol. 1.
IEEE, 2002, pp. 741–746.
[25] J. F. Sturm, “Using sedumi 1.02, a matlab toolbox for optimization over
symmetric cones,” Optimization methods and software, vol. 11, no. 1-4,
pp. 625–653, 1999.
[26] M. Krstic and P. V. Kokotovic, “Lean backstepping design for a jet
engine compressor model,” in Proceedings of International Conference
on Control Applications. IEEE, 1995, pp. 1047–1052.
... This work does not require a supermartingale property on the control barrier functions, and in particular it does not require any stability assumption on the model. The results of this article are recently generalized in [JJZ20a] in which no a-priori knowledge about the estimation accuracy is needed. Besides, the class of properties is extended to those expressed by nondeterministic finite automata (NFA), and the dynamics are also generalized to partially-observed jump-diffusion systems. ...
... Synthesis of stochastic systems with partial state information via control barrier functions is proposed in [JJZ20b,JJZ20a], as surveyed in Section 8. ...
Preprint
Stochastic hybrid systems have received significant attentions as a relevant modelling framework describing many systems, from engineering to the life sciences: they enable the study of numerous applications, including transportation networks, biological systems and chemical reaction networks, smart energy and power grids, and beyond. Automated verification and policy synthesis for stochastic hybrid systems can be inherently challenging: this is due to the heterogeneity of their dynamics (presence of continuous and discrete components), the presence of uncertainty, and in some applications the large dimension of state and input sets. Over the past few years, a few hundred articles have investigated these models, and developed diverse and powerful approaches to mitigate difficulties encountered in the analysis and synthesis of such complex stochastic systems. In this survey, we overview the most recent results in the literature and discuss different approaches, including (in)finite abstractions, verification and synthesis for temporal logic specifications, stochastic similarity relations, (control) barrier certificates, compositional techniques, and a selection of results on continuous-time stochastic systems; we finally survey recently developed software tools that implement the discussed approaches. Throughout the manuscript we discuss a few open topics to be considered as potential future research directions: we hope that this survey will guide younger researchers through a comprehensive understanding of the various challenges, tools, and solutions in this enticing and rich scientific area.
... CBFs were initially proposed in [12] and further developed in [11]. Extensions of CBFs to high-degree [6], [18], stochastic [7], Euler-Lagrange [19], [20], uncertain [21], and hybrid [22] systems have been proposed. Minimal CBFs were proposed in [23]. ...
Preprint
Full-text available
Control systems often must satisfy strict safety requirements over an extended operating lifetime. Control Barrier Functions (CBFs) are a promising recent approach to constructing simple and safe control policies. This paper proposes a framework for verifying that a CBF guarantees safety for all time and synthesizing CBFs with verifiable safety in polynomial control systems. Our approach is to show that safety of CBFs is equivalent to the non-existence of solutions to a family of polynomial equations, and then prove that this nonexistence is equivalent to a pair of sum-of-squares constraints via the Positivstellensatz of algebraic geometry. We develop this Positivstellensatz to verify CBFs, as well as generalization to high-degree systems and multiple CBF constraints. We then propose a set of heuristics for CBF synthesis, including a general alternating-descent heuristic, a specialized approach for compact safe regions, and an approach for convex unsafe regions. Our approach is illustrated on two numerical examples.
... Given an estimator with a probabilistic guarantee on the accuracy of estimations, [JJZ20b] studies the controller synthesis problem for partially-observed stochastic systems and proposes a lower bound for the probability of satisfaction of safety specifications over finite-time horizons. A synthesis framework based on control barrier functions for partially-observed jump diffusion systems enforcing complex properties expressed by deterministic finite automata is recently proposed in [JJZ20a] in which a prior knowledge of the estimation accuracy is not required anymore. ...
Preprint
Full-text available
In this paper, we propose a compositional framework for the synthesis of safety controllers for networks of partially-observed discrete-time stochastic control systems (a.k.a. continuous-space POMDPs). Given an estimator, we utilize a discretization-free approach to synthesize controllers ensuring safety specifications over finite-time horizons. The proposed framework is based on a notion of so-called local control barrier functions computed for subsystems in two different ways. In the first scheme, no prior knowledge of estimation accuracy is needed. The second framework utilizes a probability bound on the estimation accuracy using a notion of so called stochastic simulation functions. In both proposed schemes, we drive sufficient small-gain type conditions in order to compositionally construct control barrier functions for interconnected POMDPs using local barrier functions computed for subsystems. Leveraging compositionality results, the constructed control barrier functions enable us to compute lower bounds on the probabilities that the interconnected POMDPs avoid certain unsafe regions in finite-time horizons. We demonstrate the effectiveness of our proposed approaches by applying them to an adaptive cruise control problem.
Article
Control Barrier Functions (CBFs) aim to ensure safety by constraining the control input at each time step so that the system state remains within a desired safe region. This paper presents a framework for CBFs in stochastic systems in the presence of Gaussian process and measurement noise. We first consider the case where the system state is known at each time step, and present reciprocal and zero CBF constructions that guarantee safety with probability 1. We extend our results to high relative degree systems and present both general constructions and the special case of linear dynamics and affine safety constraints. We then develop CBFs for incomplete state information environments, in which the state must be estimated using sensors that are corrupted by Gaussian noise. We prove that our proposed CBF ensures safety with probability 1 when the state estimate is within a given bound of the true state, which can be achieved using an Extended Kalman Filter when the system is linear or the process and measurement noise are sufficiently small. We propose control policies that combine these CBFs with Control Lyapunov Functions in order to jointly ensure safety and stochastic stability. Our results are validated via numerical study on a multi-agent collision avoidance scenario.
Thesis
Full-text available
This dissertation is motivated by the challenges arising in the synthesis of controllers for complex systems enforcing complex specifications (usually expressed as temporal logic formulae or (in)finite strings on automata). This thesis develops several controller synthesis approaches for various complex systems without discretizing state-sets that help us to alleviate the issue of the curse of dimensionality that arises in conventional approaches based on discretizing state-sets.
Preprint
Full-text available
This paper focuses on the controller synthesis for unknown, nonlinear systems while ensuring safety constraints. Our approach consists of two steps, a learning step that uses Gaussian processes and a controller synthesis step that is based on control barrier functions. In the learning step, we use a data-driven approach utilizing Gaussian processes to learn the unknown control affine nonlinear dynamics together with a statistical bound on the accuracy of the learned model. In the second controller synthesis steps, we develop a systematic approach to compute control barrier functions that explicitly take into consideration the uncertainty of the learned model. The control barrier function not only results in a safe controller by construction but also provides a rigorous lower bound on the probability of satisfaction of the safety specification. Finally, we illustrate the effectiveness of the proposed results by synthesizing a safety controller for a jet engine example.
Article
Full-text available
e problem of probabilistic safety veri€cation of stochastic hybrid systems is to check whether the probability that a given system will reach an unsafe region from certain initial states can be bounded by some given probability threshold. Œe paper considers stochastic hybrid systems where the behavior is governed by polynomial equalities and inequalities, as for usual hybrid systems, but the initial states follow some stochastic distributions. It proposes a new barrier certi€cate based method for probabilistic safety veri€cation which guarantees the absolute safety in a in€nite time horizon that is beyond the reach of existing techniques using either statistical model checking or probabilistic reachable set computation. It also gives a novel computational approach, by building and solving a constrained optimization problem coming from veri€cation conditions of barrier certi€cates, to compute the lower bound on safety probabilities which can be compared with the given threshold. Experimental evidence is provided demonstrating the applicability of our approach on several benchmarks.
Article
Full-text available
Formal control synthesis approaches over stochastic systems have received significant attention in the past few years, in view of their ability to provide provably correct controllers for complex logical specifications in an automated fashion. Examples of complex specifications of interest include properties expressed as formulae in linear temporal logic (LTL) or as automata on infinite strings. A general methodology to synthesize controllers for such properties resorts to symbolic abstractions of the given stochastic systems. Symbolic models are discrete abstractions of the given concrete systems with the property that a controller designed on the abstraction can be refined (or implemented) into a controller on the original system. Although the recent development of techniques for the construction of symbolic models has been quite encouraging, the general goal of formal synthesis over stochastic control systems is by no means solved. A fundamental issue with the existing techniques is the known "curse of dimensionality," which is due to the need to discretize state and input sets and that results in an exponential complexity over the number of state and input variables in the concrete system. In this work we propose a novel abstraction technique for incrementally stable stochastic control systems, which does not require state-space discretization but only input set discretization, and that can be potentially more efficient (and thus scalable) than existing approaches. We elucidate the effectiveness of the proposed approach by synthesizing a schedule for the coordination of two traffic lights under some safety and fairness requirements for a road traffic model. Further we argue that this 5-dimensional linear stochastic control system cannot be studied with existing approaches based on state-space discretization due to the very large number of generated discrete states.
Article
This article focuses on synthesizing control policies for discrete-time stochastic control systems together with a lower bound on the probability that the systems satisfy the complex temporal properties. The desired properties of the system are expressed as linear temporal logic specifications over finite traces. In particular, our approach decomposes the given specification into simpler reachability tasks based on its automata representation. We, then, propose the use of so-called control barrier certificate to solve those simpler reachability tasks along with computing the corresponding controllers and probability bounds. Finally, we combine those controllers to obtain a hybrid control policy solving the considered problem. Under some assumptions, we also provide two systematic approaches for uncountable and finite input sets to search for control barrier certificates. We demonstrate the effectiveness of the proposed approach on a room temperature control and lane keeping of a vehicle modeled as a four-dimensional single-track kinematic model. We compare our results with the discretization-based methods in the literature.
Article
This paper is concerned with a compositional approach for constructing both infinite (reduced-order models) and finite abstractions (a.k.a. finite Markov decision processes) of large-scale interconnected discrete-time stochastic control systems. The proposed framework is based on the notion of stochastic simulation functions enabling us to use an abstract system as a substitution of the original one in the controller design process with guaranteed error bounds. In the first part of the paper, we derive sufficient small-gain type conditions for the compositional quantification of the probabilistic distance between the interconnection of stochastic control subsystems and that of their infinite abstractions. We then construct infinite abstractions together with their corresponding stochastic simulation functions for a class of discrete-time nonlinear stochastic control systems. In the second part of the paper, we leverage small-gain type conditions for the compositional construction of finite abstractions. We propose an approach to construct finite Markov decision processes (MDPs) as finite abstractions of the concrete models or their reduced-order versions satisfying
Book
This book bridges fundamental gaps between control theory and formal methods. Although it focuses on discrete-time linear and piecewise affine systems, it also provides general frameworks for abstraction, analysis, and control of more general models. The book is self-contained, and while some mathematical knowledge is necessary, readers are not expected to have a background in formal methods or control theory. It rigorously defines concepts from formal methods, such as transition systems, temporal logics, model checking and synthesis. It then links these to the infinite state dynamical systems through abstractions that are intuitive and only require basic convex-analysis and control-theory terminology, which is provided in the appendix. Several examples and illustrations help readers understand and visualize the concepts introduced throughout the book.
Chapter
In this chapter, we introduce the two classes of discrete-time dynamical systems that we will focus on in the rest of the book: piecewise affine control systems with polytopic parameter uncertainties and switched linear systems. As particular instantiations of the first class, we define autonomous systems, fixed parameter systems, and combinations of the above. We define embeddings of such systems into (infinite) transition systems. This enables formal definitions for their semantics and the use of abstractions to map analysis and control problems for such systems to verification and synthesis problems for finite transition systems.
Article
Safety critical systems involve the tight coupling between potentially conflicting control objectives and safety constraints. As a means of creating a formal framework for controlling systems of this form, and with a view toward automotive applications, this paper develops a methodology that allows safety conditions—expressed as control barrier functions— to be unified with performance objectives—expressed as control Lyapunov functions—in the context of real-time optimizationbased controllers. Safety conditions are specified in terms of forward invariance of a set, and are verified via two novel generalizations of barrier functions; in each case, the existence of a barrier function satisfying Lyapunov-like conditions implies forward invariance of the set, and the relationship between these two classes of barrier functions is characterized. In addition, each of these formulations yields a notion of control barrier function (CBF), providing inequality constraints in the control input that, when satisfied, again imply forward invariance of the set. Through these constructions, CBFs can naturally be unified with control Lyapunov functions (CLFs) in the context of a quadratic program (QP); this allows for the achievement of control objectives (represented by CLFs) subject to conditions on the admissible states of the system (represented by CBFs). The mediation of safety and performance through a QP is demonstrated on adaptive cruise control and lane keeping, two automotive control problems that present both safety and performance considerations coupled with actuator bounds.