Synthesis of Partially Observed Jump-Diffusion Systems via Control Barrier Functions

Abstract

In this paper, we study formal synthesis of control policies for partially observed jump-diffusion systems against complex logic specifications. Given a state estimator, we utilize a discretization-free approach for formal synthesis of control policies by using a notation of control barrier functions without requiring any knowledge of the estimation accuracy. Our goal is to synthesize an offline control policy providing (potentially maximizing) a lower bound on the probability that the trajectories of the partially observed jump-diffusion system satisfy some complex specifications expressed by deterministic finite automata. Finally, we illustrate the effectiveness of the proposed results by synthesizing a policy for a jet engine example.

Full text

Synthesis of Partially Observed Jump-Diffusion
Systems via Control Barrier Functions
Niloofar Jahanshahi†, Pushpak Jagtap†, and Majid Zamani
Abstract—In this paper, we study formal synthesis of control
policies for partially observed jump-diffusion systems against
complex logic specifications. Given a state estimator, we utilize
a discretization-free approach for formal synthesis of control
policies by using a notation of control barrier functions without
requiring any knowledge of the estimation accuracy. Our goal
is to synthesize an offline control policy providing (potentially
maximizing) a lower bound on the probability that the trajecto-
ries of the partially observed jump-diffusion system satisfy some
complex specifications expressed by deterministic finite automata.
Finally, we illustrate the effectiveness of the proposed results by
synthesizing a policy for a jet engine example.
Index Terms—Stochastic control systems, Control barrier func-
tions, Controller synthesis, Output feedback control.
I. INTRODUCTION
RECENT years have witnessed a growing interest in
formal synthesis of controllers for complex systems
against complex logic specifications [1]. These specifications
are usually expressed using temporal logic formulae or as
(in)finite strings over finite automata. Several approaches
based on finite abstraction have been widely used to solve
such synthesis problems. Existing techniques include policy
synthesis enforcing linear temporal logic specifications for
non-stochastic systems [2], [3] and for stochastic ones [4],
[5], [6]. When dealing with large systems, these approaches
suffer severely from the curse of dimensionality (i.e., compu-
tational complexity grows exponentially with the dimension
of the state set). In order to overcome the large computational
burden, a discretization-free approach, based on control barrier
functions has shown potential to solve the formal synthesis
problems (See [7], [8], [9], [10] and references therein). The
aforementioned works assume the availability of complete
state information. However, in many real applications we do
not have access to complete state information. Motivated by
this limitation, the recent result in [11] provides the synthesis
of controllers enforcing invariance properties for stochastic
control systems with incomplete information by assuming a
prior knowledge of the control barrier functions. In our recent
†The authors contributed equally to this work.
This work was supported in part by the H2020 ERC Starting Grant
AutoCPS (grant agreement No. 804639), the German Research Foundation
(DFG) through the grants ZA 873/1-1 and the Research Training Group
2428, and the TUM International Graduate School of Science and Engineering
(IGSSE).
N. Jahanshahi is with the Computer Science Department, Ludwig
Maximilian University of Munich, Germany. P. Jagtap is with the
Department of Electrical and Computer Engineering, Technical University
of Munich, Germany. M. Zamani is with the Computer Science
Department, University of Colorado Boulder, USA. M. Zamani is with
the Computer Science Department, Ludwig Maximilian University of
Munich, Germany. Emails: niloofar.jahanshahi@lmu.de,
pushpak.jagtap@tum.de,majid.zamani@colorado.edu.
result [12], we consider the problem of synthesizing controllers
for partially observed stochastic control systems. In particular,
we search for a control barrier function that provides a
controller along with a lower bound on the probability that
the system satisfies invariance specifications over a finite-time
horizon. Similar to [11], this work also assumes the existence
of an estimator with a given probabilistic accuracy. Then we
provide the overall probability threshold using the probability
bound on the estimator accuracy and that of the trajectories of
the estimator satisfying the invariance specifications, obtained
via control barrier functions.
The contributions of this paper in comparison with those of
[11], [12] are twofold. First, we provide an offline controller
synthesis approach enforcing complex logic specifications
expressed by (non)deterministic finite automata for partially
observed jump-diffusion systems. As a special case, those
properties include invariance ones. Second, we provide an
approach for computing lower bound on the probability that
the system satisfies given specifications over a finite-time
horizon without requiring any knowledge of the estimator’s
accuracy. Finally, we demonstrate the effectiveness of the
proposed results on a nonlinear jet engine example.
II. PRELIMINARIES AND PROBLEM DEFINITION
Notations: We denote the set of natural, real, and non-
negative real numbers by N,R, and R+
0, respectively. We use
Rnto denote the n-dimensional Euclidean space and Rn×rto
denote the space of real matrices with nrows and rcolumns.
We denote by ei∈Rnthe vector whose all elements are
zero, except the ith element, which is one. Given a matrix
A∈Rn×n, Tr(A)represents trace of Awhich is the sum
of all diagonal elements of A. The zero matrix in Rn×mis
denoted by 0n×m. Given sets Xand Y, we donate f:X→Y
an ordinary map from Xto Yand the notation |X|denotes
the cardinality of set X.
A. Partially Observed Jump-Diffusion Systems
Let the triplet (Ω,F,P)denote a probability space with
a sample space Ω, filtration F, and the probability measure
P. The filtration F= (Fs)s≥0satisfies the usual conditions
of right continuity and completeness [13]. Let (Wks)s≥0be
¯rk-dimensional F-Brownian motions, k= 1,2. Let (Pks)s≥0
be a ¯qk-dimensional F-Poisson processes, with k= 1,2.
We assume that the Poisson processes and Brownian motions
are independent of each other. The Poisson process Pks :=
[P1
ks;· · · ;P¯qk
ks ]models ¯qkkinds of events, k= 1,2, whose
occurrences are assumed to be independent of each other. We

consider the partially observed jump-diffusion system (po-
JDS), denoted by S, which is described by the following
stochastic differential equations (SDE)
S:(dξ=f(ξ, υ) d t+g1(ξ) d W1t+r1(ξ) d P1t,
dy=h(ξ) d t+g2(ξ) d W2t+r2(ξ) d P2t,(II.1)
where ξ(t)∈X⊆Rnis the value of solution process ξof
S,υ(t)∈U⊆Rmis the input vector, and y(t)∈Rpis the
output vector representing the noisy partial observation at time
t∈R+
0P-almost surely (P-a.s.). Functions f:X×U→Rn,
g1:X→Rnׯr1,g2:X→Rpׯr2,r1:X→Rnׯq1,r2:
X→Rpׯq2, and h:X→Rpare assumed to be Lipschitz
continuous to ensure existence and uniqueness of the solution
of S[13]. Throughout the paper, we use the notation ξaυ(t)to
denote the value of the solution process of Sat time t∈R+
0
under the input signal υstarting from the initial state ξaυ(0) =
aP-a.s., in which ais a random variable that is measurable
in F0. Here, we assume that the Poisson processes Pi
ks for
any i∈ {1,...,¯qk},k= 1,2, have the rates of λki. In order
to provide the results in this paper, we raise the following
assumption on the existence of the estimator that estimates
the state of the po-JDS (II.1).
Assumption 2.1: The states of the po-JDS Sin (II.1) can be
estimated by a proper estimator ˆ
Srepresented in the form of
an SDE as:
ˆ
S: d ˆ
ξ=f(ˆ
ξ, υ) d t+Kdy−h(ˆ
ξ) d t,(II.2)
where K∈Rn×pis the estimator gain.
There are plenty of results in the literature on the computation
of estimator gain Kfor various classes of stochastic systems;
see the results in [14], [11], [15], and [16]. We define the
augmented process [ξ, ˆ
ξ]T, where ξand ˆ
ξare the solution pro-
cesses of Sand ˆ
S, respectively. The corresponding augmented
jump-diffusion system ˜
Scan be defined as:
dξ
dˆ
ξ=f(ξ, υ)
f(ˆ
ξ, υ)+0n×p0n×p
K−Kh(ξ)
h(ˆ
ξ)dt
+g1(ξ) 0nׯr2
0nׯr1Kg2(ξ)dW1t
dW2t+r1(ξ)
0nׯq1dP1t+0nׯq2
Kr2(ξ)dP2t.
(II.3)
For later use, we provide the definition of the infinitesimal
generator (denoted by operator D) for ˜
Susing Ito’s differen-
tiation [13]. Let B:X×X→Rbe a twice differentiable
function. The infinitesimal generator of Bassociated with the
system ˜
Sfor all (x, ˆx)∈X×Xand for all u∈Uis given
by
DB(x, ˆx,u) =∂xB ∂ˆxB(f(x, u)
f(ˆx, u)+0n×p0n×p
K−Kh(x)
h(ˆx))
+1
2Tr(g1(x) 0nׯr2
0nׯr1Kg2(x)g1(x) 0nׯr2
0nׯr1Kg2(x)T∂xxB ∂xˆxB
∂ˆxxB ∂ˆxˆxB)
+
¯q1
X
i=1
λ1i(B(x+r1(x)ei,ˆx)−B(x, ˆx))
+
¯q2
X
i=1
λ2i(B(x+Kr2(x)ei,ˆx)−B(x, ˆx)).(II.4)
The symbols ∂xand ∂x,ˆxin (II.4) represent first and second-
order partial derivatives with respect to x(1st argument) and
ˆx(2nd argument), respectively. Note that we dropped the
arguments of ∂xB,∂ˆxB,∂x,xB,∂x, ˆxB,∂ˆx,xB, and ∂ˆx,ˆxB
in (II.4) for the sake of simplicity.
Given a po-JDS Sin (II.1), we aim at synthesizing a control
policy that guarantees a potentially tight lower bound on the
probability that system Ssatisfies a complex specification over
a finite time horizon. The class of specifications considered in
this paper are provided in the next subsection.
Remark 2.2: The use of the augmented system ˜
Swill
allow us to provide the main result of the paper without
any correctness requirement on the observer. In particualr, our
augmented system formulation provides the user the flexibility
to design any observer by means of any technique. The
probabilistic distance between the values of state and their
estimator is natively considered in our formulation and one
does not need to quantify this distance a-priori which is needed
in the results proposed in [12], [11].
B. Specifications
In this subsection, we consider the class of specifications
expressed by nondeterministic finite automata (NFA)as de-
fined below.
Definition 2.3: [17] A nondeterministic finite automaton
(NFA)is a tuple A= (Q, Q0,Σ, δ, F ), where Qis a finite
set of states, Q0⊆Qis a set of initial states, Σis a finite set
(a.k.a. alphabet),δ:Q×Σ→P(Q)is a transition function,
where P(Q)denotes the power set of Q, and F⊆Qis a set
of accepting (or final) states.
NFA Ais called deterministic if the transition function is
defined as δ:Q×Σ→Q, and we refer to it as deterministic
finite automata (DFA). Since every NFA can be converted to
its equivalent DFA using the powerset construction [18], in
the rest of the paper, we only deal with DFA. Moreover, it
is well known that the complement of a DFA A, denoted
by Ac, is again a DFA [19]. We use the notation qσ
−→ q0
to denote transition relation (q, σ, q0)∈δ. A finite word
σ= (σ0, σ1, . . . , σk−1)∈Σkis accepted by DFA Aif there
exists a finite state run q= (q0, q1, . . . , qk)∈Qk+1 such that
q0∈Q0,qi
σi
−→ qi+1 for all 0≤i<kand qk∈F. The
accepted language of A, denoted by L(A), is the set of all
words accepted by A.
In this work, we consider those specifications given by the
accepting languages of DFA Adefined over a set of atomic
propositions Π, i.e., the alphabet Σ=Π. We should highlight
that all linear temporal logic specifications defined over finite
traces, referred to as LTLF, are recognized by DFA [20].
C. Satisfaction of Specification by po-JDS
A given po-JDS Sin (II.1) is connected to the specification
given by the accepting language of a DFA Adefined over a
set of atomic propositions Π, with the help of a measurable
labeling function L:X→Πas described in the next
definition which is similar to [21, Definition 2].
Definition 2.4: For a po-JDS Sas in (II.1) and the la-
beling function L:X→Π, a finite sequence σ(ξaυ) =
(σ0, σ1, . . . , σk−1)∈Πk,k∈N, is a finite trace of the

solution process ξaυ over a finite time horizon [0, T )⊂R+
0if
there exists an associated time sequence t0, t1, . . . , tk−1such
that t0= 0,tk=T, and for all j∈ {0,1, . . . , k −1},tj∈R+
0
following conditions hold
•tj< tj+1;
•ξaυ(tj)∈L−1(σj);
•If σj6=σj+1, then for some t0
j∈[tj, tj+1],ξaυ (t)∈
L−1(σj)for all t∈(tj, t0
j);ξaυ(t)∈L−1(σj+1 )for all
t∈(t0
j, tj+1); and either ξaυ (t0
j)∈L−1(σj)or ξaυ(t0
j)∈
L−1(σj+1).
Next, we define the probability that the solution process ξaυ of
the po-JDS Sstarting from some initial state ξaυ(0) = a∈X0
under control policy υsatisfies the specification given by DFA
A.
Definition 2.5: The finite trace corresponding to the solution
process of a po-JDS Sstarting from a∈Xand under the
control policy υover a finite-time horizon [0, T )⊂R+
0, i.e.
σ(ξaυ)=(σ0, σ1, . . . , σj, . . . , σk−1)∈Πkas in Definition
2.4, satisfies a specification given by the language of a DFA A,
denoted by σ(ξaυ)|=A, if there exists j∈ {0, . . . , k−1}such
that (σ0, σ1, . . . , σj)∈ L(A). The probability of satisfaction
of the specification given by Ais denoted by P{σ(ξaυ)|=A}.
Remark 2.6: The set of atomic propositions Π =
{p0, p1, . . . , pM}and the labeling function L:X→Π
provide a measurable partition of the state set X=∪N
i=1Xi
as Xi:= L−1(pi). Without loss of generality, we assume that
Xi6=∅for any i.
D. Problem Definition
Now, we formally define the main synthesis problem con-
sidered in this work.
Problem 2.7: Given a po-JDS Sas in (II.1), a specification
given by the accepting language of DFA A= (Q, Q0,Π, δ, F )
over a set of atomic propositions Π = {p0, p1, . . . , pM}, a
labeling function L:X→Π, and a real value ϑ∈(0,1),
compute an offline control policy υ(if existing)such that
P{σ(ξaυ)|=A} ≥ ϑ, for all a∈L−1(pi)and some i∈
{0,1, . . . , M }.
Finding a solution to Problem 2.7 (if existing) is difficult
in general. We should highlight that the proposed approach
here is sound in solving the considered synthesis problem.
This means that if the proposed method provides a solution to
a synthesis problem, then we can formally conclude that the
proposed controller renders the given specification with the
corresponding lower bound on the probability of satisfaction.
However, if the method fails to provide any solution, then
there may or may not exist a solution to the original synthesis
problem). Our approach is to compute a policy υtogether
with a lower bound ϑ. Our aim is to find the potentially
largest lower bound, which can be compared with ϑand gives
policy, i.e., a solution for Problem 2.7 if ϑ≥ϑ. Instead of
computing a control policy that guarantees the lower bound ϑ,
we compute a policy that guarantees P{σ(ξaυ)|=Ac} ≤ ¯
ϑ,
for any a∈L−1(pi)and some i∈ {0,1, . . . , M }. Then
for the same control policy the lower bound can be easily
obtained as ϑ= 1 −¯
ϑ. This is done by constructing a DFA Ac
whose language is the complement of the language of DFA
A. To synthesize a controller, we utilize the notion of control
barrier functions defined for augmented jump-diffusion system
˜
Sintroduced in the next section.
III. CON TRO L BARRIER FUNCTIONS
In this section, we provide sufficient conditions using so-
called control barrier functions under which we can provide
the upper bound on the probability that the trajectories of
system Sstarting from any initial state in X0⊆Xreach
X1⊆X. To provide a result giving an upper bound on the
reachability probability for the trajectory of S, we provide
conditions on barrier functions constructed over the augmented
system ˜
S.
Theorem 3.1: Consider a po-JDS Sas in (II.1), its estimator
ˆ
Sas in (II.2), the resulting augmented system ˜
Sas in (II.3) and
sets X0, X1⊆X. Suppose there exists a twice differentiable
function B:X×X→R+
0, constants c≥0and γ∈[0,1)
such that
∀(x, ˆx)∈X0×X0, B(x, ˆx)≤γ, (III.1)
∀(x, ˆx)∈X1×X, B(x, ˆx)≥1,(III.2)
∀ˆx∈X, ∃u∈U, ∀x∈X, DB(x, ˆx, u)≤c. (III.3)
Then the probability that the solution process ξaυ of the system
Sstarts from any initial state a∈X0and reaches region X1
under the control policy υwithin time horizon [0, T )⊂R+
0
is upper bounded by γ+cT .
Proof: By using (III.1) and the fact that X1×
X⊆(x, ˆx)∈X×X|B(x, ˆx)≥1, we have
Pξaυ(t)∈X1∧ˆ
ξˆaυ (t)∈X∃t∈[0, T )|a, ˆa≤
Psup0≤t≤TB(ξaυ(t),ˆ
ξˆaυ (t)) ≥1|a, ˆa≤B(a, ˆa) + cT ≤
γ+cT . The second inequality is obtained by utilizing the result
of [22, Theorem 1]. This implies that the probability of the
augmented trajectory of ˜
Sstaring from any (a, ˆa)∈X0×X0
and reaching X1×Xis upper bounded by γ+cT .
Now we get Pξaυ(t)∈X1∧ˆ
ξˆaυ (t)∈X∃t∈[0, T )|
a, ˆa≤Pξaυ (t)∈X1∃t∈[0, T )|a+Pˆ
ξˆaυ (t)∈X∃t∈
[0, T )|ˆa−Pξaυ (t)∈X1∨ˆ
ξˆaυ (t)∈X∃t∈[0, T )|a, ˆa.
Since, the second and last terms trivially hold with probability
1, one has Pξaυ(t)∈X1∧ˆ
ξˆaυ (t)∈X∃t∈[0, T )|a, ˆa≤
Pξaυ(t)∈X1∃t∈[0, T )|a. Now, since the right term
of the and (i.e. ∧) is held for all time, the inequality above
becomes an equality and one gets Pξaυ(t)∈X1∃t∈[0, T )|
a≤γ+T c which concludes the proof.
The function Bin Theorem 3.1 satisfying (III.1)-(III.3) is
usually referred to as the control barrier function.
Remark 3.2: Condition (III.3) implicitly associates a sta-
tionary controller u:X→Uaccording to the existential
quantifier on ufor any ˆx∈Xand is independent of choice of
x∈X. The stationary control policy υdriving the system is
readily given by υ(t) = u(ˆ
ξaυ(t)), where ˆ
ξaυ is the solution
process of the estimator.
IV. FORMAL SYN TH ES IS O F CON TROLLERS
To synthesize control policies using control barrier functions
enforcing specifications expressed by DFA A, we first provide
the decomposition of specifications into sequential reachability
tasks which will later be solved using control barrier functions.

A. Decomposition into Sequential Reachability
Consider a DFA Aexpressing the properties of interest for
the system S. Consider DFA Ac= (Q, Q0,Π, δ, F )whose
language is the complement of the language of DFA A. The
sequence q= (q0, q1, . . . , qk)∈Qk+1,k∈Nis called an
accepting state run if q0∈Q0,qk∈F, and there exists a finite
word σ= (σ0, σ1, . . . , σk−1)∈Πksuch that qi
σi
−→ qi+1
for all i∈ {0,1, . . . , k −1}. We denote the finite word
corresponding to accepting state run qby σ(q). We also
indicate the length of q∈Qk+1 by |q|, which is k+ 1. Let
Rbe the set of all finite accepting state runs starting from
q0∈Q0excluding self-loops, where
R:={q=(q0, q1, . . . , qk)∈Qk+1 |qk∈F, qi6=qi+1,∀i<k}.
Computation of Rcan be done algorithmically by viewing
Acas a directed graph G= (V,E)with vertices V=Qand
edges E ⊆ V × V such that (q, q 0)∈ E if and only if q06=q
and there exist p∈Πsuch that qp
−→ q0. For any (q, q0)∈ E,
we donate the atomic proposition associated with the edge
(q, q0)by σ(q , q0). From the construction of the graph, it is
obvious that the finite path in the graph starting from vertices
q0∈Q0and ending at qF∈Fis an accepting state run q
of Acwithout any self-loop and therefore belongs to R. One
can easily compute Rusing depth first search algorithm [23].
For each p∈Π, we define a set Rpas
Rp:= {q= (q0, q1, . . . , qk)∈ R | σ(q0, q1) = p}.(IV.1)
Decomposition into sequential reachability is performed as
follows. For any q= (q0, q1, . . . , qk)∈ Rp∀p∈Π, we
define Pp(q)as a set of all state runs of length 3,
Pp(q) := {(qi, qi+1, qi+2 )|0≤i≤k−2}.(IV.2)
Now, we define P(Ac) := Sp∈ΠSq∈RpPp(q).
Remark 4.1: Note that Pp(q) = ∅for |q|= 2. In fact, any
accepting state run of length 2specifies a subset of the state
set such that the system satisfies Acwhenever it starts from
that subset. This gives trivial zero probability for satisfying
the specification, thus neglected in the sequel.
For the illustration of the above sets, we kindly refer the
interested reader to Example 1 in [8]. Having Pp(q)in (IV.2)
as the set of state runs of length 3, in this subsection, we
provide a systematic approach to compute a policy together
with a (potentially tight) lower bound on the probability that
the solution process of Ssatisfies the specifications given by
DFA A. Given a DFA Ac, our approach relies on performing
a reachability computation over each element of P(Ac)(i.e.,
Sp∈ΠSq∈RpPp(q)), where reachability probability is upper
bounded using control barrier functions along with appropri-
ate choices of control inputs as mentioned in Theorem 3.1.
However, computation of control barrier functions and the
policies for each element ν∈ P(Ac), can cause ambiguity
while utilizing controllers in closed-loop whenever there are
more than one outgoing edges from a state of the automaton.
To resolve this ambiguity, we simply merge such reachability
problems into one reachability problem by replacing the
reachable set X1×Xin Theorem 3.1 with the union of regions
corresponding to the alphabets of all outgoing edges. Thus we
get a common control barrier function and a corresponding
controller. This enables us to partition P(Ac)and put the
elements sharing a common control barrier function and a
corresponding controller in the same partition set. These sets
can be formally defined as
µ(q,q0,∆(q0)) := {(q, q0,q00 )∈ P(Ac)
|q, q0, q 00 ∈Qand q00 ∈∆(q0)}.
The control barrier function and the controller (as discussed
in Remark 3.2) corresponding to the partition set µ(q,q0,∆(q0))
are denoted by Bµ(q,q0,∆(q0)) (x, ˆx)and uµ(q,q0,∆(q0)) (ˆx), respec-
tively. Thus, for all ν∈ P(Ac), we have
Bν(x, ˆx) = Bµ(q,q0,∆(q0)) (x, ˆx)and uν(ˆx) = uµ(q,q 0,∆(q0)) (ˆx),
if ν∈µ(q,q0,∆(q0)) .
(IV.3)
B. Control Policy
From the above discussion, one can readily observe that
we have different control policies at different locations of
the automaton which can be interpreted as a switching con-
trol policy. Next, we define the automaton representing the
switching mechanism for control policies. Consider the DFA
Ac= (Q, Q0,Π, δ, F )corresponding to the complement of
DFA Aas discussed in Section IV-A, where ∆(q)denotes
the set of all successor states of q∈Q. Now, the switching
mechanism is given by a DFA Am= (Qm, Qm0,Πm, δm, Fm),
where Qm:= Qm0∪ {(q, q0,∆(q0)) |q , q0∈Q\F} ∪ Fmis
the set of states, Qm0:= {(q0,∆(q0)) |q0∈Q0}is the set
of initial states, Πm= Π,Fm=F, and the transition relation
(qm, σ, q0
m)∈δmis defined as
•for all qm= (q0,∆(q0)) ∈Qm0,
(q0,∆(q0))σ(q0,q00)
−→ (q0,q00 ,∆(q00)), where q0
σ(q0,q00)
−→ q00;
•for all qm= (q, q0,∆(q0)) ∈Qm\(Qm0∪Fm),
–(q, q0,∆(q0)) σ(q0,q 00)
−→ (q0, q00,∆(q00 )), such that
q, q0, q 00 ∈Q,q0σ(q0,q00)
−→ q00, and q00 /∈F; and
–(q, q0,∆(q0)) σ(q0,q 00)
−→ q00, such that q, q0, q00 ∈Q,
q0σ(q0,q00)
−→ q00, and q00 ∈F.
The hybrid controller defined over augmented state-space X×
Qmthat is a candidate for solving Problem 2.7 is given by
˜
u(ˆx, qm) = uµ(q0
m)(ˆx),∀(qm, L(ˆx), q0
m)∈δm.(IV.4)
The corresponding hybrid control policy υis given by υ(t) =
˜
u(ˆ
ξ(t), qm). For the illustration of the switching mechanism,
see Example 1 in [8, Section 5]. In the next subsection,
we discuss the computation of bound on the probability of
satisfying the specification under such a policy, which then
can be used for checking if this policy is indeed a solution for
Problem 2.7.
C. Computation of Probability
The next theorem provides an upper bound on the probabil-
ity that the solution process satisfies the specifications given
by A.
Theorem 4.2: For a specification given by the accepting
language of DFA A, let Acbe the DFA corresponding to the

complement of A,Rpbe the set defined in (IV.1), and Pp
be the set of runs of length 3defined in (IV.2). Then the
probability that the solution process of the system Sstarting
from any initial state a∈L−1(p)under the hybrid control
policy υassociated with the hybrid controller (IV.4) satisfies
Acwithin time horizon [0, T )is upper bounded by
P{σ(ξaυ)|=Ac}≤ X
q∈Rp
Y{(γν+cνT)|ν=(q,q0,q00 )∈Pp(q)},
(IV.5)
where γν+cνTis the upper bound on the probability that the
solution process of Sstarts from X0:= L−1(σ(q, q0)) and
reaches X1:= L−1(σ(q0, q00 )) under control policy υwithin
time horizon [0, T )which is computed via Theorem 3.1.
Proof: The proof is similar to that of [8, Theorem 5.2]
and is omitted here due to the lack of space.
Theorem 4.2 enables us to decompose the specification into
a collection of sequential reachabilities, compute bounds on
the reachability probabilities using Theorem 3.1, and then
combine the bounds in a sum-product expression.
Remark 4.3: In case we are unable to find control barrier
functions for some of the elements ν∈ Pp(q)in (IV.5), we
replace the related term (γν+cνT)by the pessimistic bound
1and apply random control input. In order to get a non-trivial
bound in (IV.5), at least one control barrier function must be
found for each q∈ Rp.
Corollary 4.4: Given the result of Theorem 4.2, the proba-
bility that the solution process of Sstarts from any a∈L−1(p)
under control policy υand satisfies specifications given by
DFA Aover time horizon [0, T )⊂R+
0is lower-bounded by
P{σ(ξaυ)|=A} ≥ 1−P{σ(ξaυ )|=Ac}.
D. Computation of Control Barrier Functions
Proving the existence of a control barrier function and find-
ing one are in general hard problems. However, if functions f,
h,g1,g2,r1, and r2are polynomial with respect to their argu-
ments and partition sets Xi=L−1(pi), i ∈ {0,1,2, . . . , M },
are bounded semi-algebraic sets (i.e., they can be represented
by polynomial (in)equalities), one can formulate conditions
in Theorem 3.1 as a sum-of-squares (SOS) optimization
problem. See [8, Section 5.3.1.] for a detailed discussion on
a similar approach. Having an SOS optimization problem,
one can efficiently search for a polynomial control barrier
function Bν(x, ˆx)and controller uν(ˆx), for any ν∈ P(A¬ϕ)
as in (IV.3) using SOSTOOLS [24] in conjunction with a
semidefinite programming solver such as SeDuMi [25] while
minimizing constants γνand cν. Having values of γνand
cνfor all ν∈ P(A¬ϕ), one can simply utilize results of
Theorem 4.2 and Corollary 4.4 to compute a lower bound on
the probability of satisfying the given specification. Note that
it may not be possible in advance to obtain a probability bound
that is meaningful, in such cases the order of a control barrier
function needs to increase to achieve the desired probability
bound.
Remark 4.5: Under the assumption that sets X, X0, and X1
in Theorem 3.1 are compact and input set Uis finite, one can
utilize counterexample guided inductive synthesis (CEGIS)
approach to search for barrier control functions for more
general nonlinear functions f, h, g1, g2, r1, and r2in (II.1).
For more detailed discussion on CEGIS approach, we kindly
refer interested readers to the algorithm in [8, Section 5.3.2.].
Computational Complexity: The number of triplets and
hence the number of control barrier functions needed to be
computed are bounded by |Q|3, where |Q|is the number of
states in DFA A. However, this is the worst-case bound and
in practice, the number of control barrier functions is much
smaller. In the case of sum-of-squares optimization approach,
the computational complexity of finding polynomial control
barrier functions depends on both the degree of polynomials
and the number of state variables. One can easily see that
for fixed polynomial degrees, the required computations grow
polynomially with respect to the dimension of the augmented
system. For the CEGIS approach, due to its iterative nature
and lack of guarantee on termination, it is difficult to provide
any analysis on the computational complexity.
V. CAS E STU DY
We consider a nonlinear Moore-Greitzer jet engine model
in no-stall mode [26] as a partially observed jump-diffusion
systems by adding noise and jump terms which is given by:
dξ1= (−ξ2−3
2ξ2
1−1
2ξ3
1) d t+ 0.2 d W11t+ 0.9 d Pt,
dξ2= (ξ1−υ) d t+ 0.06 d W12t,
dy=ξ2dt+ 0.06 d W2t,
where ξ= [ξ1, ξ2]T,ξ1= Φ −1,ξ2= Ψ −ψ−2,Φis the
mass flow, Ψis the pressure rise, and ψis a constant. Terms
W11t, W12t, and W2tdenote the standard Brownian motions
and Ptdenotes the Poisson process with rate λ= 5. We
consider a compact state set X= [−1,3]×[−4,4] and regions
of interest X0= [0,1]×[−1,1],X1= [−1,−0.2]×[−4,−2.5],
X2= [1,3] ×[2,4], and X3=X\(X0∪X1∪X2). The set
of atomic propositions is given by Π = {p0, p1, p2, p3}with
labeling function L(xj) = pjfor all xj∈Xj,j∈ {0,1,2,3}.
The objective here is to compute a control policy that provides
a lower bound on the probability that the trajectories of
the system satisfy the specification given by the accepting
language of the DFA Agiven in Figure 1 over finite time-
horizon [0, T = 10). Language of Aentails that if we start in
X0then the system will always stay away from X1or X2.
The corresponding DFA Acaccepting complement of L(A)is
shown in Figure 1. Following Subsection IV-A, we only need
to compute a control barrier function corresponding to triplet
(q0, q1, q2).
Now with an estimator gain in (II.2) as K=
[6.1394,7.8927]T, we use SOSTOOLS and SeDuMi to com-
pute a sum-of-squares polynomial control barrier function
B(x, ˆx)of order 4, sum-of-square polynomials ψ0(x, ˆx),
ψ1(x, ˆx),ψ(x, ˆx)of order 4, with total 1125 coefficients
resulting in a computation time of about 15 minutes. The
corresponding controller of order 2 is obtained as follows:
u(ˆx)=0.7321ˆx1−1.8612ˆx1ˆx2−1.4356ˆx2.(V.1)
The values of γ= 0.099 and c= 1 ×10−5are obtained
using bisection method resulting in P{σ(ξaυ)|=A} ≥ 0.89

Fig. 1. The DFA Arepresenting specification (left) and the DFA Ac
representing complement of A(right).
Fig. 2. A few closed loop trajectories starting from different initial conditions
in X0under controller (V.1).
for all x0∈L−1(p0), as discussed in Subsection IV-D. One
can see that only one controller is enough for enforcing the
specification, thus we do not need any switching mechanism.
Figure 2 shows a few trajectories starting from different initial
conditions under the control policy (V.1).
VI. CONCLUSIONS
In this paper, we proposed a discretization-free approach
for the formal controller synthesis of partially observed jump-
diffusion systems. The proposed method computes a hybrid
control policy together with a lower bound on the probability
of satisfying complex temporal logic specifications given by
the accepting language of DFA Aover a finite-time horizon.
This is achieved by constructing control barrier functions over
an augmented system consisting of both the system and the es-
timator. As a result, the probability bound is computed without
requiring any prior information of estimation accuracy.
REFERENCES
[1] C. Belta, B. Yordanov, and E. A. Gol, Formal methods for discrete-time
dynamical systems. Springer, 2017, vol. 89.
[2] P. Tabuada, Verification and control of hybrid systems: a symbolic
approach. Springer Science & Business Media, 2009.
[3] C. Belta, B. Yordanov, and E. A. Gol, “Discrete-time dynamical
systems,” in Formal Methods for Discrete-Time Dynamical Systems.
Springer, 2017, pp. 111–118.
[4] M. Zamani, P. M. Esfahani, R. Majumdar, A. Abate, and J. Lygeros,
“Symbolic control of stochastic systems via approximately bisimilar
finite abstractions,” IEEE Transactions on Automatic Control, vol. 59,
no. 12, pp. 3135–3150, 2014.
[5] M. Zamani, I. Tkachev, and A. Abate, “Towards scalable synthesis of
stochastic control systems,” Discrete Event Dynamic Systems, vol. 27,
no. 2, pp. 341–369, 2017.
[6] A. Lavaei, S. Soudjani, and M. Zamani, “Compositional (in) finite
abstractions for large-scale interconnected stochastic systems,” IEEE
Transactions on Automatic Control, 2020.
[7] A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control barrier
function based quadratic programs for safety critical systems,” IEEE
Transactions on Automatic Control, vol. 62, no. 8, pp. 3861–3876, 2016.
[8] P. Jagtap, S. Soudjani, and M. Zamani, “Formal synthesis of
stochastic systems via control barrier certificates,” arXiv preprint
arXiv:1905.04585, 2019.
[9] P. Jagtap, A. Swikir, and M. Zamani, “Compositional construction
of control barrier functions for interconnected control systems,” in
Proceedings of the 23rd International Conference on Hybrid Systems:
Computation and Control, 2020, pp. 1–11.
[10] C. Huang, X. Chen, W. Lin, Z. Yang, and X. Li, “Probabilistic safety
verification of stochastic hybrid systems using barrier certificates,” ACM
Transactions on Embedded Computing Systems (TECS), vol. 16, no. 5s,
p. 186, 2017.
[11] A. Clark, “Control barrier functions for complete and incomplete in-
formation stochastic systems,” in 2019 American Control Conference
(ACC). IEEE, 2019, pp. 2928–2935.
[12] N. Jahanshahi, P. Jagtap, and M. Zamani, “Synthesis of stochastic
systems with partial information via control barrier functions,” 21st IFAC
World Congress, 2020.
[13] B. Øksendal and A. Sulem, Applied stochastic control of jump diffusions.
Springer Science & Business Media, 2007.
[14] X. Kai, C. Wei, and L. Liu, “Robust extended kalman filtering for
nonlinear systems with stochastic uncertainties,” IEEE Transactions on
Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 40,
no. 2, pp. 399–405, 2009.
[15] B.-S. Chen, W.-H. Chen, and H.-L. Wu, “Robust h2/h∞global
linearization filter design for nonlinear stochastic systems,” IEEE trans-
actions on circuits and systems I: Regular Papers, vol. 56, no. 7, pp.
1441–1454, 2008.
[16] C.-S. Tseng, “Robust fuzzy filter design for a class of nonlinear
stochastic systems,” IEEE Transactions on Fuzzy Systems, vol. 15, no. 2,
pp. 261–274, 2007.
[17] C. Baier and J.-P. Katoen, Principles of model checking. MIT press,
2008.
[18] F. Bonchi and D. Pous, “Checking nfa equivalence with bisimulations
up to congruence,” ACM SIGPLAN Notices, vol. 48, no. 1, pp. 457–468,
2013.
[19] J. E. Hopcroft, R. Motwani, and J. D. Ullman, “Introduction to automata
theory, languages, and computation,” Acm Sigact News, vol. 32, no. 1,
pp. 60–65, 2001.
[20] G. De Giacomo and M. Vardi, “Synthesis for ltl and ldl on finite
traces,” in Twenty-Fourth International Joint Conference on Artificial
Intelligence, 2015.
[21] T. Wongpiromsarn, U. Topcu, and A. Lamperski, “Automata theory
meets barrier certificates: Temporal logic verification of nonlinear sys-
tems,” IEEE Transactions on Automatic Control, vol. 61, no. 11, pp.
3344–3355, 2015.
[22] H. Kushner, “Stochastic stability and control, ser,” Mathematics in
Science and Engineering. New York: Academic Press, 1967.
[23] S. J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach,
2nd ed. Pearson Education, 2003.
[24] S. Prajna, A. Papachristodoulou, and P. A. Parrilo, “Introducing sostools:
A general purpose sum of squares programming solver,” in Proceedings
of the 41st IEEE Conference on Decision and Control, 2002., vol. 1.
IEEE, 2002, pp. 741–746.
[25] J. F. Sturm, “Using sedumi 1.02, a matlab toolbox for optimization over
symmetric cones,” Optimization methods and software, vol. 11, no. 1-4,
pp. 625–653, 1999.
[26] M. Krstic and P. V. Kokotovic, “Lean backstepping design for a jet
engine compressor model,” in Proceedings of International Conference
on Control Applications. IEEE, 1995, pp. 1047–1052.