Content uploaded by Pushpak Jagtap

Author content

All content in this area was uploaded by Pushpak Jagtap on Jul 02, 2020

Content may be subject to copyright.

Synthesis of Partially Observed Jump-Diffusion

Systems via Control Barrier Functions

Niloofar Jahanshahi†, Pushpak Jagtap†, and Majid Zamani

Abstract—In this paper, we study formal synthesis of control

policies for partially observed jump-diffusion systems against

complex logic speciﬁcations. Given a state estimator, we utilize

a discretization-free approach for formal synthesis of control

policies by using a notation of control barrier functions without

requiring any knowledge of the estimation accuracy. Our goal

is to synthesize an ofﬂine control policy providing (potentially

maximizing) a lower bound on the probability that the trajecto-

ries of the partially observed jump-diffusion system satisfy some

complex speciﬁcations expressed by deterministic ﬁnite automata.

Finally, we illustrate the effectiveness of the proposed results by

synthesizing a policy for a jet engine example.

Index Terms—Stochastic control systems, Control barrier func-

tions, Controller synthesis, Output feedback control.

I. INTRODUCTION

RECENT years have witnessed a growing interest in

formal synthesis of controllers for complex systems

against complex logic speciﬁcations [1]. These speciﬁcations

are usually expressed using temporal logic formulae or as

(in)ﬁnite strings over ﬁnite automata. Several approaches

based on ﬁnite abstraction have been widely used to solve

such synthesis problems. Existing techniques include policy

synthesis enforcing linear temporal logic speciﬁcations for

non-stochastic systems [2], [3] and for stochastic ones [4],

[5], [6]. When dealing with large systems, these approaches

suffer severely from the curse of dimensionality (i.e., compu-

tational complexity grows exponentially with the dimension

of the state set). In order to overcome the large computational

burden, a discretization-free approach, based on control barrier

functions has shown potential to solve the formal synthesis

problems (See [7], [8], [9], [10] and references therein). The

aforementioned works assume the availability of complete

state information. However, in many real applications we do

not have access to complete state information. Motivated by

this limitation, the recent result in [11] provides the synthesis

of controllers enforcing invariance properties for stochastic

control systems with incomplete information by assuming a

prior knowledge of the control barrier functions. In our recent

†The authors contributed equally to this work.

This work was supported in part by the H2020 ERC Starting Grant

AutoCPS (grant agreement No. 804639), the German Research Foundation

(DFG) through the grants ZA 873/1-1 and the Research Training Group

2428, and the TUM International Graduate School of Science and Engineering

(IGSSE).

N. Jahanshahi is with the Computer Science Department, Ludwig

Maximilian University of Munich, Germany. P. Jagtap is with the

Department of Electrical and Computer Engineering, Technical University

of Munich, Germany. M. Zamani is with the Computer Science

Department, University of Colorado Boulder, USA. M. Zamani is with

the Computer Science Department, Ludwig Maximilian University of

Munich, Germany. Emails: niloofar.jahanshahi@lmu.de,

pushpak.jagtap@tum.de,majid.zamani@colorado.edu.

result [12], we consider the problem of synthesizing controllers

for partially observed stochastic control systems. In particular,

we search for a control barrier function that provides a

controller along with a lower bound on the probability that

the system satisﬁes invariance speciﬁcations over a ﬁnite-time

horizon. Similar to [11], this work also assumes the existence

of an estimator with a given probabilistic accuracy. Then we

provide the overall probability threshold using the probability

bound on the estimator accuracy and that of the trajectories of

the estimator satisfying the invariance speciﬁcations, obtained

via control barrier functions.

The contributions of this paper in comparison with those of

[11], [12] are twofold. First, we provide an ofﬂine controller

synthesis approach enforcing complex logic speciﬁcations

expressed by (non)deterministic ﬁnite automata for partially

observed jump-diffusion systems. As a special case, those

properties include invariance ones. Second, we provide an

approach for computing lower bound on the probability that

the system satisﬁes given speciﬁcations over a ﬁnite-time

horizon without requiring any knowledge of the estimator’s

accuracy. Finally, we demonstrate the effectiveness of the

proposed results on a nonlinear jet engine example.

II. PRELIMINARIES AND PROBLEM DEFINITION

Notations: We denote the set of natural, real, and non-

negative real numbers by N,R, and R+

0, respectively. We use

Rnto denote the n-dimensional Euclidean space and Rn×rto

denote the space of real matrices with nrows and rcolumns.

We denote by ei∈Rnthe vector whose all elements are

zero, except the ith element, which is one. Given a matrix

A∈Rn×n, Tr(A)represents trace of Awhich is the sum

of all diagonal elements of A. The zero matrix in Rn×mis

denoted by 0n×m. Given sets Xand Y, we donate f:X→Y

an ordinary map from Xto Yand the notation |X|denotes

the cardinality of set X.

A. Partially Observed Jump-Diffusion Systems

Let the triplet (Ω,F,P)denote a probability space with

a sample space Ω, ﬁltration F, and the probability measure

P. The ﬁltration F= (Fs)s≥0satisﬁes the usual conditions

of right continuity and completeness [13]. Let (Wks)s≥0be

¯rk-dimensional F-Brownian motions, k= 1,2. Let (Pks)s≥0

be a ¯qk-dimensional F-Poisson processes, with k= 1,2.

We assume that the Poisson processes and Brownian motions

are independent of each other. The Poisson process Pks :=

[P1

ks;· · · ;P¯qk

ks ]models ¯qkkinds of events, k= 1,2, whose

occurrences are assumed to be independent of each other. We

consider the partially observed jump-diffusion system (po-

JDS), denoted by S, which is described by the following

stochastic differential equations (SDE)

S:(dξ=f(ξ, υ) d t+g1(ξ) d W1t+r1(ξ) d P1t,

dy=h(ξ) d t+g2(ξ) d W2t+r2(ξ) d P2t,(II.1)

where ξ(t)∈X⊆Rnis the value of solution process ξof

S,υ(t)∈U⊆Rmis the input vector, and y(t)∈Rpis the

output vector representing the noisy partial observation at time

t∈R+

0P-almost surely (P-a.s.). Functions f:X×U→Rn,

g1:X→Rn×¯r1,g2:X→Rp×¯r2,r1:X→Rn×¯q1,r2:

X→Rp×¯q2, and h:X→Rpare assumed to be Lipschitz

continuous to ensure existence and uniqueness of the solution

of S[13]. Throughout the paper, we use the notation ξaυ(t)to

denote the value of the solution process of Sat time t∈R+

0

under the input signal υstarting from the initial state ξaυ(0) =

aP-a.s., in which ais a random variable that is measurable

in F0. Here, we assume that the Poisson processes Pi

ks for

any i∈ {1,...,¯qk},k= 1,2, have the rates of λki. In order

to provide the results in this paper, we raise the following

assumption on the existence of the estimator that estimates

the state of the po-JDS (II.1).

Assumption 2.1: The states of the po-JDS Sin (II.1) can be

estimated by a proper estimator ˆ

Srepresented in the form of

an SDE as:

ˆ

S: d ˆ

ξ=f(ˆ

ξ, υ) d t+Kdy−h(ˆ

ξ) d t,(II.2)

where K∈Rn×pis the estimator gain.

There are plenty of results in the literature on the computation

of estimator gain Kfor various classes of stochastic systems;

see the results in [14], [11], [15], and [16]. We deﬁne the

augmented process [ξ, ˆ

ξ]T, where ξand ˆ

ξare the solution pro-

cesses of Sand ˆ

S, respectively. The corresponding augmented

jump-diffusion system ˜

Scan be deﬁned as:

dξ

dˆ

ξ=f(ξ, υ)

f(ˆ

ξ, υ)+0n×p0n×p

K−Kh(ξ)

h(ˆ

ξ)dt

+g1(ξ) 0n×¯r2

0n×¯r1Kg2(ξ)dW1t

dW2t+r1(ξ)

0n×¯q1dP1t+0n×¯q2

Kr2(ξ)dP2t.

(II.3)

For later use, we provide the deﬁnition of the inﬁnitesimal

generator (denoted by operator D) for ˜

Susing Ito’s differen-

tiation [13]. Let B:X×X→Rbe a twice differentiable

function. The inﬁnitesimal generator of Bassociated with the

system ˜

Sfor all (x, ˆx)∈X×Xand for all u∈Uis given

by

DB(x, ˆx,u) =∂xB ∂ˆxB(f(x, u)

f(ˆx, u)+0n×p0n×p

K−Kh(x)

h(ˆx))

+1

2Tr(g1(x) 0n×¯r2

0n×¯r1Kg2(x)g1(x) 0n×¯r2

0n×¯r1Kg2(x)T∂xxB ∂xˆxB

∂ˆxxB ∂ˆxˆxB)

+

¯q1

X

i=1

λ1i(B(x+r1(x)ei,ˆx)−B(x, ˆx))

+

¯q2

X

i=1

λ2i(B(x+Kr2(x)ei,ˆx)−B(x, ˆx)).(II.4)

The symbols ∂xand ∂x,ˆxin (II.4) represent ﬁrst and second-

order partial derivatives with respect to x(1st argument) and

ˆx(2nd argument), respectively. Note that we dropped the

arguments of ∂xB,∂ˆxB,∂x,xB,∂x, ˆxB,∂ˆx,xB, and ∂ˆx,ˆxB

in (II.4) for the sake of simplicity.

Given a po-JDS Sin (II.1), we aim at synthesizing a control

policy that guarantees a potentially tight lower bound on the

probability that system Ssatisﬁes a complex speciﬁcation over

a ﬁnite time horizon. The class of speciﬁcations considered in

this paper are provided in the next subsection.

Remark 2.2: The use of the augmented system ˜

Swill

allow us to provide the main result of the paper without

any correctness requirement on the observer. In particualr, our

augmented system formulation provides the user the ﬂexibility

to design any observer by means of any technique. The

probabilistic distance between the values of state and their

estimator is natively considered in our formulation and one

does not need to quantify this distance a-priori which is needed

in the results proposed in [12], [11].

B. Speciﬁcations

In this subsection, we consider the class of speciﬁcations

expressed by nondeterministic ﬁnite automata (NFA)as de-

ﬁned below.

Deﬁnition 2.3: [17] A nondeterministic ﬁnite automaton

(NFA)is a tuple A= (Q, Q0,Σ, δ, F ), where Qis a ﬁnite

set of states, Q0⊆Qis a set of initial states, Σis a ﬁnite set

(a.k.a. alphabet),δ:Q×Σ→P(Q)is a transition function,

where P(Q)denotes the power set of Q, and F⊆Qis a set

of accepting (or ﬁnal) states.

NFA Ais called deterministic if the transition function is

deﬁned as δ:Q×Σ→Q, and we refer to it as deterministic

ﬁnite automata (DFA). Since every NFA can be converted to

its equivalent DFA using the powerset construction [18], in

the rest of the paper, we only deal with DFA. Moreover, it

is well known that the complement of a DFA A, denoted

by Ac, is again a DFA [19]. We use the notation qσ

−→ q0

to denote transition relation (q, σ, q0)∈δ. A ﬁnite word

σ= (σ0, σ1, . . . , σk−1)∈Σkis accepted by DFA Aif there

exists a ﬁnite state run q= (q0, q1, . . . , qk)∈Qk+1 such that

q0∈Q0,qi

σi

−→ qi+1 for all 0≤i<kand qk∈F. The

accepted language of A, denoted by L(A), is the set of all

words accepted by A.

In this work, we consider those speciﬁcations given by the

accepting languages of DFA Adeﬁned over a set of atomic

propositions Π, i.e., the alphabet Σ=Π. We should highlight

that all linear temporal logic speciﬁcations deﬁned over ﬁnite

traces, referred to as LTLF, are recognized by DFA [20].

C. Satisfaction of Speciﬁcation by po-JDS

A given po-JDS Sin (II.1) is connected to the speciﬁcation

given by the accepting language of a DFA Adeﬁned over a

set of atomic propositions Π, with the help of a measurable

labeling function L:X→Πas described in the next

deﬁnition which is similar to [21, Deﬁnition 2].

Deﬁnition 2.4: For a po-JDS Sas in (II.1) and the la-

beling function L:X→Π, a ﬁnite sequence σ(ξaυ) =

(σ0, σ1, . . . , σk−1)∈Πk,k∈N, is a ﬁnite trace of the

solution process ξaυ over a ﬁnite time horizon [0, T )⊂R+

0if

there exists an associated time sequence t0, t1, . . . , tk−1such

that t0= 0,tk=T, and for all j∈ {0,1, . . . , k −1},tj∈R+

0

following conditions hold

•tj< tj+1;

•ξaυ(tj)∈L−1(σj);

•If σj6=σj+1, then for some t0

j∈[tj, tj+1],ξaυ (t)∈

L−1(σj)for all t∈(tj, t0

j);ξaυ(t)∈L−1(σj+1 )for all

t∈(t0

j, tj+1); and either ξaυ (t0

j)∈L−1(σj)or ξaυ(t0

j)∈

L−1(σj+1).

Next, we deﬁne the probability that the solution process ξaυ of

the po-JDS Sstarting from some initial state ξaυ(0) = a∈X0

under control policy υsatisﬁes the speciﬁcation given by DFA

A.

Deﬁnition 2.5: The ﬁnite trace corresponding to the solution

process of a po-JDS Sstarting from a∈Xand under the

control policy υover a ﬁnite-time horizon [0, T )⊂R+

0, i.e.

σ(ξaυ)=(σ0, σ1, . . . , σj, . . . , σk−1)∈Πkas in Deﬁnition

2.4, satisﬁes a speciﬁcation given by the language of a DFA A,

denoted by σ(ξaυ)|=A, if there exists j∈ {0, . . . , k−1}such

that (σ0, σ1, . . . , σj)∈ L(A). The probability of satisfaction

of the speciﬁcation given by Ais denoted by P{σ(ξaυ)|=A}.

Remark 2.6: The set of atomic propositions Π =

{p0, p1, . . . , pM}and the labeling function L:X→Π

provide a measurable partition of the state set X=∪N

i=1Xi

as Xi:= L−1(pi). Without loss of generality, we assume that

Xi6=∅for any i.

D. Problem Deﬁnition

Now, we formally deﬁne the main synthesis problem con-

sidered in this work.

Problem 2.7: Given a po-JDS Sas in (II.1), a speciﬁcation

given by the accepting language of DFA A= (Q, Q0,Π, δ, F )

over a set of atomic propositions Π = {p0, p1, . . . , pM}, a

labeling function L:X→Π, and a real value ϑ∈(0,1),

compute an ofﬂine control policy υ(if existing)such that

P{σ(ξaυ)|=A} ≥ ϑ, for all a∈L−1(pi)and some i∈

{0,1, . . . , M }.

Finding a solution to Problem 2.7 (if existing) is difﬁcult

in general. We should highlight that the proposed approach

here is sound in solving the considered synthesis problem.

This means that if the proposed method provides a solution to

a synthesis problem, then we can formally conclude that the

proposed controller renders the given speciﬁcation with the

corresponding lower bound on the probability of satisfaction.

However, if the method fails to provide any solution, then

there may or may not exist a solution to the original synthesis

problem). Our approach is to compute a policy υtogether

with a lower bound ϑ. Our aim is to ﬁnd the potentially

largest lower bound, which can be compared with ϑand gives

policy, i.e., a solution for Problem 2.7 if ϑ≥ϑ. Instead of

computing a control policy that guarantees the lower bound ϑ,

we compute a policy that guarantees P{σ(ξaυ)|=Ac} ≤ ¯

ϑ,

for any a∈L−1(pi)and some i∈ {0,1, . . . , M }. Then

for the same control policy the lower bound can be easily

obtained as ϑ= 1 −¯

ϑ. This is done by constructing a DFA Ac

whose language is the complement of the language of DFA

A. To synthesize a controller, we utilize the notion of control

barrier functions deﬁned for augmented jump-diffusion system

˜

Sintroduced in the next section.

III. CON TRO L BARRIER FUNCTIONS

In this section, we provide sufﬁcient conditions using so-

called control barrier functions under which we can provide

the upper bound on the probability that the trajectories of

system Sstarting from any initial state in X0⊆Xreach

X1⊆X. To provide a result giving an upper bound on the

reachability probability for the trajectory of S, we provide

conditions on barrier functions constructed over the augmented

system ˜

S.

Theorem 3.1: Consider a po-JDS Sas in (II.1), its estimator

ˆ

Sas in (II.2), the resulting augmented system ˜

Sas in (II.3) and

sets X0, X1⊆X. Suppose there exists a twice differentiable

function B:X×X→R+

0, constants c≥0and γ∈[0,1)

such that

∀(x, ˆx)∈X0×X0, B(x, ˆx)≤γ, (III.1)

∀(x, ˆx)∈X1×X, B(x, ˆx)≥1,(III.2)

∀ˆx∈X, ∃u∈U, ∀x∈X, DB(x, ˆx, u)≤c. (III.3)

Then the probability that the solution process ξaυ of the system

Sstarts from any initial state a∈X0and reaches region X1

under the control policy υwithin time horizon [0, T )⊂R+

0

is upper bounded by γ+cT .

Proof: By using (III.1) and the fact that X1×

X⊆(x, ˆx)∈X×X|B(x, ˆx)≥1, we have

Pξaυ(t)∈X1∧ˆ

ξˆaυ (t)∈X∃t∈[0, T )|a, ˆa≤

Psup0≤t≤TB(ξaυ(t),ˆ

ξˆaυ (t)) ≥1|a, ˆa≤B(a, ˆa) + cT ≤

γ+cT . The second inequality is obtained by utilizing the result

of [22, Theorem 1]. This implies that the probability of the

augmented trajectory of ˜

Sstaring from any (a, ˆa)∈X0×X0

and reaching X1×Xis upper bounded by γ+cT .

Now we get Pξaυ(t)∈X1∧ˆ

ξˆaυ (t)∈X∃t∈[0, T )|

a, ˆa≤Pξaυ (t)∈X1∃t∈[0, T )|a+Pˆ

ξˆaυ (t)∈X∃t∈

[0, T )|ˆa−Pξaυ (t)∈X1∨ˆ

ξˆaυ (t)∈X∃t∈[0, T )|a, ˆa.

Since, the second and last terms trivially hold with probability

1, one has Pξaυ(t)∈X1∧ˆ

ξˆaυ (t)∈X∃t∈[0, T )|a, ˆa≤

Pξaυ(t)∈X1∃t∈[0, T )|a. Now, since the right term

of the and (i.e. ∧) is held for all time, the inequality above

becomes an equality and one gets Pξaυ(t)∈X1∃t∈[0, T )|

a≤γ+T c which concludes the proof.

The function Bin Theorem 3.1 satisfying (III.1)-(III.3) is

usually referred to as the control barrier function.

Remark 3.2: Condition (III.3) implicitly associates a sta-

tionary controller u:X→Uaccording to the existential

quantiﬁer on ufor any ˆx∈Xand is independent of choice of

x∈X. The stationary control policy υdriving the system is

readily given by υ(t) = u(ˆ

ξaυ(t)), where ˆ

ξaυ is the solution

process of the estimator.

IV. FORMAL SYN TH ES IS O F CON TROLLERS

To synthesize control policies using control barrier functions

enforcing speciﬁcations expressed by DFA A, we ﬁrst provide

the decomposition of speciﬁcations into sequential reachability

tasks which will later be solved using control barrier functions.

A. Decomposition into Sequential Reachability

Consider a DFA Aexpressing the properties of interest for

the system S. Consider DFA Ac= (Q, Q0,Π, δ, F )whose

language is the complement of the language of DFA A. The

sequence q= (q0, q1, . . . , qk)∈Qk+1,k∈Nis called an

accepting state run if q0∈Q0,qk∈F, and there exists a ﬁnite

word σ= (σ0, σ1, . . . , σk−1)∈Πksuch that qi

σi

−→ qi+1

for all i∈ {0,1, . . . , k −1}. We denote the ﬁnite word

corresponding to accepting state run qby σ(q). We also

indicate the length of q∈Qk+1 by |q|, which is k+ 1. Let

Rbe the set of all ﬁnite accepting state runs starting from

q0∈Q0excluding self-loops, where

R:={q=(q0, q1, . . . , qk)∈Qk+1 |qk∈F, qi6=qi+1,∀i<k}.

Computation of Rcan be done algorithmically by viewing

Acas a directed graph G= (V,E)with vertices V=Qand

edges E ⊆ V × V such that (q, q 0)∈ E if and only if q06=q

and there exist p∈Πsuch that qp

−→ q0. For any (q, q0)∈ E,

we donate the atomic proposition associated with the edge

(q, q0)by σ(q , q0). From the construction of the graph, it is

obvious that the ﬁnite path in the graph starting from vertices

q0∈Q0and ending at qF∈Fis an accepting state run q

of Acwithout any self-loop and therefore belongs to R. One

can easily compute Rusing depth ﬁrst search algorithm [23].

For each p∈Π, we deﬁne a set Rpas

Rp:= {q= (q0, q1, . . . , qk)∈ R | σ(q0, q1) = p}.(IV.1)

Decomposition into sequential reachability is performed as

follows. For any q= (q0, q1, . . . , qk)∈ Rp∀p∈Π, we

deﬁne Pp(q)as a set of all state runs of length 3,

Pp(q) := {(qi, qi+1, qi+2 )|0≤i≤k−2}.(IV.2)

Now, we deﬁne P(Ac) := Sp∈ΠSq∈RpPp(q).

Remark 4.1: Note that Pp(q) = ∅for |q|= 2. In fact, any

accepting state run of length 2speciﬁes a subset of the state

set such that the system satisﬁes Acwhenever it starts from

that subset. This gives trivial zero probability for satisfying

the speciﬁcation, thus neglected in the sequel.

For the illustration of the above sets, we kindly refer the

interested reader to Example 1 in [8]. Having Pp(q)in (IV.2)

as the set of state runs of length 3, in this subsection, we

provide a systematic approach to compute a policy together

with a (potentially tight) lower bound on the probability that

the solution process of Ssatisﬁes the speciﬁcations given by

DFA A. Given a DFA Ac, our approach relies on performing

a reachability computation over each element of P(Ac)(i.e.,

Sp∈ΠSq∈RpPp(q)), where reachability probability is upper

bounded using control barrier functions along with appropri-

ate choices of control inputs as mentioned in Theorem 3.1.

However, computation of control barrier functions and the

policies for each element ν∈ P(Ac), can cause ambiguity

while utilizing controllers in closed-loop whenever there are

more than one outgoing edges from a state of the automaton.

To resolve this ambiguity, we simply merge such reachability

problems into one reachability problem by replacing the

reachable set X1×Xin Theorem 3.1 with the union of regions

corresponding to the alphabets of all outgoing edges. Thus we

get a common control barrier function and a corresponding

controller. This enables us to partition P(Ac)and put the

elements sharing a common control barrier function and a

corresponding controller in the same partition set. These sets

can be formally deﬁned as

µ(q,q0,∆(q0)) := {(q, q0,q00 )∈ P(Ac)

|q, q0, q 00 ∈Qand q00 ∈∆(q0)}.

The control barrier function and the controller (as discussed

in Remark 3.2) corresponding to the partition set µ(q,q0,∆(q0))

are denoted by Bµ(q,q0,∆(q0)) (x, ˆx)and uµ(q,q0,∆(q0)) (ˆx), respec-

tively. Thus, for all ν∈ P(Ac), we have

Bν(x, ˆx) = Bµ(q,q0,∆(q0)) (x, ˆx)and uν(ˆx) = uµ(q,q 0,∆(q0)) (ˆx),

if ν∈µ(q,q0,∆(q0)) .

(IV.3)

B. Control Policy

From the above discussion, one can readily observe that

we have different control policies at different locations of

the automaton which can be interpreted as a switching con-

trol policy. Next, we deﬁne the automaton representing the

switching mechanism for control policies. Consider the DFA

Ac= (Q, Q0,Π, δ, F )corresponding to the complement of

DFA Aas discussed in Section IV-A, where ∆(q)denotes

the set of all successor states of q∈Q. Now, the switching

mechanism is given by a DFA Am= (Qm, Qm0,Πm, δm, Fm),

where Qm:= Qm0∪ {(q, q0,∆(q0)) |q , q0∈Q\F} ∪ Fmis

the set of states, Qm0:= {(q0,∆(q0)) |q0∈Q0}is the set

of initial states, Πm= Π,Fm=F, and the transition relation

(qm, σ, q0

m)∈δmis deﬁned as

•for all qm= (q0,∆(q0)) ∈Qm0,

(q0,∆(q0))σ(q0,q00)

−→ (q0,q00 ,∆(q00)), where q0

σ(q0,q00)

−→ q00;

•for all qm= (q, q0,∆(q0)) ∈Qm\(Qm0∪Fm),

–(q, q0,∆(q0)) σ(q0,q 00)

−→ (q0, q00,∆(q00 )), such that

q, q0, q 00 ∈Q,q0σ(q0,q00)

−→ q00, and q00 /∈F; and

–(q, q0,∆(q0)) σ(q0,q 00)

−→ q00, such that q, q0, q00 ∈Q,

q0σ(q0,q00)

−→ q00, and q00 ∈F.

The hybrid controller deﬁned over augmented state-space X×

Qmthat is a candidate for solving Problem 2.7 is given by

˜

u(ˆx, qm) = uµ(q0

m)(ˆx),∀(qm, L(ˆx), q0

m)∈δm.(IV.4)

The corresponding hybrid control policy υis given by υ(t) =

˜

u(ˆ

ξ(t), qm). For the illustration of the switching mechanism,

see Example 1 in [8, Section 5]. In the next subsection,

we discuss the computation of bound on the probability of

satisfying the speciﬁcation under such a policy, which then

can be used for checking if this policy is indeed a solution for

Problem 2.7.

C. Computation of Probability

The next theorem provides an upper bound on the probabil-

ity that the solution process satisﬁes the speciﬁcations given

by A.

Theorem 4.2: For a speciﬁcation given by the accepting

language of DFA A, let Acbe the DFA corresponding to the

complement of A,Rpbe the set deﬁned in (IV.1), and Pp

be the set of runs of length 3deﬁned in (IV.2). Then the

probability that the solution process of the system Sstarting

from any initial state a∈L−1(p)under the hybrid control

policy υassociated with the hybrid controller (IV.4) satisﬁes

Acwithin time horizon [0, T )is upper bounded by

P{σ(ξaυ)|=Ac}≤ X

q∈Rp

Y{(γν+cνT)|ν=(q,q0,q00 )∈Pp(q)},

(IV.5)

where γν+cνTis the upper bound on the probability that the

solution process of Sstarts from X0:= L−1(σ(q, q0)) and

reaches X1:= L−1(σ(q0, q00 )) under control policy υwithin

time horizon [0, T )which is computed via Theorem 3.1.

Proof: The proof is similar to that of [8, Theorem 5.2]

and is omitted here due to the lack of space.

Theorem 4.2 enables us to decompose the speciﬁcation into

a collection of sequential reachabilities, compute bounds on

the reachability probabilities using Theorem 3.1, and then

combine the bounds in a sum-product expression.

Remark 4.3: In case we are unable to ﬁnd control barrier

functions for some of the elements ν∈ Pp(q)in (IV.5), we

replace the related term (γν+cνT)by the pessimistic bound

1and apply random control input. In order to get a non-trivial

bound in (IV.5), at least one control barrier function must be

found for each q∈ Rp.

Corollary 4.4: Given the result of Theorem 4.2, the proba-

bility that the solution process of Sstarts from any a∈L−1(p)

under control policy υand satisﬁes speciﬁcations given by

DFA Aover time horizon [0, T )⊂R+

0is lower-bounded by

P{σ(ξaυ)|=A} ≥ 1−P{σ(ξaυ )|=Ac}.

D. Computation of Control Barrier Functions

Proving the existence of a control barrier function and ﬁnd-

ing one are in general hard problems. However, if functions f,

h,g1,g2,r1, and r2are polynomial with respect to their argu-

ments and partition sets Xi=L−1(pi), i ∈ {0,1,2, . . . , M },

are bounded semi-algebraic sets (i.e., they can be represented

by polynomial (in)equalities), one can formulate conditions

in Theorem 3.1 as a sum-of-squares (SOS) optimization

problem. See [8, Section 5.3.1.] for a detailed discussion on

a similar approach. Having an SOS optimization problem,

one can efﬁciently search for a polynomial control barrier

function Bν(x, ˆx)and controller uν(ˆx), for any ν∈ P(A¬ϕ)

as in (IV.3) using SOSTOOLS [24] in conjunction with a

semideﬁnite programming solver such as SeDuMi [25] while

minimizing constants γνand cν. Having values of γνand

cνfor all ν∈ P(A¬ϕ), one can simply utilize results of

Theorem 4.2 and Corollary 4.4 to compute a lower bound on

the probability of satisfying the given speciﬁcation. Note that

it may not be possible in advance to obtain a probability bound

that is meaningful, in such cases the order of a control barrier

function needs to increase to achieve the desired probability

bound.

Remark 4.5: Under the assumption that sets X, X0, and X1

in Theorem 3.1 are compact and input set Uis ﬁnite, one can

utilize counterexample guided inductive synthesis (CEGIS)

approach to search for barrier control functions for more

general nonlinear functions f, h, g1, g2, r1, and r2in (II.1).

For more detailed discussion on CEGIS approach, we kindly

refer interested readers to the algorithm in [8, Section 5.3.2.].

Computational Complexity: The number of triplets and

hence the number of control barrier functions needed to be

computed are bounded by |Q|3, where |Q|is the number of

states in DFA A. However, this is the worst-case bound and

in practice, the number of control barrier functions is much

smaller. In the case of sum-of-squares optimization approach,

the computational complexity of ﬁnding polynomial control

barrier functions depends on both the degree of polynomials

and the number of state variables. One can easily see that

for ﬁxed polynomial degrees, the required computations grow

polynomially with respect to the dimension of the augmented

system. For the CEGIS approach, due to its iterative nature

and lack of guarantee on termination, it is difﬁcult to provide

any analysis on the computational complexity.

V. CAS E STU DY

We consider a nonlinear Moore-Greitzer jet engine model

in no-stall mode [26] as a partially observed jump-diffusion

systems by adding noise and jump terms which is given by:

dξ1= (−ξ2−3

2ξ2

1−1

2ξ3

1) d t+ 0.2 d W11t+ 0.9 d Pt,

dξ2= (ξ1−υ) d t+ 0.06 d W12t,

dy=ξ2dt+ 0.06 d W2t,

where ξ= [ξ1, ξ2]T,ξ1= Φ −1,ξ2= Ψ −ψ−2,Φis the

mass ﬂow, Ψis the pressure rise, and ψis a constant. Terms

W11t, W12t, and W2tdenote the standard Brownian motions

and Ptdenotes the Poisson process with rate λ= 5. We

consider a compact state set X= [−1,3]×[−4,4] and regions

of interest X0= [0,1]×[−1,1],X1= [−1,−0.2]×[−4,−2.5],

X2= [1,3] ×[2,4], and X3=X\(X0∪X1∪X2). The set

of atomic propositions is given by Π = {p0, p1, p2, p3}with

labeling function L(xj) = pjfor all xj∈Xj,j∈ {0,1,2,3}.

The objective here is to compute a control policy that provides

a lower bound on the probability that the trajectories of

the system satisfy the speciﬁcation given by the accepting

language of the DFA Agiven in Figure 1 over ﬁnite time-

horizon [0, T = 10). Language of Aentails that if we start in

X0then the system will always stay away from X1or X2.

The corresponding DFA Acaccepting complement of L(A)is

shown in Figure 1. Following Subsection IV-A, we only need

to compute a control barrier function corresponding to triplet

(q0, q1, q2).

Now with an estimator gain in (II.2) as K=

[6.1394,7.8927]T, we use SOSTOOLS and SeDuMi to com-

pute a sum-of-squares polynomial control barrier function

B(x, ˆx)of order 4, sum-of-square polynomials ψ0(x, ˆx),

ψ1(x, ˆx),ψ(x, ˆx)of order 4, with total 1125 coefﬁcients

resulting in a computation time of about 15 minutes. The

corresponding controller of order 2 is obtained as follows:

u(ˆx)=0.7321ˆx1−1.8612ˆx1ˆx2−1.4356ˆx2.(V.1)

The values of γ= 0.099 and c= 1 ×10−5are obtained

using bisection method resulting in P{σ(ξaυ)|=A} ≥ 0.89

Fig. 1. The DFA Arepresenting speciﬁcation (left) and the DFA Ac

representing complement of A(right).

Fig. 2. A few closed loop trajectories starting from different initial conditions

in X0under controller (V.1).

for all x0∈L−1(p0), as discussed in Subsection IV-D. One

can see that only one controller is enough for enforcing the

speciﬁcation, thus we do not need any switching mechanism.

Figure 2 shows a few trajectories starting from different initial

conditions under the control policy (V.1).

VI. CONCLUSIONS

In this paper, we proposed a discretization-free approach

for the formal controller synthesis of partially observed jump-

diffusion systems. The proposed method computes a hybrid

control policy together with a lower bound on the probability

of satisfying complex temporal logic speciﬁcations given by

the accepting language of DFA Aover a ﬁnite-time horizon.

This is achieved by constructing control barrier functions over

an augmented system consisting of both the system and the es-

timator. As a result, the probability bound is computed without

requiring any prior information of estimation accuracy.

REFERENCES

[1] C. Belta, B. Yordanov, and E. A. Gol, Formal methods for discrete-time

dynamical systems. Springer, 2017, vol. 89.

[2] P. Tabuada, Veriﬁcation and control of hybrid systems: a symbolic

approach. Springer Science & Business Media, 2009.

[3] C. Belta, B. Yordanov, and E. A. Gol, “Discrete-time dynamical

systems,” in Formal Methods for Discrete-Time Dynamical Systems.

Springer, 2017, pp. 111–118.

[4] M. Zamani, P. M. Esfahani, R. Majumdar, A. Abate, and J. Lygeros,

“Symbolic control of stochastic systems via approximately bisimilar

ﬁnite abstractions,” IEEE Transactions on Automatic Control, vol. 59,

no. 12, pp. 3135–3150, 2014.

[5] M. Zamani, I. Tkachev, and A. Abate, “Towards scalable synthesis of

stochastic control systems,” Discrete Event Dynamic Systems, vol. 27,

no. 2, pp. 341–369, 2017.

[6] A. Lavaei, S. Soudjani, and M. Zamani, “Compositional (in) ﬁnite

abstractions for large-scale interconnected stochastic systems,” IEEE

Transactions on Automatic Control, 2020.

[7] A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control barrier

function based quadratic programs for safety critical systems,” IEEE

Transactions on Automatic Control, vol. 62, no. 8, pp. 3861–3876, 2016.

[8] P. Jagtap, S. Soudjani, and M. Zamani, “Formal synthesis of

stochastic systems via control barrier certiﬁcates,” arXiv preprint

arXiv:1905.04585, 2019.

[9] P. Jagtap, A. Swikir, and M. Zamani, “Compositional construction

of control barrier functions for interconnected control systems,” in

Proceedings of the 23rd International Conference on Hybrid Systems:

Computation and Control, 2020, pp. 1–11.

[10] C. Huang, X. Chen, W. Lin, Z. Yang, and X. Li, “Probabilistic safety

veriﬁcation of stochastic hybrid systems using barrier certiﬁcates,” ACM

Transactions on Embedded Computing Systems (TECS), vol. 16, no. 5s,

p. 186, 2017.

[11] A. Clark, “Control barrier functions for complete and incomplete in-

formation stochastic systems,” in 2019 American Control Conference

(ACC). IEEE, 2019, pp. 2928–2935.

[12] N. Jahanshahi, P. Jagtap, and M. Zamani, “Synthesis of stochastic

systems with partial information via control barrier functions,” 21st IFAC

World Congress, 2020.

[13] B. Øksendal and A. Sulem, Applied stochastic control of jump diffusions.

Springer Science & Business Media, 2007.

[14] X. Kai, C. Wei, and L. Liu, “Robust extended kalman ﬁltering for

nonlinear systems with stochastic uncertainties,” IEEE Transactions on

Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 40,

no. 2, pp. 399–405, 2009.

[15] B.-S. Chen, W.-H. Chen, and H.-L. Wu, “Robust h2/h∞global

linearization ﬁlter design for nonlinear stochastic systems,” IEEE trans-

actions on circuits and systems I: Regular Papers, vol. 56, no. 7, pp.

1441–1454, 2008.

[16] C.-S. Tseng, “Robust fuzzy ﬁlter design for a class of nonlinear

stochastic systems,” IEEE Transactions on Fuzzy Systems, vol. 15, no. 2,

pp. 261–274, 2007.

[17] C. Baier and J.-P. Katoen, Principles of model checking. MIT press,

2008.

[18] F. Bonchi and D. Pous, “Checking nfa equivalence with bisimulations

up to congruence,” ACM SIGPLAN Notices, vol. 48, no. 1, pp. 457–468,

2013.

[19] J. E. Hopcroft, R. Motwani, and J. D. Ullman, “Introduction to automata

theory, languages, and computation,” Acm Sigact News, vol. 32, no. 1,

pp. 60–65, 2001.

[20] G. De Giacomo and M. Vardi, “Synthesis for ltl and ldl on ﬁnite

traces,” in Twenty-Fourth International Joint Conference on Artiﬁcial

Intelligence, 2015.

[21] T. Wongpiromsarn, U. Topcu, and A. Lamperski, “Automata theory

meets barrier certiﬁcates: Temporal logic veriﬁcation of nonlinear sys-

tems,” IEEE Transactions on Automatic Control, vol. 61, no. 11, pp.

3344–3355, 2015.

[22] H. Kushner, “Stochastic stability and control, ser,” Mathematics in

Science and Engineering. New York: Academic Press, 1967.

[23] S. J. Russell and P. Norvig, Artiﬁcial Intelligence: A Modern Approach,

2nd ed. Pearson Education, 2003.

[24] S. Prajna, A. Papachristodoulou, and P. A. Parrilo, “Introducing sostools:

A general purpose sum of squares programming solver,” in Proceedings

of the 41st IEEE Conference on Decision and Control, 2002., vol. 1.

IEEE, 2002, pp. 741–746.

[25] J. F. Sturm, “Using sedumi 1.02, a matlab toolbox for optimization over

symmetric cones,” Optimization methods and software, vol. 11, no. 1-4,

pp. 625–653, 1999.

[26] M. Krstic and P. V. Kokotovic, “Lean backstepping design for a jet

engine compressor model,” in Proceedings of International Conference

on Control Applications. IEEE, 1995, pp. 1047–1052.