# Vojtěch Forejt's research while affiliated with University of Oxford and other places

**What is this page?**

This page lists the scientific contributions of an author, who either does not have a ResearchGate profile, or has not yet added these contributions to their profile.

It was automatically created by ResearchGate to create a record of this author's body of work. We create such pages to advance our goal of creating and maintaining the most comprehensive scientific repository possible. In doing so, we process publicly available (personal) data relating to the author as a member of the scientific community.

If you're a ResearchGate member, you can follow this page to keep up with this author's work.

If you are this author, and you don't want us to display this page anymore, please let us know.

It was automatically created by ResearchGate to create a record of this author's body of work. We create such pages to advance our goal of creating and maintaining the most comprehensive scientific repository possible. In doing so, we process publicly available (personal) data relating to the author as a member of the scientific community.

If you're a ResearchGate member, you can follow this page to keep up with this author's work.

If you are this author, and you don't want us to display this page anymore, please let us know.

## Publications (39)

The model-checking approach was originally formulated for verifying qualitative properties of systems, for example safety and liveness (see Chap. 2), and subsequently extended to also handle quantitative features, such as real time (see Chap. 29), continuous flows (see Chap. 30), as well as stochastic phenomena, where system evolution is governed b...

We study the bisimilarity problem for probabilistic pushdown automata (pPDA) and subclasses thereof. Our definition of pPDA allows both probabilistic and non-deterministic branching, generalising the classical notion of pushdown automata (without epsilon-transitions). We first show a general characterization of probabilistic bisimilarity in terms o...

Bounded-rate multimode systems are hybrid systems that switch freely among a finite set of modes, and whose dynamics are specified by a finite number of real-valued variables with mode-dependent rates that vary within given bounded sets. The scheduler repeatedly proposes a time and a mode, while the environment chooses an allowable rate for that mo...

We study controller synthesis problems for finite-state Markov decision processes, where the objective is to optimize the expected mean-payoff performance and stability (also known as variability in the literature). We argue that the basic notion of expressing the stability using the statistical variance of the mean payoff is sometimes insufficient...

Probabilistic timed automata are a suitable formalism to model systems with real-time, nondeterministic and probabilistic behaviour. We study two-player zero-sum games on such automata where the objective of the game is specified as the expected time to reach a target. The two players---called player Min and player Max---compete by proposing timed...

We study stochastic two-player turn-based games in which the objective of one player is to ensure several infinite-horizon total reward objectives, while the other player attempts to spoil at least one of the objectives. The games have previously been shown not to be determined, and an approximation algorithm for computing a Pareto curve has been g...

We study graphs and two-player games in which rewards are assigned to states, and the goal of the players is to satisfy or dissatisfy certain property of the generated outcome, given as a mean payoff property. Since the notion of mean-payoff does not reflect possible fluctuations from the mean-payoff along a run, we propose definitions and algorith...

Quantitative extensions of temporal logics have recently attracted
significant attention. In this work, we study frequency LTL (fLTL), an
extension of LTL which allows to speak about frequencies of events along an
execution. Such an extension is particularly useful for probabilistic systems
that often cannot fulfil strict qualitative guarantees on...

We propose novel controller synthesis techniques for probabilistic systems
modelled using stochastic two-player games: one player acts as a controller,
the second represents its environment, and probability is used to capture
uncertainty arising due to, for example, unreliable sensors or faulty system
components. Our aim is to generate robust contr...

We study frequency linear-time temporal logic (fLTL) which extends the
linear-time temporal logic (LTL) with a path operator G^p allowing to express
that on a path, certain formula holds with at least a given frequency $p$, thus
relaxing the semantics of the usual $\mathbf{G}$ operator of LTL. Such logic is
particularly useful in probabilistic syst...

We present MultiGain, a tool to synthesize strategies for Markov decision
processes (MDPs) with multiple mean-payoff objectives. Our models are described
in PRISM, and our tool uses the existing interface and simulator of PRISM. Our
tool extends PRISM by adding novel algorithms for multiple mean-payoff
objectives, and also provides features such as...

We study the language equivalence problem for probabilistic pushdown automata (pPDA) and their subclasses. We show that the problem is interreducible with the multiplicity equivalence problem for context-free grammars, the decidability of which has been open for several decades. Interreducibility also holds for pPDA with one control state.
In contr...

The Message Passing Interface (MPI) is the standard API for high-performance and scientific computing. Communication deadlocks are a frequent problem in MPI programs, and this paper addresses the problem of discovering such deadlocks. We begin by showing that if an MPI program is single-path, the problem of discovering communication deadlocks is NP...

We present a general framework for applying machine-learning algorithms to
the verification of Markov decision processes (MDPs). The primary goal of these
techniques is to improve performance by avoiding an exhaustive exploration of
the state space. Our framework focuses on probabilistic reachability, which is
a core property for verification, and...

We study the problem of achieving a given value in Markov decision processes (MDPs) with several independent discounted reward objectives. We consider a generalised version of discounted reward objectives, in which the amount of discounting depends on the states visited and on the objective. This definition extends the usual definition of discounte...

Solvency games, introduced by Berger et al., provide an abstract framework
for modelling decisions of a risk-averse investor, whose goal is to avoid ever
going broke. We study a new variant of this model, where, in addition to
stochastic environment and fixed increments and decrements to the investor's
wealth, we introduce interest, which is earned...

We study two-player stochastic games, where the goal of one player is to satisfy a formula given as a positive boolean combination of expected total reward objectives and the behaviour of the second player is adversarial. Such games are important for modelling, synthesis and verification of open systems with stochastic behaviour. We show that findi...

We study the complexity of central controller synthesis problems for
finite-state Markov decision processes, where the objective is to optimize both
the expected mean-payoff performance of the system and its stability.
We argue that the basic theoretical notion of expressing the stability in
terms of the variance of the mean-payoff (called global v...

We present PRISM-games, a model checker for stochastic multi-player games, which supports modelling, automated verification and strategy synthesis for probabilistic systems with competitive or cooperative behaviour. Models are described in a probabilistic extension of the Reactive Modules language and properties are expressed using rPATL, which ext...

Bounded-rate multi-mode systems (BMMS) are hybrid systems that can switch
freely among a finite set of modes, and whose dynamics is specified by a finite
number of real-valued variables with mode-dependent rates that can vary within
given bounded sets. The schedulability problem for BMMS is defined as an
infinite-round game between two players---th...

We study the bisimilarity problem for probabilistic pushdown automata (pPDA)
and subclasses thereof. Our definition of pPDA allows both probabilistic and
non-deterministic branching, generalising the classical notion of pushdown
automata (without epsilon-transitions). Our first contribution is a general
construction that reduces checking bisimilari...

Probabilistic verification techniques have been proposed for runtime analysis of adaptive software systems, with the verification results being used to steer the system so that it satisfies certain Quality-of-Service requirements. Since systems evolve over time, and verification results are required promptly, efficiency is an essential issue. To ad...

We study stochastic two-player games where the goal of one player is to achieve precisely a given expected value of the objective function, while the goal of the opponent is the opposite. Potential applications for such games include controller synthesis problems where the optimisation objective is to maximise or minimise a given payoff function wh...

Multi-objective probabilistic model checking provides a way to verify
several, possibly conflicting, quantitative properties of a stochastic system.
It has useful applications in controller synthesis and compositional
probabilistic verification. However, existing methods are based on linear
programming, which limits the scale of systems that can be...

We present automatic verification techniques for the modelling and analysis of probabilistic systems that incorporate competitive behaviour. These systems are modelled as turn-based stochastic multi-player games, in which the players can either collaborate or compete in order to achieve a particular goal. We define a temporal logic called rPATL for...

We study Markov decision processes (MDPs) with multiple limit-average (or mean-payoff) functions. We consider two different objectives, namely, expectation and satisfaction objectives. Given an MDP with kk reward functions, in the expectation objective the goal is to maximize the expected limit- average value, and in the satisfaction objective the...

We study Markov decision processes (MDPs) with multiple limit-average (or
mean-payoff) functions. We consider two different objectives, namely,
expectation and satisfaction objectives. Given an MDP with k limit-average
functions, in the expectation objective the goal is to maximize the expected
limit-average value, and in the satisfaction objective...

We present a verification framework for analysing multiple quantitative objectives of systems that exhibit both nondeterministic
and stochastic behaviour. These systems are modelled as probabilistic automata, enriched with cost or reward structures that
capture, for example, energy usage or performance metrics. Quantitative properties of these mode...

This tutorial provides an introduction to probabilistic model checking, a technique for automatically verifying quantitative
properties of probabilistic systems. We focus on Markov decision processes (MDPs), which model both stochastic and nondeterministic
behaviour. We describe methods to analyse a wide range of their properties, including specifi...

In an expected reachability-time game (ERTG) two players, Min and Max, move a token along the transitions of a probabilistic timed automaton, so as to minimise and maximise, respectively, the expected time to reach a target. These games are concurrent since at each step of the game both players choose a timed move (a time delay and action under the...

We define stochastic timed games, which extend two-player timed games with probabilities (following a recent approach by Baier et al), and which extend in a natural way continuous-time Markov decision processes. We focus on the reachability problem for these games, and ask whether one of the players has a strategy to ensure that the probability of...

In this paper we study complexity of the model-checking problem for probabilistic pushdown automata (pPDA) and qualitative fragments of two branching-time logics PCTL* and PECTL*. We prove tha this problem is in 2-EXPTIME for pPDA and qualitative PCTL*. Consequently, we prove that model-checking of stateless pPDA (pBPA) and both qualitative PCTL* a...

We study continuous-time stochastic games with time-bounded reachability objectives and time-abstract strategies. We show that each vertex in such a game has a value (i.e., an equilibrium probability), and we classify the conditions under which optimal strategies exist. Further, we show how to compute ε-optimal strategies in finite games and provid...

We show that the controller synthesis and verification problems for Markov decision processes with qualitative PECTL objectives are 2-EXPTIME complete. More precisely, the algorithms are polynomial in the size of a given Markov decision process and doubly exponential in the size of a given qualitative PECTL formula. Moreover, we show that if a give...

We study the satisfiability problem for qualitative PCTL (probabilistic computation tree logic), which is obtained from "ordinary" CTL by replacing the EX, AX, EU, and AU operators with their qualitative counterparts X <sup>></sup> <sup>0</sup>, X <sup>=</sup> <sup>1</sup>, U <sup>></sup> <sup>0</sup>, and U <sup>=</sup> <sup>1</sup>, respectively....

We consider a class of infinite-state Markov decision processes generated by stateless pushdown automata. This class corresponds to -player games over graphs generated by BPA systems or (equivalently) 1-exit recursive state machines. An extended reachability objective is specified by two sets S and T of safe and terminal stack configurations, where...

We consider a class of finite
1\frac121\frac{1}{2}
-player games (Markov decision processes) where the winning objectives are specified in the branching-time temporal logic
qPECTL* (an extension of the qualitative PCTL*). We study decidability and complexity of existence of a winning strategy in
these games. We identify a fragment of qPECTL* call...

We consider stochastic turn-based games where the winning objectives are given by formulae of the branching-time logic PCTL. These games are generally not determined and winning strategies may require memory and or randomization. Our main results concern history-dependent strategies. In particular, we show that the problem whether there exists a hi...

We consider a class of infinite-state Markov decision processes generated by state- less pushdown automata. This class corresponds to 112-player games over graphs generated by BPA systems or (equivalently) 1-exit recursive state machines. An extended reachability objective is specified by two sets S and T of safe and terminal stack configurations,...

## Citations

... MDP can also be considered as the extension of DTMC due to the additional nondeterminism provided by the model apart from probabilistic behavior. 63 It is a mathematical framework that supports the modeling of decision-making in situations. In addition, a part of the outcomes is randomly obtained (probability of state transition) while another part of the outcomes can be defined by the decision-maker (rewards). ...

... Window parity objectives, especially bounded variants, are closely related to the notion of finitary ω-regular games, e.g., [18], and the semantics of prompt-ltl [29]. The window mechanism can be used to ensure a certain form of (local) guarantee over paths; different techniques have been considered in stochastic models [10,13,7]. Timed automata have numerous extensions, e.g., hybrid systems (e.g., [9] and references therein) and probabilistic timed automata (e.g., [32]); the window mechanism could prove useful in these richer settings. Finally, we recall that game models provide a framework for the synthesis of correct-by-construction controllers [34]. ...

... For conjunctive reachability, it is only known that the Pareto set can be approximated [1] with guaranteed precision, even in non-stopping games. For total reward, the problem is proven decidable only for stopping games with two-dimensional queries [9] but it can be approximated in higher dimensional stopping SGs [19]. If only deterministic strategies are allowed, the exact problem is undecidable [19], and so are generalized mean-payoff objectives in SGs [37]. ...

... Related work. The window mechanism has seen numerous extensions in addition to the previously mentioned works, e.g., [5,3,11,15,28,35,8]. Window parity objectives, especially bounded variants, are closely related to the notion of finitary ω-regular games, e.g., [18], and the semantics of prompt-ltl [29]. ...

... Two-player (concurrent) probabilistic timed games were introduced in [24]. The authors demonstrated that such games are not determined (even when all clock constraints are closed) and investigated the complexity of expected time reachability for such games. ...

... Predictive Trace Verification: Dynamic analyses in the form of enumerative and symbolic explorations of the state space have also been studied at length. Symbolic and predictive trace analysis by encoding the thread interleavings of a program is a popular approach that is investigated in [27], [28], [29]. Similar approaches have been applied to the verification of message passing programs [30]. ...

... In recent research, probabilistic verification has taken an important place in the verification process since it allows to drive the system to satisfy the wanted quality of service requirements. In [Forejt et al., 2012], an incremental probabilistic technique has been proposed for the run-time analysis of adaptive software systems. Probability is characteristic of reconfiguration events. ...

... Some main characteristic features of Storm that help to push the state of the art in probabilistic model checking are that Storm -contains efficient implementations of well-known and mature model checking algorithms for discrete-time and continuous-time Markov chains and Markov decision processes, but also for the more general Markov automata [49], a model containing probabilistic branching, nondeterminism, and exponentially distributed delays 2 ; -supports explicit state and symbolic (BDD-based) model checking as well as a mixture of these modes to handle a wider range of models; -has a modular setup, enabling the easy exchange of different solvers and distinct decision diagram packages; its current release supports about 15 solvers and two BDD packages. -extends probabilistic model checking with the possibility of generating (high-level) counterexample [39], synthesizing permissive schedulers [46], symbolic bisimulation minimization [119,121] as well as game-based abstraction of infinite-state MDPs [120]. ...

Reference: The probabilistic model checker Storm

... Such extensions come at a cost, and for example the "frequency until" operator can make the controller-synthesis problem undecidable already for nonstochastic systems [7,8]. It turns out [19,30,31] that a way of providing significant added expressive power while preserving tractability is to extend LTL only by the "frequency globally" formulae G ≥p ϕ. Such a formula is satisfied if the long-run frequency of satisfying ϕ on an infinite path is at least p. ...

... The LP-based approach of [27] and the iterative approach of [28] are both implemented in PRISM [45] and Storm [40]. The LP formulation of [11,20] is implemented in MultiGain [12], an extension of PRISM for multi-objective LRA rewards. ...