Vojtěch Forejt's research while affiliated with University of Oxford and other places

Publications (39)

Chapter
The model-checking approach was originally formulated for verifying qualitative properties of systems, for example safety and liveness (see Chap. 2), and subsequently extended to also handle quantitative features, such as real time (see Chap. 29), continuous flows (see Chap. 30), as well as stochastic phenomena, where system evolution is governed b...
Article
Full-text available
We study the bisimilarity problem for probabilistic pushdown automata (pPDA) and subclasses thereof. Our definition of pPDA allows both probabilistic and non-deterministic branching, generalising the classical notion of pushdown automata (without epsilon-transitions). We first show a general characterization of probabilistic bisimilarity in terms o...
Article
Bounded-rate multimode systems are hybrid systems that switch freely among a finite set of modes, and whose dynamics are specified by a finite number of real-valued variables with mode-dependent rates that vary within given bounded sets. The scheduler repeatedly proposes a time and a mode, while the environment chooses an allowable rate for that mo...
Article
Full-text available
We study controller synthesis problems for finite-state Markov decision processes, where the objective is to optimize the expected mean-payoff performance and stability (also known as variability in the literature). We argue that the basic notion of expressing the stability using the statistical variance of the mean payoff is sometimes insufficient...
Article
Full-text available
Probabilistic timed automata are a suitable formalism to model systems with real-time, nondeterministic and probabilistic behaviour. We study two-player zero-sum games on such automata where the objective of the game is specified as the expected time to reach a target. The two players---called player Min and player Max---compete by proposing timed...
Conference Paper
We study stochastic two-player turn-based games in which the objective of one player is to ensure several infinite-horizon total reward objectives, while the other player attempts to spoil at least one of the objectives. The games have previously been shown not to be determined, and an approximation algorithm for computing a Pareto curve has been g...
Article
Full-text available
We study graphs and two-player games in which rewards are assigned to states, and the goal of the players is to satisfy or dissatisfy certain property of the generated outcome, given as a mean payoff property. Since the notion of mean-payoff does not reflect possible fluctuations from the mean-payoff along a run, we propose definitions and algorith...
Conference Paper
Full-text available
Quantitative extensions of temporal logics have recently attracted significant attention. In this work, we study frequency LTL (fLTL), an extension of LTL which allows to speak about frequencies of events along an execution. Such an extension is particularly useful for probabilistic systems that often cannot fulfil strict qualitative guarantees on...
Article
Full-text available
We propose novel controller synthesis techniques for probabilistic systems modelled using stochastic two-player games: one player acts as a controller, the second represents its environment, and probability is used to capture uncertainty arising due to, for example, unreliable sensors or faulty system components. Our aim is to generate robust contr...
Article
Full-text available
We study frequency linear-time temporal logic (fLTL) which extends the linear-time temporal logic (LTL) with a path operator G^p allowing to express that on a path, certain formula holds with at least a given frequency $p$, thus relaxing the semantics of the usual $\mathbf{G}$ operator of LTL. Such logic is particularly useful in probabilistic syst...
Conference Paper
Full-text available
We present MultiGain, a tool to synthesize strategies for Markov decision processes (MDPs) with multiple mean-payoff objectives. Our models are described in PRISM, and our tool uses the existing interface and simulator of PRISM. Our tool extends PRISM by adding novel algorithms for multiple mean-payoff objectives, and also provides features such as...
Article
We study the language equivalence problem for probabilistic pushdown automata (pPDA) and their subclasses. We show that the problem is interreducible with the multiplicity equivalence problem for context-free grammars, the decidability of which has been open for several decades. Interreducibility also holds for pPDA with one control state. In contr...
Conference Paper
The Message Passing Interface (MPI) is the standard API for high-performance and scientific computing. Communication deadlocks are a frequent problem in MPI programs, and this paper addresses the problem of discovering such deadlocks. We begin by showing that if an MPI program is single-path, the problem of discovering communication deadlocks is NP...
Conference Paper
Full-text available
We present a general framework for applying machine-learning algorithms to the verification of Markov decision processes (MDPs). The primary goal of these techniques is to improve performance by avoiding an exhaustive exploration of the state space. Our framework focuses on probabilistic reachability, which is a core property for verification, and...
Conference Paper
We study the problem of achieving a given value in Markov decision processes (MDPs) with several independent discounted reward objectives. We consider a generalised version of discounted reward objectives, in which the amount of discounting depends on the states visited and on the objective. This definition extends the usual definition of discounte...
Article
Full-text available
Solvency games, introduced by Berger et al., provide an abstract framework for modelling decisions of a risk-averse investor, whose goal is to avoid ever going broke. We study a new variant of this model, where, in addition to stochastic environment and fixed increments and decrements to the investor's wealth, we introduce interest, which is earned...
Conference Paper
We study two-player stochastic games, where the goal of one player is to satisfy a formula given as a positive boolean combination of expected total reward objectives and the behaviour of the second player is adversarial. Such games are important for modelling, synthesis and verification of open systems with stochastic behaviour. We show that findi...
Article
Full-text available
We study the complexity of central controller synthesis problems for finite-state Markov decision processes, where the objective is to optimize both the expected mean-payoff performance of the system and its stability. We argue that the basic theoretical notion of expressing the stability in terms of the variance of the mean-payoff (called global v...
Conference Paper
Full-text available
We present PRISM-games, a model checker for stochastic multi-player games, which supports modelling, automated verification and strategy synthesis for probabilistic systems with competitive or cooperative behaviour. Models are described in a probabilistic extension of the Reactive Modules language and properties are expressed using rPATL, which ext...
Article
Full-text available
Bounded-rate multi-mode systems (BMMS) are hybrid systems that can switch freely among a finite set of modes, and whose dynamics is specified by a finite number of real-valued variables with mode-dependent rates that can vary within given bounded sets. The schedulability problem for BMMS is defined as an infinite-round game between two players---th...
Article
Full-text available
We study the bisimilarity problem for probabilistic pushdown automata (pPDA) and subclasses thereof. Our definition of pPDA allows both probabilistic and non-deterministic branching, generalising the classical notion of pushdown automata (without epsilon-transitions). Our first contribution is a general construction that reduces checking bisimilari...
Conference Paper
Full-text available
Probabilistic verification techniques have been proposed for runtime analysis of adaptive software systems, with the verification results being used to steer the system so that it satisfies certain Quality-of-Service requirements. Since systems evolve over time, and verification results are required promptly, efficiency is an essential issue. To ad...
Conference Paper
We study stochastic two-player games where the goal of one player is to achieve precisely a given expected value of the objective function, while the goal of the opponent is the opposite. Potential applications for such games include controller synthesis problems where the optimisation objective is to maximise or minimise a given payoff function wh...
Conference Paper
Multi-objective probabilistic model checking provides a way to verify several, possibly conflicting, quantitative properties of a stochastic system. It has useful applications in controller synthesis and compositional probabilistic verification. However, existing methods are based on linear programming, which limits the scale of systems that can be...
Article
We present automatic verification techniques for the modelling and analysis of probabilistic systems that incorporate competitive behaviour. These systems are modelled as turn-based stochastic multi-player games, in which the players can either collaborate or compete in order to achieve a particular goal. We define a temporal logic called rPATL for...
Conference Paper
We study Markov decision processes (MDPs) with multiple limit-average (or mean-payoff) functions. We consider two different objectives, namely, expectation and satisfaction objectives. Given an MDP with kk reward functions, in the expectation objective the goal is to maximize the expected limit- average value, and in the satisfaction objective the...
Article
Full-text available
We study Markov decision processes (MDPs) with multiple limit-average (or mean-payoff) functions. We consider two different objectives, namely, expectation and satisfaction objectives. Given an MDP with k limit-average functions, in the expectation objective the goal is to maximize the expected limit-average value, and in the satisfaction objective...
Conference Paper
Full-text available
We present a verification framework for analysing multiple quantitative objectives of systems that exhibit both nondeterministic and stochastic behaviour. These systems are modelled as probabilistic automata, enriched with cost or reward structures that capture, for example, energy usage or performance metrics. Quantitative properties of these mode...
Conference Paper
Full-text available
This tutorial provides an introduction to probabilistic model checking, a technique for automatically verifying quantitative properties of probabilistic systems. We focus on Markov decision processes (MDPs), which model both stochastic and nondeterministic behaviour. We describe methods to analyse a wide range of their properties, including specifi...
Conference Paper
Full-text available
In an expected reachability-time game (ERTG) two players, Min and Max, move a token along the transitions of a probabilistic timed automaton, so as to minimise and maximise, respectively, the expected time to reach a target. These games are concurrent since at each step of the game both players choose a timed move (a time delay and action under the...
Conference Paper
We define stochastic timed games, which extend two-player timed games with probabilities (following a recent approach by Baier et al), and which extend in a natural way continuous-time Markov decision processes. We focus on the reachability problem for these games, and ask whether one of the players has a strategy to ensure that the probability of...
Article
Full-text available
In this paper we study complexity of the model-checking problem for probabilistic pushdown automata (pPDA) and qualitative fragments of two branching-time logics PCTL* and PECTL*. We prove tha this problem is in 2-EXPTIME for pPDA and qualitative PCTL*. Consequently, we prove that model-checking of stateless pPDA (pBPA) and both qualitative PCTL* a...
Conference Paper
We study continuous-time stochastic games with time-bounded reachability objectives and time-abstract strategies. We show that each vertex in such a game has a value (i.e., an equilibrium probability), and we classify the conditions under which optimal strategies exist. Further, we show how to compute ε-optimal strategies in finite games and provid...
Conference Paper
Full-text available
We show that the controller synthesis and verification problems for Markov decision processes with qualitative PECTL objectives are 2-EXPTIME complete. More precisely, the algorithms are polynomial in the size of a given Markov decision process and doubly exponential in the size of a given qualitative PECTL formula. Moreover, we show that if a give...
Conference Paper
Full-text available
We study the satisfiability problem for qualitative PCTL (probabilistic computation tree logic), which is obtained from "ordinary" CTL by replacing the EX, AX, EU, and AU operators with their qualitative counterparts X <sup>></sup> <sup>0</sup>, X <sup>=</sup> <sup>1</sup>, U <sup>></sup> <sup>0</sup>, and U <sup>=</sup> <sup>1</sup>, respectively....
Article
We consider a class of infinite-state Markov decision processes generated by stateless pushdown automata. This class corresponds to -player games over graphs generated by BPA systems or (equivalently) 1-exit recursive state machines. An extended reachability objective is specified by two sets S and T of safe and terminal stack configurations, where...
Conference Paper
We consider a class of finite 1\frac121\frac{1}{2} -player games (Markov decision processes) where the winning objectives are specified in the branching-time temporal logic qPECTL* (an extension of the qualitative PCTL*). We study decidability and complexity of existence of a winning strategy in these games. We identify a fragment of qPECTL* call...
Conference Paper
We consider stochastic turn-based games where the winning objectives are given by formulae of the branching-time logic PCTL. These games are generally not determined and winning strategies may require memory and or randomization. Our main results concern history-dependent strategies. In particular, we show that the problem whether there exists a hi...
Conference Paper
Full-text available
We consider a class of infinite-state Markov decision processes generated by state- less pushdown automata. This class corresponds to 112-player games over graphs generated by BPA systems or (equivalently) 1-exit recursive state machines. An extended reachability objective is specified by two sets S and T of safe and terminal stack configurations,...

Citations

... MDP can also be considered as the extension of DTMC due to the additional nondeterminism provided by the model apart from probabilistic behavior. 63 It is a mathematical framework that supports the modeling of decision-making in situations. In addition, a part of the outcomes is randomly obtained (probability of state transition) while another part of the outcomes can be defined by the decision-maker (rewards). ...
... Window parity objectives, especially bounded variants, are closely related to the notion of finitary ω-regular games, e.g., [18], and the semantics of prompt-ltl [29]. The window mechanism can be used to ensure a certain form of (local) guarantee over paths; different techniques have been considered in stochastic models [10,13,7]. Timed automata have numerous extensions, e.g., hybrid systems (e.g., [9] and references therein) and probabilistic timed automata (e.g., [32]); the window mechanism could prove useful in these richer settings. Finally, we recall that game models provide a framework for the synthesis of correct-by-construction controllers [34]. ...
... For conjunctive reachability, it is only known that the Pareto set can be approximated [1] with guaranteed precision, even in non-stopping games. For total reward, the problem is proven decidable only for stopping games with two-dimensional queries [9] but it can be approximated in higher dimensional stopping SGs [19]. If only deterministic strategies are allowed, the exact problem is undecidable [19], and so are generalized mean-payoff objectives in SGs [37]. ...
... Related work. The window mechanism has seen numerous extensions in addition to the previously mentioned works, e.g., [5,3,11,15,28,35,8]. Window parity objectives, especially bounded variants, are closely related to the notion of finitary ω-regular games, e.g., [18], and the semantics of prompt-ltl [29]. ...
... Two-player (concurrent) probabilistic timed games were introduced in [24]. The authors demonstrated that such games are not determined (even when all clock constraints are closed) and investigated the complexity of expected time reachability for such games. ...
... Predictive Trace Verification: Dynamic analyses in the form of enumerative and symbolic explorations of the state space have also been studied at length. Symbolic and predictive trace analysis by encoding the thread interleavings of a program is a popular approach that is investigated in [27], [28], [29]. Similar approaches have been applied to the verification of message passing programs [30]. ...
... In recent research, probabilistic verification has taken an important place in the verification process since it allows to drive the system to satisfy the wanted quality of service requirements. In [Forejt et al., 2012], an incremental probabilistic technique has been proposed for the run-time analysis of adaptive software systems. Probability is characteristic of reconfiguration events. ...
... Some main characteristic features of Storm that help to push the state of the art in probabilistic model checking are that Storm -contains efficient implementations of well-known and mature model checking algorithms for discrete-time and continuous-time Markov chains and Markov decision processes, but also for the more general Markov automata [49], a model containing probabilistic branching, nondeterminism, and exponentially distributed delays 2 ; -supports explicit state and symbolic (BDD-based) model checking as well as a mixture of these modes to handle a wider range of models; -has a modular setup, enabling the easy exchange of different solvers and distinct decision diagram packages; its current release supports about 15 solvers and two BDD packages. -extends probabilistic model checking with the possibility of generating (high-level) counterexample [39], synthesizing permissive schedulers [46], symbolic bisimulation minimization [119,121] as well as game-based abstraction of infinite-state MDPs [120]. ...
... Such extensions come at a cost, and for example the "frequency until" operator can make the controller-synthesis problem undecidable already for nonstochastic systems [7,8]. It turns out [19,30,31] that a way of providing significant added expressive power while preserving tractability is to extend LTL only by the "frequency globally" formulae G ≥p ϕ. Such a formula is satisfied if the long-run frequency of satisfying ϕ on an infinite path is at least p. ...
... The LP-based approach of [27] and the iterative approach of [28] are both implemented in PRISM [45] and Storm [40]. The LP formulation of [11,20] is implemented in MultiGain [12], an extension of PRISM for multi-objective LRA rewards. ...