Chapter

Striver: Stream Runtime Verification for Real-Time Event-Streams: 18th International Conference, RV 2018, Limassol, Cyprus, November 10–13, 2018, Proceedings

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

We study the problem of monitoring rich properties of real-time event streams, and propose a solution based on Stream Runtime Verification (SRV), where observations are described as output streams of data computed from input streams of data. SRV allows a clean separation between the temporal dependencies among incoming events, and the concrete operations that are performed during the monitoring.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... In the context of CPS, timed regular expressions (TRE) [6,7], quantitative regular expressions (QRE) [2][3][4]19], Signal Temporal Logic (STL) [18] and various stream languages [10,11,[15][16][17] have been used as popular formalisms for specifying properties of CPS behaviors. QREs are a powerful formalism that combines quantitative computations over data with regular expression-based matching. ...
... 9, 17, 7, −5) in ϕ pulse that instantiates the ideal shape (red line) of the first pulse depicted in Fig. 1. Let w 1 = w [7,12) , w 2 = w [12,15) , w 3 = w [15,18) , w 4 = w [18,21) and w 5 = w [21,26) , with: ...
... 9, 17, 7, −5) in ϕ pulse that instantiates the ideal shape (red line) of the first pulse depicted in Fig. 1. Let w 1 = w [7,12) , w 2 = w [12,15) , w 3 = w [15,18) , w 4 = w [18,21) and w 5 = w [21,26) , with: ...
Article
Full-text available
Modern cyber-physical systems (CPS) and the Internet of things (IoT) are data factories generating, measuring and recording huge amounts of time series. The useful information in time series is usually present in the form of sequential patterns. We propose shape expressions as a declarative language for specification and extraction of rich temporal patterns from possibly noisy data. Shape expressions are regular expressions with arbitrary (linear, exponential, sinusoidal, etc.) shapes with parameters as atomic predicates and additional constraints on these parameters. We associate with shape expressions novel noisy semantics that combines regular expression matching semantics with statistical regression. We study essential properties of the language and propose an efficient heuristic for approximate matching of shape expressions. We demonstrate the applicability of this technique on two case studies from the health and the avionics domains.
... Examples include counting events, specifying robustness or generating models or This work was funded in part by the Madrid Regional Government under project "S2018/TCS-4339 (BLOQUES-CM)", by EU H2020 project 731535 "Elastest" and by Spanish National Project "BOSCO (PGC2018-102210-B-100)". quantitative verdicts. See [10,14,17] for examples illustrating the expressivity of SRV languages. ...
... All these approaches consider only Boolean verdicts. SRV can generate verdicts from arbitrary data domains, but all previous SRV efforts, from Lola [10], Lola2.0 [14], Copilot [21,22] and extensions to timed event streams, like TeSSLa [8], RT-Lola [15] or Striver [17] assume a centralized monitoring setting. ...
Conference Paper
We study the problem of decentralized monitoring of stream runtime verification specifications. Decentralized monitoring uses distributed monitors that communicate via a synchronous network, a communication setting common in many cyber-physical systems like automotive CPSs. Previous approaches to decentralized monitoring were restricted to logics like LTL logics that provide Boolean verdicts. We solve here the decentralized monitoring problem for the more general setting of stream runtime verification. Additionally, our solution handles network topologies while previous decentralized monitoring works assumed that every pair of nodes can communicate directly. We also introduce a novel property on specifications, called decentralized efficient monitorability, that guarantees that the online monitoring can be performed with bounded resources. Finally, we report the results of an empirical evaluation of an implementation and compare the expressive power and efficiency against state-of-the-art decentralized monitoring tools like Themis.
... e main difference between RTL and Striver [21] is that RTL has both variable-rate and fixed-rate streams and provides convenient, native operators such as sample-and-hold and sliding windows that translate between the two types of streams. e fixed rate in RTL allows for a more direct translation to a hardware implementation of the monitor. ...
Preprint
Full-text available
An essential part of cyber-physical systems is the online evaluation of real-time data streams. Especially in systems that are intrinsically safety-critical, a dedicated monitoring component inspecting data streams to detect problems at runtime greatly increases the confidence in a safe execution. Such a monitor needs to be based on a specification language capable of expressing complex, high-level properties using only the accessible low-level signals. Moreover, tight constraints on computational resources exacerbate the requirements on the monitor. Thus, several existing approaches to monitoring are not applicable due to their dependence on an operating system. We present an FPGA-based monitoring approach by compiling an RTLola specification into synthesizable VHDL code. RTLola is a stream-based specification language capable of expressing complex real-time properties while providing an upper bound on the execution time and memory requirements. The statically determined memory bound allows for a compilation to an FPGA with a fixed size. An advantage of FPGAs is a simple integration process in existing systems and superb executing time. The compilation results in a highly parallel implementation thanks to the modular nature of RTLola specifications. This further increases the maximal event rate the monitor can handle.
... It is not clear how one can define the event rate in the non-metric trace model they consider. Metric extensions of Lola [25,34] require subsequent time-stamps to strictly increase, which limits the event rate in the discrete time setting. ...
Article
Full-text available
A monitoring algorithm is trace-length independent if its space consumption does not depend on the number of events processed. The analysis of many monitoring algorithms has aimed at establishing their trace-length independence. But a monitor’s space consumption can depend on characteristics of the trace other than its size. We put forward the stronger notion of event-rate independence, where a monitor’s space usage does not depend on the event rate, i.e., the number of events in a fixed time unit. This property is critical for monitoring voluminous streams of events with a high arrival rate. We propose a new algorithm for metric temporal logic (MTL) that is almost event-rate independent, where “almost” denotes a logarithmic dependence on the event rate: the algorithm must store the event rate as a number. Afterwards, we investigate more expressive logics. In particular, we extend linear dynamic logic with past operators and metric features. The resulting metric dynamic logic (MDL) offers the quantitative temporal conveniences of MTL while increasing its expressiveness. We show how to modify our MTL algorithm in a modular way, yielding an almost event-rate independent monitor for MDL. Finally, we compare our algorithms with traditional monitoring approaches, providing empirical evidence that almost event-rate independence matters in practice.
... Somewhat surprisingly, visualization has not played a major role in monitoring research before. Despite a wide range of monitoring approaches, from formal logic [9,12,17,25] to stream-based specification languages [6,7,10,15], most tools have in common that they rely on textual, rather than visual, methods for data presentation. This paper shows that stream-based monitoring languages like RTLola are very well suited to carry out the needed data processing for useful visualizations. ...
Preprint
Stream-based runtime monitors are used in safety-critical applications such as Unmanned Aerial Systems (UAS) to compute comprehensive statistics and logical assessments of system health that provide the human operator with critical information in hand-over situations. In such applications, a visual display of the monitoring data can be much more helpful than the textual alerts provided by a more traditional user interface. This visualization requires extensive real-time data processing, which includes the synchronization of data from different streams, filtering and aggregation, and priorization and management of user attention. We present a visualization approach for the \rtlola monitoring framework. Our approach is based on the principle that the necessary data processing is the responsibility of the monitor itself, rather than the responsibility of some external visualization tool. We show how the various aspects of the data transformation can be described as RTLola stream equations and linked to the visualization component through a bidirectional synchronous interface. In our experience, this approach leads to highly informative visualizations as well as to understandable and easily maintainable monitoring code.
... Unlike RTLola, BeepBeep and Copilot assume a synchronous computation model, where all events arrive at a fixed rate. Two asynchronous real-time monitoring approaches are TeSSLa [19] and Striver [13]. TeSSLa allows for monitoring piece-wise constant signals where streams can emit events at different speeds with arbitrary latencies. ...
Chapter
With ever increasing autonomy of cyber-physical systems, monitoring becomes an integral part for ensuring the safety of the system at runtime. \(\text {StreamLAB} \) is a monitoring framework with high degree of expressibility and strong correctness guarantees. Specifications are written in \(\text {RTLola} \), a stream-based specification language with formal semantics. \(\text {StreamLAB} \) provides an extensive analysis of the specification, including the computation of memory consumption and run-time guarantees. We demonstrate the applicability of \(\text {StreamLAB} \) on typical monitoring tasks for cyber-physical systems, such as sensor validation and system health checks.
... Our focus on RTLola is motivated by recent work on RTLola-based monitoring for UAS [2] and other cyber-physical systems [3,7]. It should be possible, however, to develop similar optimizations for other stream-based monitoring languages like TeSSLa [5] and Striver [11]. ...
Preprint
Runtime monitors that are specified in a stream-based monitoring language tend to be easier to understand, maintain, and reuse than those written in a standard programming language. Because of their formal semantics, such specification languages are also a natural choice for safety-critical applications. Unlike for standard programming languages, there is, however, so far very little support for automatic code optimization. In this paper, we present the first collection of code transformations for the stream-based monitoring language RTLola. We show that classic compiler optimizations, such as Sparse Conditional Constant Propagation and Common Subexpression Elimination, can be adapted to monitoring specifications. We also develop new transformations -- Pacing Type Refinement and Filter Refinement -- which exploit the specific modular structure of RTLola as well as the implementation freedom afforded by a declarative specification language. We demonstrate the significant impact of the code transformations on benchmarks from the monitoring of unmanned aircraft systems (UAS).
... Further work shall be done to provide a suitable specification language to interactively verify distributed systems. We want to study the combination of decentralized monitoring [EF17] and stream runtime verification [GS18] with our approach. ...
Thesis
Computers are ubiquitous.We trust them for a huge and increasing number of tasks, some critical.Consequences of software defects are various, from little annoyances to the loss of multiple lives.Hence, ensuring software reliability is instrumental.Fixing bugs is a very time-consuming activity of the software development cycle.In this thesis, we present interactive runtime verification (i-RV), which combines runtime verification and interactive debugging.Runtime verification is a formal method to study the behavior of a system at runtime.It consists in matching runtime traces of a system at runtime against behavioral properties.These properties are part of the system specification.Interactive debugging consists in studying a system at runtime in order to understand its bugs and fix them, inspecting its internal state interactively.Interactive runtime verification aims to make interactive debugging less tedious and more systematic by leveraging the rigorous and automated aspects of runtime verification.We aim to ease the debugging part of the software development cycle.We define an efficient and convenient way to check behavioral properties automatically on a program using an interactive debugger.We gather bug detection and bug understanding in an integrated workflow, by guiding interactive debugging using runtime verification.We provide a formal model for interactively runtime verified programs.We model the execution of a program under a debugger composed with a monitor (for verdict emission) and a scenario (for steering the debugging session).We provide guarantees on the soundness of the verdicts issued by the monitor by exhibiting a weak simulation (relation) between the initial program and the interactively runtime verified program.Moreover, we provide an algorithmic view of this model suitable for producing implementations.We then introduce a distributed and adaptive framework for interactive runtime verification.It allows checking several requirements simultaneously and debugging a distributed system composed of communicating processes.The monitors, the scenario and the debugged programs themselves run distributed using an orchestrating protocol which we verified using the SPIN model checker.Our distributed framework is designed to adapt to existing components.We present Verde, an implementation of interactive runtime verification.A first version is based on the GNU Debugger (GDB) to interactively runtime verify C and C++ programs.A second version, Dist-Verde, is an implementation of our distributed framework compatible with C and C++ programs through GDB and Java programs through JDB, the Java Debugger.We report on experiments using Verde assessing the usefulness of interactive runtime verification and the performance of our implementation.Our results show that interactive runtime verification is applicable in a variety of cases and helps to study bugs.
... For more expressive specification languages, establishing the correctness of the monitor is much more challenging. Especially problematic is the use of interpreters, which read the specification as input and then rely on complicated and error-prone software to interpret the specification dynamically at runtime [5,11,[13][14][15]. Recently, however, much effort has gone into the development of compilers. ...
Preprint
The safety of cyber-physical systems rests on the correctness of their monitoring mechanisms. This is problematic if the specification of the monitor is implemented manually or interpreted by unreliable software. We present a verifying compiler that translates specifications given in the stream-based monitoring language Lola to implementations in Rust. The generated code contains verification annotations that enable the Viper toolkit to automatically prove functional correctness, absence of memory faults, and guaranteed termination. The compiler parallelizes the evaluation of different streams in the monitor based on a dependency analysis of the specification. We present encouraging experimental results obtained with monitor specifications found in the literature. For every specification, our approach was able to either produce a correctness proof or to uncover errors in the specification.
... Moreover, RTLola comes with the feature of computing aggregations over sliding windows, and allows for the decoupling of the computation of output streams from variable input event rates via fixed-rate clocks. The main difference between RTLola and Striver [21] is that RT-Lola has both variable-rate and fixed-rate streams and provides convenient, native operators such as sample-and-hold and sliding windows that translate between the two types of streams. The fixed rate in RTLola allows for a more direct translation to a hardware implementation of the monitor. ...
Article
An essential part of cyber-physical systems is the online evaluation of real-time data streams. Especially in systems that are intrinsically safety-critical, a dedicated monitoring component inspecting data streams to detect problems at runtime greatly increases the confidence in a safe execution. Such a monitor needs to be based on a specification language capable of expressing complex, high-level properties using only the accessible low-level signals. Moreover, tight constraints on computational resources exacerbate the requirements on the monitor. Thus, several existing approaches to monitoring are not applicable due to their dependence on an operating system. We present an FPGA-based monitoring approach by compiling an RTLola specification into synthesizable VHDL code. RTLola is a stream-based specification language capable of expressing complex real-time properties while providing an upper bound on the execution time and memory requirements. The statically determined memory bound allows for a compilation to an FPGA with a fixed size. An advantage of FPGAs is a simple integration process in existing systems and superb executing time. The compilation results in a highly parallel implementation thanks to the modular nature of RTLola specifications. This further increases the maximal event rate the monitor can handle.
... SRV offers declarative specifications where offset expressions allow accessing streams at different moments in time, including future instants. Most previous SRV developments [9,11] and their extensions to event-based systems [8,11,12,17] focus on efficiently implementing the temporal engine, promising that new datatypes can be incorporated easily. However, in practice, adding a datatype requires modifying the parser, the internal representation and the runtime system. ...
Chapter
Full-text available
We present , an extensible Stream Runtime Verification (SRV) tool, that borrows from the functional language Haskell (1) rich types for data in events and verdicts; and (2) functional features for parametrization, libraries, high-order specification transformations, etc. SRV is a formal dynamic analysis technique that generalizes Runtime Verification (RV) algorithms from temporal logics like LTL to stream monitoring, allowing the computation of verdicts richer than Booleans (quantitative values and beyond). The keystone of SRV is the clean separation between temporal dependencies and data computations. However, in spite of this theoretical separation previous engines include hardwired implementations of just a few datatypes, requiring complex changes in the tool chain to incorporate new data types. Additionally, when previous tools implement features like parametrization these are implemented in an ad-hoc way. In contrast, is implemented as a Haskell embedded DSL, borrowing datatypes and functional aspects from Haskell, resulting in an extensible engine (The tool is available open-source at http://github.com/imdea-software/hlola ). We illustrate through several examples, including a UAV monitoring infrastructure with predictive characteristics that has been validated in online runtime verification in real mission planning.
... Stream-based systems such as stream-based RV [18,40] and CEP are bottom-up. Data in streams is eventually aggregated into more complex information and relayed to a higher level. ...
Article
Full-text available
We use runtime verification (RV) to check various specifications in a smart apartment. The specifications can be broken down into three types: behavioral correctness of the apartment sensors, detection of specific user activities (known as activities of daily living), and composition of specifications of the previous types. The context of the smart apartment provides us with a complex system with a large number of components with two different hierarchies to group specifications and sensors: geographically within the same room, floor or globally in the apartment, and logically following the different types of specifications. We leverage a recent approach to decentralized RV of decentralized specifications, where monitors have their own specifications and communicate together to verify more general specifications. We leverage the hierarchies, modularity and re-use afforded by decentralized specifications to: (1) scale beyond existing centralized RV techniques, and (2) greatly reduce computation and communication costs.
... Further work shall be done to provide a suitable specification language to interactively verify distributed systems. We want to study the combination of decentralized monitoring [EF17] and stream runtime verification [GS18] with our approach. ...
Thesis
Les ordinateurs sont partout. Nous leur faisons confiance pour un grand, et grandissant, nombre de tâches, parmi lesquelles certaines sont critiques. Les conséquences des bogues logiciels sont diverses, de l'agacement léger à la mort de plusieurs personnes. Ainsi, il est important de s'assurer que les logiciels sont fiables.Corriger les bogues est une activité très chronophage du processus de développement logiciel. Dans cette thèse, nous présentons la vérification interactive à l'exécution, qui combine la vérification à l'exécution et le débogage interactif. La vérification à l'exécution est une méthode formelle pour étudier le comportement d'un système à l'exécution. Elle consiste à faire correspondre des traces d'exécutions d'un système avec des propriétés comportementales. Ces propriétés font partie des spécifications du système. Le débogage interactif consiste à étudier un système durant son exécution pour comprendre ses bogues et les corriger en inspectant interactivement son état interne. La vérification interactive à l'exécution a pour objectif de rendre le débogage interactif moins fastidieux et plus systématique en s'appuyant sur les aspects automatiques et rigoureux de la vérification à l'exécution. Nous avons pour but de faciliter la partie débogage du processus de développement logiciel. Nous définissons une manière efficace et pratique de vérifier des propriétés comportementales automatiquement sur un programme en utilisant un débogueur interactif. Nous rassemblons la détection et la compréhension de bogues dans une méthodologie intégrée, en guidant le débogage interactif avec la vérification à l'exécution.Nous fournissons un modèle formel pour les programmes vérifiés interactivement à l'exécution. Nous modélisons l'exécution d'un programme en train d'être débogué, composé avec un moniteur (pour l'émission de verdicts) et un scénario (pour conduire la session de débogage). Nous fournissons des garanties sur la validité des verdicts produits par le moniteur en nous appuyant sur une relation de simulation faible entre le programme initial et le programme vérifié interactivement. De plus, nous fournissons une vue algorithmique du modèle adaptée à l'écriture d'implémentations. Nous introduisons ensuite un cadre et une architecture distribuée pour la vérification interactive à l'exécution. Cela permet de vérifier plusieurs propriétés simultanément et de déboguer un système distribué, composé de multiples processus communicants. Les moniteurs, le scénario et les programmes débogués eux-mêmes s'exécutent de façon distribuée en utilisant un protocole que nous avons vérifié avec le vérificateur de modèles SPIN. Notre architecture distribuée est conçue pour s'adapter à des composants existants.Nous présentons Verde, une implémentation de la vérification interactive à l'exécution. Une première version est basée sur le débogueur GNU (GDB) pour vérifier interactivement des programmes C et C++. Une deuxième version, Dist-Verde, est une implémentation de notre architecture distribuée compatible avec les programmes C et C++ à travers GDB et les programmes Java à travers JDB, le débogueur Java.Nous présentons des expérimentations en utilisant Verde, évaluant l'utilité de l'approche et les performances de notre implémentation. Nos résultats montrent que la vérification interactive à l'exécution est applicable dans une variété de cas et aide à étudier les bogues.
Chapter
Stream-based runtime monitors are used in safety-critical applications such as Unmanned Aerial Systems (UAS) to compute comprehensive statistics and logical assessments of system health that provide the human operator with critical information in hand-over situations. In such applications, a visual display of the monitoring data can be much more helpful than the textual alerts provided by a more traditional user interface. This visualization requires extensive real-time data processing, which includes the synchronization of data from different streams, filtering and aggregation, and priorization and management of user attention. We present a visualization approach for the RTLola monitoring framework. Our approach is based on the principle that the necessary data processing is the responsibility of the monitor itself, rather than the responsibility of some external visualization tool. We show how the various aspects of the data transformation can be described as RTLola stream equations and linked to the visualization component through a bidirectional synchronous interface. In our experience, this approach leads to highly informative visualizations as well as to understandable and easily maintainable monitoring code.
Chapter
We present a novel approach to the offline monitoring of specifications expressed in metric temporal logic (MTL). Our monitoring algorithm exploits multiple one-way reading heads that traverse a trace sequentially. We present both theoretical and practical results that show this substantially improves upon the state-of-the-art. In particular, our algorithm is the first offline monitoring algorithm for MTL with past and bounded-future temporal operators that is almost trace-length independent and outputs a trace of Boolean verdicts denoting the monitored formula’s satisfaction at every position in the input trace. In addition, our algorithm’s worst-case space complexity is linear in the formula size, while previous algorithms were exponential. Moreover, we compare our implementation of the algorithm with another almost trace-length independent tool that outputs non-standard verdicts to achieve this space complexity. Our tool used less memory and runs significantly faster, for example yielding a 10-fold improvement on average on random formulas, while producing better output.
Chapter
The safety of cyber-physical systems rests on the correctness of their monitoring mechanisms. This is problematic if the specification of the monitor is implemented manually or interpreted by unreliable software. We present a verifying compiler that translates specifications given in the stream-based monitoring language Lola to implementations in Rust. The generated code contains verification annotations that enable the Viper toolkit to automatically prove functional correctness, absence of memory faults, and guaranteed termination. The compiler parallelizes the evaluation of different streams in the monitor based on a dependency analysis of the specification. We present encouraging experimental results obtained with monitor specifications found in the literature. For every specification, our approach was able to either produce a correctness proof or to uncover errors in the specification.
Chapter
Runtime Verification (RV) studies how to analyze execution traces of a system under observation. Stream Runtime Verification (SRV) applies stream transformations to obtain information from observed traces. Incomplete traces with information missing in gaps pose a common challenge when applying RV and SRV techniques to real-world systems as RV approaches typically require the complete trace without missing parts. This paper presents a solution to perform SRV on incomplete traces based on abstraction. We use TeSSLa as specification language for non-synchronized timed event streams and define abstract event streams representing the set of all possible traces that could have occurred during gaps in the input trace. We show how to translate a TeSSLa specification to its abstract counterpart that can propagate gaps through the transformation of the input streams and thus generate sound outputs even if the input streams contain gaps and events with imprecise values. The solution has been implemented as a set of macros for the original TeSSLa and an empirical evaluation shows the feasibility of the approach.
Chapter
In stream-based runtime monitoring, streams of data, called input streams, which involve data collected from the system at runtime, are translated into new streams of data, called output streams, which define statistical measures and verdicts on the system based on the input data. The advantage of this setup is an easy-to-use and modular way for specifying monitors with rich verdicts, provided with formal guarantees on the complexity of the monitor.
Chapter
Cyber-physical systems (CPS) and the Internet-of-Things (IoT) result in a tremendous amount of generated, measured and recorded time-series data. Extracting temporal segments that encode patterns with useful information out of these huge amounts of data is an extremely difficult problem. We propose shape expressions as a declarative formalism for specifying, querying and extracting sophisticated temporal patterns from possibly noisy data. Shape expressions are regular expressions with arbitrary (linear, exponential, sinusoidal, etc.) shapes with parameters as atomic predicates and additional constraints on these parameters. We equip shape expressions with a novel noisy semantics that combines regular expression matching semantics with statistical regression. We characterize essential properties of the formalism and propose an efficient approximate shape expression matching procedure. We demonstrate the wide applicability of this technique on two case studies.
Chapter
We develop a monitoring algorithm for metric dynamic logic, an extension of metric temporal logic with regular expressions. The monitor computes whether a given formula is satisfied at every position in an input trace of time-stamped events. Our monitor follows the multi-head paradigm: it reads the input simultaneously at multiple positions and moves its reading heads asynchronously. This mode of operation results in unprecedented space complexity guarantees for metric dynamic logic: The monitor’s memory consumption neither depends on the event-rate, i.e., the number of events within a fixed time-unit, nor on the numeric constants occurring in the quantitative temporal constraints in the given formula. We formally prove our algorithm correct in the Isabelle proof assistant, integrate it in the Hydra monitoring tool, and empirically demonstrate its strong performance.
Chapter
Stream Runtime Verification (SRV) is a formal dynamic analysis technique that generalizes runtime verification algorithms from temporal logics like LTL to stream monitoring, allowing the computation of richer verdicts than Booleans (quantitative values or even arbitrary data). The core of SRV algorithms is a clean separation between temporal dependencies and data computations. In spite of this theoretical separation previous engines include ad-hoc implementations of just a few data types, requiring complex changes in the tools to incorporate new data types.
Chapter
Runtime monitors that are specified in a stream-based monitoring language tend to be easier to understand, maintain, and reuse than those written in a standard programming language. Because of their formal semantics, such specification languages are also a natural choice for safety-critical applications. Unlike for standard programming languages, there is, however, so far very little support for automatic code optimization. In this paper, we present the first collection of code transformations for the stream-based monitoring language RTLola. We show that classic compiler optimizations, such as Sparse Conditional Constant Propagation and Common Subexpression Elimination, can be adapted to monitoring specifications. We also develop new transformations—Pacing Type Refinement and Filter Refinement—which exploit the specific modular structure of RTLola as well as the implementation freedom afforded by a declarative specification language. We demonstrate the significant impact of the code transformations on benchmarks from the monitoring of unmanned aircraft systems (UAS).
Chapter
We study the spectra of time-event and of synchronous-asynchronous models of computation for runtime verification, in particular in the context of stream runtime verification (SRV). Most runtime verification formalisms do not involve a notion of time, either by having inputs at all instants (like LTL or Lola) or by reacting to external events in an event-driven fashion (like MOP). Other formalisms consider notions of real-time, ranging from the collection and periodic processing of events to complex computations of the times at which events exist or are produced (like TeSSLa or Striver). Also, some monitoring languages assume that all inputs and outputs change values at once (synchronous), while others allow changes independently (asynchronous).
Chapter
Runtime verification deals with checking correctness properties on the runs of a system under scrutiny. To achieve this, it addresses a variety of sub-problems related to monitoring of systems: These range from the appropriate design of a specification language over efficient monitor generation as hardware and software monitors to solutions for instrumenting the monitored system, preferably in a non-intrusive way. Further aspects play a role for the usability of a runtime verification toolchain, e.g. availability, sufficient documentation and the existence of a developer community. In this paper we present the TeSSLa ecosystem, a runtime verification framework built around the stream runtime verification language TeSSLa: It provides a rich toolchain of mostly freely available compilers for monitor generation on different hardware and software backends, as well as instrumentation mechanisms for various runtime verification requirements. Additionally, we highlight how the online resources and supporting tools of the community-driven project enable the productive usage of stream runtime verification.
Article
Full-text available
Over the last 20 years, runtime verification (RV) has grown into a diverse and active field, which has stimulated the development of numerous theoretical frameworks and practical tools. Many of the tools are at first sight very different and challenging to compare. Yet, there are similarities. In this work, we classify RV tools within a high-level taxonomy of concepts. We first present this taxonomy and discuss its different dimensions. Then, we survey the existing RV tools and, where possible with the support of tool authors, classify them according to the taxonomy. While the classification continually evolves, this article presents a snapshot with 60 state-of-the-art RV tools. We believe that this work is an important step in establishing a common terminology in RV and enabling a meaningful comparison of existing RV tools.
Chapter
Stream runtime verification (SRV) is a formalism to express monitors as relations between typed input streams (observations) and typed output streams (data verdicts). In SRV, the actual data operations are separated from the temporal dependencies, therefore generalizing monitoring algorithms for temporal logics into the computation of richer verdicts. In this paper we study a new and powerful feature, which consists of lifting the execution of monitors to functions that can be used in defining expressions of enclosing specifications. At runtime, the outer monitor invokes the inner monitor passing a list of input events, called a slice. We present nested monitors for synchronous streams and for real-time event streams, allowing the elegant description of many specifications of interest, while still keeping the resources bounded.
Chapter
We investigate online monitoring algorithms over dense-time and continuous-time signals for properties written in metric temporal logic (MTL). We consider an abstract algebraic semantics based on complete lattices, which subsumes the Boolean (qualitative) semantics and the real-valued robustness (quantitative) semantics. Our semantics also extends to truth values that are partially ordered and allows the modeling of uncertainty in satisfaction. We propose a compositional approach for the construction of online monitors based on a class of infinite-state deterministic signal transducers that (1) are allowed to produce the output signal with some bounded delay relative to the input signal, and (2) do not introduce unbounded variability in the output signal. A key ingredient of our monitoring framework is a novel efficient algorithm for sliding-window aggregation over dense-time signals.
Chapter
We present HStriver, an extensible stream runtime verification tool for event streams. The tool consists of a runtime verification engine for (1) real-time events streams where individual observations and verdicts can occur at arbitrary times, and (2) rich data in the observations and verdicts. This rich setting allows, for example, encoding as HStriver specifications quantitative semantics of logics like STL, including different notions of robustness. The keystone of stream runtime verification (SRV) is the clean separation between temporal dependencies and data computations. To encode the data values and computations involved in the monitoring process we borrow (almost) arbitrary data-types from Haskell. These types are transparently lifted to the specification language and incorporated in the engine, so they can be used as the types of the inputs (observations), outputs (verdicts), and intermediate streams. The resulting extensible language is then embedded, alongside the temporal evaluation engine (which is agnostic to the types) into Haskell as an embedded Domain Specific Langauge (eDSL). Morever, the availability of functional features in the specification language enables the direct implementation of desirable features in HStriver like parametrization (using functions that return stream specifications), etc. The resulting tool is a flexible and extensible stream runtime verification engine for real-time streams. We illustrate the use of the tool on many sophisticated real-time specifications, including realistic signal temporal logic (STL) properties of existing designs.
Chapter
Online runtime verification is a formal dynamic technique that studies how to monitor formal specifications incrementally against an input trace. Often, an observed prefix of a behavior is not enough to emit a definite verdict and the monitor must wait to receive more information. Monitorability classifies the set of properties depending on the feasibility to obtain a verdict after a finite observation. Havelund and Peled [20] classified LTL properties according to whether an observation can be extended to a definite answer. In this paper we present a framework that extends the classification of Havelund and Peled to verdict domains that are richer than Booleans, obtaining a monitorability setting under which some of the verdicts (but not others) can be discarded after a sequence of observations. We study two instances of this setting, quantitative temporal logics and partially ordered domains for stream runtime verification, and we illustrate using examples the different elements of the taxonomy. Finally, we also consider how assumptions on the set of behaviors can improve monitorability, and how imprecise observations can impair monitorability.
Chapter
This paper considers two kinds of optimizations for a specification language compiler for stream-based runtime verification: (i) the manual addition of core functions with dedicated translation schemas and (ii) an improved initialization that simplifies subsequent constant propagation. We employ both optimizations within the open source runtime verification framework TeSSLa, which comes with a trans-compiler as synthesis tool which translates TeSSLa specifications to Scala code eventually running on the JVM. Our evaluation shows that the first optimization improves the efficiency of the resulting monitor significantly while the second gets lost within the variety of optimizations present for the back end systems.
Chapter
We revisit Stream Runtime Verification for synchronous systems. Stream Runtime Verification (SRV) is a declarative formalism to express monitors using streams, which aims to be a simple and expressive specification language. The goal of SRV is to allow engineers to describe both correctness/failure assertions and interesting statistical measures for system profiling and coverage analysis. The monitors generated are useful for testing, under actual deployment, and to analyze logs.
Conference Paper
Full-text available
We present TeSSLa, a specification language based on stream run-time verification, designed for monitoring a specific class of real-time signals. Our monitors can observe concurrent systems with a shared clock, but where each component reports observations as signals that arrive to the monitor at different speeds and with different and varying latencies. The signals and streams that TeSSLa supports (including inputs and final verdicts) are not restricted to be Booleans but can be data from richer domains, including integers and reals with arithmetic operations and aggregations. Consequently, TeSSLa can be used both for checking logical properties, and for computing statistics and general numeric temporal metrics (and properties on these richer metrics). We present an online evaluation algorithm for TeSSLa specifications and show a formal proof of the correctness of concurrent implementations of the evaluation algorithm. Finally, we report an empirical evaluation of a highly concurrent Erlang implementation of the monitoring algorithm.
Conference Paper
Full-text available
We introduce Lola 2.0, a stream-based specification language for the precise description of complex security properties in network traffic. The language extends the specification language Lola with two new features: template stream expressions, which allow input data to be carried along the stream, and dynamic stream generation, where new monitors can be invoked during the monitoring process for the monitoring of new subtasks on their own time scale. Lola 2.0 is simple and expressive: it combines the ease-of-use of rule-based specification languages like Snort with the expressiveness of heavy-weight scripting languages or temporal logics previously needed for the description of complex stateful dependencies and statistical measures. Lola 2.0 specifications are monitored by incrementally constructing output streams from input streams, while maintaining a store of partially evaluated expressions. We demonstrate the flexibility and expressivity of Lola 2.0 using a prototype implementation on several practical examples.
Conference Paper
Full-text available
Runtime verification is the process of checking a property on a trace of events produced by the execution of a computational system. Runtime verification techniques have recently focused on parametric specifications where events take data values as parameters. These techniques exist on a spectrum inhabited by both efficient and expressive techniques. These characteristics are usually shown to be conflicting – in state-of-the-art solutions, efficiency is obtained at the cost of loss of expressiveness and vice-versa. To seek a solution to this conflict we explore a new point on the spectrum by defining an alternative runtime verification approach. We introduce a new formalism for concisely capturing expressive specifications with parameters. Our technique is more expressive than the currently most efficient techniques while at the same time allowing for optimizations.
Article
Full-text available
Runtime monitors have been proposed as a means to increase the reliability of safety-critical systems. In particular, this report addresses runtime monitors for distributed hard real-time systems, such as fault-tolerant data buses and control systems for avionics and spacecraft are explored. This class of systems has had little attention from the monitoring community. The need for monitors is shown by discussing examples of avionic systems failure. We survey related work in the field of runtime monitoring. Several potential monitoring architectures for distributed real-time systems are presented along with a discussion of how they might be used to monitor properties of interest.
Article
Full-text available
Techniques for efficiently evaluating future time Linear Temporal Logic (abbreviated LTL) formulae on finite execution traces are presented. While the standard models of LTL are infinite traces, finite traces appear naturally when testing and/or monitoring real applications that only run for limited time periods. A finite trace variant of LTL is formally defined, together with an immediate executable semantics which turns out to be quite inefficient if used directly, via rewriting, as a monitoring procedure. Then three algorithms are investigated. First, a simple synthesis algorithm for monitors based on dynamic programming is presented; despite the efficiency of the generated monitors, they unfortunately need to analyze the trace backwards, thus making them unusable in most practical situations. To circumvent this problem, two rewritingbased practical algorithms are further investigated, one using rewriting directly as a means for online monitoring, and the other using rewriting to generate automata-like monitors, called binary transition tree finite state machines (and abbreviated BTT-FSMs). Both rewriting algorithms are implemented in Maude, an executable specification language based on a very efficient implementation of term rewriting. The first rewriting algorithm essentially consists of a set of
Conference Paper
Full-text available
We present a rule-based framework for defining and implementing finite trace monitoring logics, including future and past time temporal logic, extended regular expressions, real-time logics, interval logics, forms of quantified temporal logics, and so on. Our logic, Eagle, is implemented as a Java library and involves novel techniques for rule definition, manipulation and execution. Monitoring is done on a state-by-state basis, without storing the execution trace.
Chapter
Full-text available
A program verifier determines whether a program satisfies a specification. Ideally verification is achieved by static analysis without executing the code. However, program verification is unsolvable in general. The interactive approach, for example with a human guiding a theorem prover, does not in practice scale to large software systems. Some restricted kinds of specifications can, however, be checked automatically, for example type definitions. Also static analysis of properties such as un-initialized variables, null-pointer de-referencing, and arraybound violations scales to production programs on the order of hundreds of thousands of lines of code. Even concurrency-related problems such as data races and deadlocks can to some extent be checked statically, although often resulting in false positives. However, going beyond these simple properties to arbitrarily complex behavior specification and scaling to ever-growing production program size is undoubtedly a challenge, and in our opinion we cannot expect regular economic use of program verification of arbitrary properties to be fully achieved within the 15 year time horizon of the challenge.
Article
Full-text available
Ordinary software engineers and programmers can easily understand regular patterns, as shown by the immense interest in and the success of scripting languages like Perl, based essentially on regular expression pattern matching. We believe that regular expressions provide an elegant and powerful specification language also for monitoring requirements, because an execution trace of a program is in fact a string of states. Extended regular expressions (EREs) add complementation to regular expressions, which brings additional benefits by allowing one to specify patterns that must not occur during an execution. Complementation gives one the power to express patterns on strings more compactly. In this paper we present a technique to generate optimal monitors from EREs. Our monitors are deterministic finite automata (DFA) and our novel contribution is to generate them using a modern coalgebraic technique called coinduction. Based on experiments with our implementation, which can be publicly tested and used over the web, we believe that our technique is more efficient than the simplistic method based on complementation of automata which can quickly lead to a highly-exponential state explosion.
Conference Paper
Full-text available
We consider the problem of reasoning with linear temporal logic on truncated paths. A truncated path is a path that is finite, but not necessarily maximal. Truncated paths arise naturally in several areas, among which are incomplete verification methods (such as simulation or bounded model checking) and hardware resets. We present a formalism for reasoning about truncated paths, and analyze its characteristics.
Conference Paper
Full-text available
In thispaper we introducea variant oftemporal logictailoredfor spec- ifying desired properties of continuous signals. The logic is based on a bounded subset of the real-time logic MITL, augmented with a static mapping from con- tinuous domains into propositions. From formulae in this logic we create auto- matically property monitors that can check whether a given signal of bounded length and finite variability satisfies the property. A prototype implementation of this procedure was used to check properties of simulation traces generated by Matlab/Simulink.
Conference Paper
Full-text available
The problem of testing a linear temporal logic (LTL) formula on a finite execution trace of events, generated by an executing program, occurs naturally in runtime analysis of software. An algorithm which takes a past time LTL formula and generates an efficient dynamic programming algorithm is presented. The generated algorithm tests whether the formula is satisfied by a finite trace of events given as input and runs in linear time, its constant depending on the size of the LTL formula. The memory needed is constant, also depending on the size of the formula. Further optimizations of the algorithm are suggested. Past time operators suitable for writing succinct specifications are introduced and shown definitionally equivalent to the standard operators. This work is part of the PathExplorer project, the objective of which it is to construct a flexible framework for monitoring and analyzing program executions.
Conference Paper
Full-text available
We address the problem of runtime monitoring for hard real-time programs—a domain in which correctness is critical yet has largely been overlooked in the runtime monitoring community. We describe the challenges to runtime monitoring for this domain as well as an approach to satisfy the challenges. The core of our approach is a language and compiler called Copilot. Copilot is a stream-based dataflow language that generates small constant-time and constant-space C programs, implementing embedded monitors. Copilot also generates its own scheduler, obviating the need for an underlying real-time operating system.
Article
Full-text available
This article studies runtime verification of properties expressed either in lineartime temporal logic (LTL) or timed lineartime temporal logic (TLTL). It classifies runtime verification in identifying its distinguishing features to model checking and testing, respectively. It introduces a three-valued semantics (with truth values true, false, inconclusive) as an adequate interpretation as to whether a partial observation of a running system meets an LTL or TLTL property. For LTL, a conceptually simple monitor generation procedure is given, which is optimal in two respects: First, the size of the generated deterministic monitor is minimal, and, second, the monitor identifies a continuously monitored trace as either satisfying or falsifying a property as early as possible. The feasibility of the developed methodology is demontrated using a collection of real-world temporal logic specifications. Moreover, the presented approach is related to the properties monitorable in general and is compared to existing concepts in the literature. It is shown that the set of monitorable properties does not only encompass the safety and cosafety properties but is strictly larger. For TLTL, the same road map is followed by first defining a three-valued semantics. The corresponding construction of a timed monitor is more involved, yet, as shown, possible.
Article
Full-text available
We propose a formal definition for the timed asynchronous distributed system model. We present extensive measurements of actual message and process scheduling delays and hardware clock drifts. These measurements confirm that this model adequately describes current distributed systems such as a network of workstations. We also give an explanation of why practically needed services, such as consensus or leader election, which are not implementable in the time-free model, are implementable in the timed asynchronous system model
Article
Full-text available
In this paper we define timed regular expressions, a formalism for specifying discrete behaviors augmented with timing information, and prove that its expressive power is equivalent to the timed automata of Alur and Dill. This result is the timed analogue of Kleene Theorem and, similarly to that result, the hard part in the proof is the translation from automata to expressions. This result is extended from finite to infinite (in the sense of B uchi) behaviors. In addition to these fundamental results, we give a clean algebraic framework for two commonly-accepted formalism for timed behaviors, time-event sequences and piecewise-constant signals. 1
Chapter
The term Cyber-Physical Systems (CPS) typically refers to engineered, physical and biological systems monitored and/or controlled by an embedded computational core. The behaviour of a CPS over time is generally characterised by the evolution of physical quantities, and discrete software and hardware states. In general, these can be mathematically modelled by the evolution of continuous state variables for the physical components interleaved with discrete events. Despite large effort and progress in the exhaustive verification of such hybrid systems, the complexity of CPS models limits formal verification of safety of their behaviour only to small instances. An alternative approach, closer to the practice of simulation and testing, is to monitor and to predict CPS behaviours at simulation-time or at runtime. In this chapter, we summarise the state-of-the-art techniques for qualitative and quantitative monitoring of CPS behaviours. We present an overview of some of the important applications and, finally, we describe the tools supporting CPS monitoring and compare their main features.
Article
We introduce RTLola, a new stream-based specification language for the description of real-time properties of reactive systems. In real-time applications, data arrives at varying rates and in most cases it is hard to predict the input rate. The integration of sliding windows over real-time intervals with aggregation functions into RTLola allows us to detach fixed-rate output streams from the varying rate input streams. However, the number of input values within a single window instance can grow arbitrarily large disallowing any guarantees on the expected memory consumption. A feature of RTLola is that it allows for an automatic memory analysis that guides the user in identifying the computationally expensive specifications. For specifications using only certain classes of aggregation functions, we can give a precise memory bound. Furthermore, assuming a fixed monitor output rate, we can provide memory guarantees which can be computed statically. To demonstrate the features of RTLola, we evaluate the language and the implementation experimentally.
Conference Paper
Stream runtime verification (SRV), pioneered by the tool LOLA, is a declarative approach to specify synchronous monitors. In SRV, monitors are described by specifying dependencies between output streams of values and input streams of values. The declarative nature of SRV enables a separation between (1) the evaluation algorithms, and (2) the monitor storage and its individual updates. This separation allows SRV to be lifted from conventional failure monitors into richer domains to collect statistics of traces. Moreover, SRV allows to easily identify specifications that can be efficiently monitored online, and to generate efficient schedules for offline monitors. In spite of these attractive features, many important theoretical problems about SRV are still open. In this paper, we address complexity, expressiveness, succinctness, and closure issues for the subclass of Boolean SRV (BSRV) specifications. Additionally, we show that for this subclass, offline monitoring can be performed with only two passes (one forward and one backward) over the input trace in spite of the alternation of past and future references in the BSRV specification.
Conference Paper
In this paper and its accompanying tutorial, we discuss the topic of teaching runtime verification. The aim of the tutorial is twofold. On the one hand, a condensed version of a course currently given by the author will be given within the available tutorial time, giving an idea about the topics of the course. On the other hand, the experience gained by giving the course should also be presented and discussed with the audience. The overall goal is to simplify the work of colleagues developing standard and well accepted courses in the field of runtime verification.
Article
In this paper, a brief account of the field of runtime verification is given. Starting with a definition of runtime verification, a comparison to well-known verification techniques like model checking and testing is provided, and applications in which runtime verification brings out its distinguishing features are pointed out. Moreover, extensions of runtime verification such as monitor-oriented programming, and monitor-based runtime reflection are sketched and their similarities and differences are discussed. Finally, the use of runtime verification for contract enforcement is briefly pointed out.
Conference Paper
The paper introduces the construct of temporal testers as a compositional basis for the construction of automata corresponding to temporal formulas in the PSL logic. Temporal testers can be viewed as (non-deterministic) transducers that, at any point, output a boolean value which is 1 iff the corresponding temporal formula holds starting at the current position. The main advantage of testers, compared to acceptors (such as Büchi automata) is that they are compositional. Namely, a tester for a compound formula can be constructed out of the testers for its sub-formulas. In this paper, we extend the application of the testers method from LTL to the logic PSL. Besides providing the construction of testers for PSL, we indicate how the symbolic representation of the testers can be directly utilized for efficient model checking and run-time monitoring
  • E Bartocci
  • Y Falcone
Monitoring metric first-order temporal properties
  • D A Basin
  • F Klaedtke
  • S Müller
  • E Zalinescu
  • David Basin
TeSSLa: temporal stream-based specification language
  • L Convent
  • S Hungerecker
  • M Leucker
  • T Scheffel
  • M Schmitz
  • D Thoma