MeMo - methods of model quality

Abstract

Model driven development as implemented by the Simulink-Stateflow- TargetLink tool chain facilitates the efficient development of software for embedded processors. But there are only a few automated quality assurance techniques comparable to those known from traditional software development that can be applied in early phases of the development process. This is a serious problem since the generated software especially in the automotive area has to fulfill very high safety requirements. In this paper, we present our project Methods of Model Quality1 in which we develop automated quality assurance methods for early development phases to improve the current unsatisfying situation. These methods comprise static analyses for domainspecific error detection, analyses to identify the most error-prone model parts through model metrics and furthermore slicing techniques for analyses support and result visualization. To estimate the model maintainability and changeability a quality model including architecture and design analyses is proposed as well. The expected results of our project will help to reduce development time and costs as well as to improve code quality and reliability.

Full text

MeMo – Methods of Model Quality
Wei Hu, Joachim Wegener Ingo Stürmer
Berner & Mattner Systemtechnik GmbH Model Engineering Solutions GmbH
Robert Reicherdt, Elke Salecker, Sabine Glesner
Technische Universität Berlin
Abstract: Model driven development as implemented by the Simulink-Stateflow-
TargetLink tool chain facilitates the efficient development of software for embedded
processors. But there are only a few automated quality assurance techniques com-
parable to those known from traditional software development that can be applied in
early phases of the development process. This is a serious problem since the generated
software especially in the automotive area has to fulfill very high safety requirements.
In this paper, we present our project Methods of Model Quality1in which we develop
automated quality assurance methods for early development phases to improve the
current unsatisfying situation. These methods comprise static analyses for domain-
specific error detection, analyses to identify the most error-prone model parts through
model metrics and furthermore slicing techniques for analyses support and result vi-
sualization. To estimate the model maintainability and changeability a quality model
including architecture and design analyses is proposed as well. The expected results
of our project will help to reduce development time and costs as well as to improve
code quality and reliability.
1 Introduction
Model-based development is meanwhile common practice within a wide range of auto-
motive embedded software development projects. Following the model-based approach
means focusing on graphical models as the central development artifact to specify, design
and implement software. These models are usually realized using data-flow and control-
flow oriented modeling languages such as MATLAB/Simulink and Stateflow by The Math-
Works [Mat10]. This modeling language is widespread in the automotive domain, where
controller models are used to specify, design and implement software models that are used
as a basis for embedded controller code generation (engine control, central locking sys-
tem, etc.). Code generators make it possible to automatically generate efficient C code
directly from these models. Due to the maturity of the code generators available and the
application of model-based code generation in combination with model-based testing, it
is meanwhile also common practice to use Simulink controller models for safety-relevant
applications (e.g. automatic braking systems, X-by-wire systems). The model-based code
generation approach implies that the quality and complexity of the models used for code
1The project is funded by the Investitionsbank Berlin (IBB) within the EFRE program.

generation have a direct influence on the quality of the generated C code. Due to the fact,
that the controller models used for code generation are of increasing complexity, analytical
as well as constructive quality assurance methods are to be carried out as early as possible.
With the so far available methods and tools, quality aspects such as maintainability, reuse
and extendibility of huge controller models can only be addressed in a limited way. One
reason for this is that fact metrics and methods are not sufficiently available in order to e.g.
structure the models.
The MeMo project (MeMo: Methods of Model Quality) focuses on quality assurance
methods in order to increase the maturity of controller models used for serial production
purposes. Three main topics are addressed within MeMo:
1. Development of slicing techniques for analyzing Simulink models
2. Data- and control-flow driven error analysis methods for Simulink
3. Analysis of the model’s design and its model architecture
Theses topics are presented in the following chapters.
2 Slicing
Slicing is used to simplify a program according to a slicing criterion, which is usually a
program point and a set of variables of interest. All statements that do not affect the slicing
criterion are removed from the program such that its semantics with respect to the slicing
criterion is preserved. Slicing is applied as support for tasks such as debugging or reducing
the state space for model checking.
Since [Wei81] introduced program slicing, a lot of program slicing techniques have been
developed. Most of these techniques use dependence graphs that are directed, rooted
graphs with nodes representing program statements and edges representing direct depen-
dencies between those statements. An overview about program slicing is given in [Tip95].
The concept of slicing was extended to slice graphical notations. Androutsopoulos pre-
sented an approach [ACH+09] for slicing Extended Finite State Machines. Wang pre-
sented an approach for slicing UML State Charts [WDQ02] by transforming them into
Extended Hierarchical Automata (EHA) [MLS97]. To the best of our knowledge neither
algorithms for slicing Simulink nor for slicing Stateflow are known in the literature.
For slicing models containing Simulink as well as Stateflow parts we plan to develop an
integrated slicing approach that will use dependence graphs for slicing both Simulink and
Stateflow. Simulink is a visual data flow oriented notation where data dependencies can
easily derived from signal lines. Control dependencies are either explicit like For,If or
Switch-Case-blocks, etc. or implicit like Switch or Multiport Switch-blocks and condi-
tional subsystems. We treat functional blocks as statements and signals as variables like
in programs so that we are able to map them to nodes and edges when constructing the
dependence graph. We define the slicing criterion C(B,S) where Bis a block and Sis a

set of incoming or outgoing signals. Stateflow is to some extend a Statechart derivate but
also contains portions of flowcharts. Wang et al. defined a set of dependence relations on
EHAs that describe different kinds of control and data dependence. They also use syn-
chronization dependence to handle events. These dependence relations still need further
investigation with respect to the semantics of Stateflow but seem to be applicable. We
define our slicing criterion C(S,T,V) where Sis a set of states, Tis a set of transitions and
Vis a set of variables. We plan to remove statements, events and graphical elements that
are not affected or do not affect the variables in V.
In our project we plan to use Simulink and Stateflow model slicing to support static anal-
yses and to improve result visualization. In software engineering, such as in automotive
industries, variants are used to model highly configurable software systems. We plan to
use slicing to remove inactive variants before we start our static analyses to reduce the
number of unjustified error messages due to over-approximation caused by interdepen-
dencies between mutual exclusive variants. Often the source of an error does not appear at
the point where the error is found but in previous blocks. Hence our visualization will use
slicing to highlight the preceding blocks, that are relevant to the signal values of the block,
where an error has been detected. Both identified application fields for slicing are relevant
with respect to achievement of our overall project goals.
3 Static Analyses for Simulink and Stateflow Models
Static analysis tools like Coverity or Polyspace are integrated in the software development
process in many companies. They help to detect failures early in the development cycle
and thus can remarkably reduce development costs. A good survey on tools used in in-
dustrial projects can be found in [EN08]. These tools detect source code specific errors
e.g. memory leaks or null pointer dereferences that are difficult to detect through code
reviews or testing. Analyzing code generated from MATLAB Simulink/Stateflow mod-
els with these tools is cumbersome, since errors detected in the generated code must be
manually back traced into the model. Moreover, the failure checks implemented by these
tools are not really suitable for it. Many of them are not relevant for this class of software
systems, other domain-specific errors are not checked.
In our project we plan to develop failure analyses that focus on errors specific to MAT-
LAB Simulink/Stateflow models. Therefore we have conducted in the first phase of our
project a study 1) to identify relevant error classes and 2) to identifiy criteria that a failure
analysis must fulfil to guarantee its applicablility for real-world models and its acceptance
by practitioners. Based on our results we have defined the following failure classes, for
which we plan to develop analyses:
1. illegal arithmetic operations
2. incomplete or inaccessible model parts
3. improper fixed-point data type scaling

To the first failure class belong errors that are known from traditional static source code
analysis as for example division by zero or range violations. Range violations in MATLAB
Simulink/Stateflow models may appear at control inputs for multiport switch blocks. The
control input value is used as data port index and can have any arbitrary, user defined value.
It may originate from any other part of the model what can easily lead to invalid run time
values and hence to unexpected behaviour of the model. Errors of the second class are
comparable to uninitialized variable values or dead code. The third class of errors results
from the fact that a model’s floating point data types must be mapped onto fixed point data
types for code generation if the target processor does not support floating point operations.
This can result in unintended loss of precision that leads to unexpected model behaviour.
During our evaluation of static analysis methods we came to the conclusion that the ac-
ceptance of a static analysis tool is greatly influenced by its scalability. This implies for
our project that our methods must be capable to deal with models that comprises up to
200.000 blocks. Another important aspect is the tool’s false positive rate that indicates the
number of unjustified error messages. Since all reported possible error candidates must be
checked manually a great number of spurious error messages can have a contrary effect i.e.
the tool’s deployment will cause an increased development time while the model quality
does not improve. To avoid this problem we can benefit from the experiences with static
analyses for source code and adapt successfully employed techniques as for example error
message filtering.
4 Model Quality Assessment
In a model-based development process the system model plays a central role for require-
ments, verification and validation, testing, and ultimately product deployment with or
without automatic code generation. Although MATLAB/Simulink/Stateflow has estab-
lished itself as de facto industry standard for model-based development especially in the
automotive branch, there exist only few works concerning the model quality. The modeling
guidelines of MathWorks Automotive Advisory Board (MAAB) [Boa07] are probably the
most elaborated guidelines for MATLAB/Simulink/Stateflow models in praxis. Automatic
inspection of some modeling rules, e.g. naming conventions and parameter configurations,
have been successfully realized in different commercial tools. However, these guideline
checks do not provide information about the internal quality of a model. Deissencoeck
et al. [DHJ+08] and Pham et al. [PNN+09] demonstrated methods to search for model
clones, i.e. copy&paste model parts without or with change(s), which are well known
bad practices for software maintainability. Stürmer et al. [SPR10] presented an approach
based on Halstead metrics to assess the Simulink/Stateflow model complexity.
These works have established a solid foundation for further research on model quality.
Within our MeMo project, we are developing a quality model, which illuminates different
internal quality characteristics w.r.t. changeability and maintainability, includes metrics as
well, with which the quality attributes can be evaluated quantitatively or qualitatively. By
means of the metrics assessment we are then able to deliver an inner picture of the model
quality.

Basically there are three steps to evaluate the quality of a software product. Firstly, define
and specify the quality characteristics. Secondly, develop metrics based on the charac-
teristics and thirdly, evaluate the metrics and interpret the result. Started from the ISO
9126-1 [ISO], which defines a general quality model for internal and external quality for
software product, we propose a quality model with seven internal quality characteristics
for MATLAB/Simulink/Stateflow models: Readability/Understandability,Changeabili-
ty/Extensibility,Analysability,Testability,Reusability,Stability and Conformity. These
quality features are further divided into subcharacteristics, for each subcharacteristic there
are metrics developed for the assessment.
Take testability as example to outline our approach. The testability estimates the effort
needed to validate the modified model. We consider thereby attributes like module depen-
dency, module interfacing and model-code consistency. Of these three attributes, module
dependency and module interfacing are calculated with metrics adapted from slice-based
measures of coupling and cohesion for conventional programming languages. The model-
code consistency is a particular feature within Simulink/Stateflow models, for there exist
blocks which contribute only to graphical simplification and organization in model. They
have, however, no impact on the generated code. The selection of modeling blocks is not
always straightforward, often there are tradeoffs between modeling aspects like require-
ments, flexibility and simulation performance. With that in mind, the model-code consis-
tency characteristic can only be evaluated qualitatively. Should all the above stated quality
characteristics be evaluated, we will be able to estimate the effort needed for changing and
maintaining a high integrity and/or safety-critical model.
5 Conclusion
In our project MeMo we address research questions that are of high practical relevance
due to the increasing importance of model based development in safety critical areas. The
expected results will enable to integrate automatic quality assurance techniques early in
the development cycle and hence will help to improve quality and reliability of the imple-
mented systems.
The cooperation of industrial partners and academic researchers within our project has two
main advantages with a positive impact on the project’s success. Due to our cooperation
we have available case studies from industrial development projects. This possibility con-
siderably facilitates the evaluation of our methods. Furthermore, our cooperation speeds
up both the knowledge transfer from research to industry and the feedback on the devel-
oped approaches.
References
[ACH+09] Kelly Androutsopoulos, David Clark, Mark Harman, Zheng Li und Laurence Tratt. Con-
trol Dependence for Extended Finite State Machines. In Marsha Chechik und Martin

Wirsing, Hrsg., FASE, Jgg. 5503 of Lecture Notes in Computer Science, Seiten 216–230.
Springer, 2009.
[Boa07] MathWorks Automotive Advisory Board. Control Algorithm Modeling Guidelines Us-
ing MATLAB R
, Simulink R
, and Stateflow R
, 2007. Version 2.1.
[DHJ+08] F. Deissenboeck, B. Hummel, E. Jürgens, B. Schätz, S. Wagner, J.-F. Girard und
S. Teuchert. Clone detection in automotive model-based development. In Proceed-
ings of the 30th international conference on Software engineering, ICSE ’08, Seiten
603–612, New York, NY, USA, 2008. ACM.
[EN08] Pär Emanuelsson und Ulf Nilsson. A Comparative Study of Industrial Static Analysis
Tools. Electr. Notes Theor. Comput. Sci., 217:5–21, 2008.
[ISO] ISO/IEC9126-1:2001. Software engineering – Product quality – Part 1: Quality model.
[Mat10] The MathWorks, 2010. http://www.mathworks.com.
[MLS97] Erich Mikk, Yassine Lakhnech und Michael Siegel. Hierarchical Automata as Model
for Statecharts. In R. K. Shyamasundar und Kazunori Ueda, Hrsg., ASIAN, Jgg. 1345
of Lecture Notes in Computer Science, Seiten 181–196. Springer, 1997.
[PNN+09] Nam H. Pham, Hoan Anh Nguyen, Tung Thanh Nguyen, Jafar M. Al-Kofahi und
Tien N. Nguyen. Complete and accurate clone detection in graph-based models. In
Proceedings of the 31st International Conference on Software Engineering, ICSE ’09,
Seiten 276–286, Washington, DC, USA, 2009. IEEE Computer Society.
[SPR10] I. Stürmer, H. Pohlheim und T. Rogier. Berechnung und Visualisierung der Model-
lkomplexität bei der modellbasierten Entwicklung sicherheits-relevanter Software. In
Automotive - Safety & Security 2010, Seiten 69–82. Shaker Verlag, June 2010.
[Tip95] Frank Tip. A survey of program slicing techniques. J. Prog. Lang., 3(3), 1995.
[WDQ02] Ji Wang, Wei Dong und Zhichang Qi. Slicing Hierarchical Automata for Model Check-
ing UML Statecharts. In Chris George und Huaikou Miao, Hrsg., ICFEM, Jgg. 2495 of
Lecture Notes in Computer Science, Seiten 435–446. Springer, 2002.
[Wei81] Mark Weiser. Program slicing. In Proceedings of the 5th international conference on
Software engineering, ICSE ’81, Seiten 439–449, Piscataway, NJ, USA, 1981. IEEE
Press.