Pure Modelica Unit Testing: From Mathematical Algorithms to Physical Modeling

Abstract

Modelica is distinguished by its capabilities for equation-based syntax. Nevertheless, not only physical models consist of equations but also mathematical algorithms do. This presentation demonstrates early ad-hoc testing experiences of mathematical algorithms developed in Modelica for computing dynamic parameter sensitivities at the model level. The underlying approach has been converged to a formal way for unit testing of Modelica physical components. In particular, the open-source XogenyTest Modelica library for unit testing including recent enhancements from the author is demonstrated.

While the majority of the current set of Modelica-based unit testing technologies assume that the tester knows in advance the correct reference solution of a unit test, the philosophy of the proposed approach depends on providing two different implementations of a small system comprising the component to be tested. A unit test would then consist of the object-oriented implementation of the system representation together with the equivalent explicit equation system. If both implementation don't give semi-identical results, the unit test fails. Conversely, it is practically hard to have two identically wrong solution trajectories.

The presentation demonstrates some use cases for major improvement of recently developed Modelica libraries. The advantage of this approach is its simplicity and suitability. It allows Modelica libraries to be shipped together with its unit tests, like the open-source GenKinetics library.

Full text

Atiyah Elsheikh
Mathemodica.com
4th February 2019
Atiyah.Elsheikh@mathemodica.com
Pure Modelica Unit Testing:
From Mathematical Algorithms to
Physical Modeling
OpenModelica Annual Workshop 2019

Outline
●First testing experiences
–Mathematical algorithms in Modelica
●Unit testing in Modelica
–existing technologies
●A bit more “formal” Unit testing
–Upgrading to Physical Modeling

Motivation
“Simulation tools not only perform numerical
solutions based on the system equations but
also assist the modeler in systems analysis.
Doubtlessly the most important systems analysis
tool is Sensitivity Analysis ...
Wiechert, W., Noack, S., and Elsheikh, A. (2010).
Modeling languages for biochemical network
simulation: Reaction vs equation based approaches.
Advances in Biochemical Engineering / Biotechnology

31/05/18 4
SA of Modelica Models
Modelica model
F
( ˙
x , x , p , t
)=
0
x
(
0
)=
x
0
(
p
)
x
0
x
(
t
)
p
simulate
Dynamic Parameter
Sensitivities (DPS)
∂
x
∂
p
(
t
)
∂
x
∂
x
0
(
t
)

31/05/18 5
ADModelica (2007)
OMC
Compiler
XML
Modelica
XML
Parser
AnalyzerVisualizer
Unparser
ADModelica

31/05/18 6
Sample of generated code
(I/II)
•Declaration part

31/05/18 7
Sample of generated code
(II/II)
•Equation part

31/05/18 8
Library L
Component C1
Component C2
Component C3
Library L'
Component C'1
Component C'2
Component C'3
AD
Algorithmic Differentiation (AD)
of Modelica libraries (2011)
extend

31/05/18 9
Algorithmic differentiation Modelica Libraries
Algorithmically differentiated
Modelica library for biochmeical
reaction networks
https://github.com/modelica-3rdparty/ADGenKinetics
The ADGenKinetics Library The ADMSL Library
Serves as an example of
algorithmically differentiated
Modelica library
https://github.com/AIT-CES-LAB/ADMSL
2011 2014

31/05/18 10
Chua Circuit
Standard Modelica model for an electrical circuit
Modelica.Electrical.Analog.Basic.Examples.ChuaCircuit
This examples simulates
the current and voltage
at all components

31/05/18 11
Chua Circuit importing ADMSL
The sensitivities of the current w.r.t. L
∂X.i
∂L.L

31/05/18 12
Chua Circuit importing ADMSL
The sensitivities of the voltage at L
w.r.t. all parameters
∂VL
∂X.p

Applications of DPS
1) Modeling-Orietned
Control Coefficients, Local SA, Parameter Sweeping Studies,
Model Simplification, ...
2) Statistical
Regression Analysis, Global SA, Identifiability Analysis, ...
3) Optimization
Cost-functions expressed in terms of DPS
Atiyah Elsheikh and Sergei Kucherenko. (2019)
Dynamic parameter sensitivities: Summary of applications – version 1.0.
Technical Report, to appear

31/05/18 14
Several snapshots at different time points
Scaled Parameter Sensitivities
CC
p
x
=
p
x
(
t
)
∂
x
∂
p
(
t
)

31/05/18 15
Correlation among parameters
Based on Fischer Information Matrices
Strongly
Correlated
Parameters
Correlation among parameters

31/05/18 16
Testing AD
∂
x
∂
p
j
(
t , p
)=
x
(
t , p
+
e
j
δ
j
)−
x
(
t , p
)
δ
j
+
O
(
δ
j
)
δ
j
=
ϵ
p
j
Finite Difference Methods
1)Evaluate DPS with FD
externally with Matlab
small models
2) Simulate AD models
3) Check if the curves behave similarly
Sometimes compute Relative errors
4)Investigate cause of errors, if any
Adhoc Testing (2007-2014)

PSTools Library
●Promotes the usage of PS at Modelica level
●Serves as utility package for arbitrary Modelica libraries
●PS Package
–Extensive set of examples for analytical derivatives
●Including hybrid systems
–Generic Models for advanced FD
–Second-order DPS
●Tools Package
–Taylor series approximation
–Parameter Sweeping studies
–Control Coefficients
–...
Under Development (2019)
Highly Experimental
Unit Testing is must

FD with PSTools
model ParM
extends Utilities.Parameterized(
NP = 2,
_P = {0.4, 0.5},
PNAME = {"p1", "p2"},
NX = 2,
_X = {_M.x1, _M.x2},
XNAME = {"x1", "x2"}
);
protected
MyModel _M(
p1 = _P[1],
p2 = _P[2]);
end ParM;
model FDParM
PSTools.PS.FD.CD2
PS(redeclare replaceable
Model ParModel = ParM);
end FDParM;

Parameter Sweeping Studies
Jan Peter Axelsson & Atiyah Elsheikh
Example of Sensitivity Analysis with the Bioprocess Library
Modprod 2019

Unit Testing Technologies for Modelica
●CSV Compare – ESI ITI
●BuildingPy – LBNL
●PySimulator – DLR
●test.openmodelica.org
●XogenyTest – Xogeny
Marco Kessler, Testing Tutorial
Dassault Systems, Modelica 2017 Prague (Link)
●Model Management –
Dymola
●Testing tool kits (many
Companies)
●Testing Library --
Dymola

XogenyTest
●Pure Modelica
●Minimal
●external scripting
●log files
● Easy to use
● No dependencies
Michael Tiller
https://github.com/xogeny/XogenyTest
Tiller, Michael M., and Burit Kittirungsi.
"UnitTesting: A Library for Modelica Unit Testing"
Modelica Conference Vienna, Austria, 2006
Not meant to be
comprehensive but
definitely an ideal
getting start

XogenyTest Low level functions
asserting whether
an expected value lies within
-neighbourhood of a reference
value (i.e. )
within the same order of
magnitude
x
ϵ
N
ϵ
(
r
)
r
|
x
−
r
|
< ϵ
or
|
log
x
r
|
< ϵ

XogenyTest
Models
asserting an expected
value w.r.t. a reference
value at time
or at time
x
r
T∈{t0,tf}
|
x
(
T
)−
r
|
< ϵ
t∈(t0, t f)
|
x
(
t
)−
r
|
< ϵ

XogenyTest
Models
asserting an expected
event occurs at time
Asserting a trajectory at
discrete time points
w.r.t. reference values
|
x
(
t
i
)−
r
(
t
i
)
|
<ϵ
r
(
t
i
)
e
(
t
)
T
e
([
T
−ϵ
,T
+ϵ])→
True
x
(
t
)
{
t
1,
t
2
,
...
, t
N
}

Examples (I/II)
model CheckSuccess
Real x = time^2;
AssertTrajectory check_x(
actual=x,
expected=[0,0; 1,1; 2,4; 3,9]);
Annotation(
TestCase(
action="simulate",
result="success"),
experiment(StopTime=4));
end CheckSuccess;

Examples (II/II)
model CheckSuccess
Real x = time;
AssertBecomesTrueAt
check_event(event=(x>2), at=2);
Annotation(
TestCase(
action="simulate",
result="success"),
experiment(StopTime=4));
end CheckSuccess;

XogenyTest
Models
Assert thata signal
is of an average value
over
Asset that difference between
trajectories and
Is less than accu. error
1
t
f
−
t
0
|
∫
t
0
t
f
u
(
t
)
dt
|
−
a
<ϵ
u
(
t
)
a
[
t
0
,t
f
]
x
1
(
t
)
x
2
(
t
)
A
err
∫
t
0
t
f
|
x
1
(
t
)−
x
2
(
t
)
|
dt
<
A
err

Testing AD vs. FD
model TestT1BioProcess
import PSTools.Utilities.unitVector;
Utilities.Validate dmu_dKs(
AccErr=1E-1,
name="test T1.Processions dmu / dKs");
PSTools.PS.T1.BioProcess.Bioprocess PSAD(
NG=1,
g_Ks=unitVector(1, NG));
PSTools.PS.FD.BioProcess.BioProcess PSFD;
...

equation
dmu_dKs.T1 = PSAD.g_mu[1];
dmu_dKs.T2 = PSFD.g_mu[1];
end TestT1BioProcess;
Visual unit testing
Utilities.Validate
T1
T2
Check

Case Study 1 PSTools Library
// der(VS) = -qS * VX;
g_VS[i] = -g_qS[i] * VX
- qS * g_VX[i];
UT_TestT1_FAIL
// der(VS) = -qS * VX;
der(g_VS[i]) = -g_qS[i] * VX
- qS * g_VX[i];
UT_TestT1_WORK

GenKinetics
2018 R 1.0-alpha
v=∏a
KAa+ [ Aa]
KAa
⋅∏b
KIb
KIb+ [ Ib]⋅
Vmax
fwd ∏i
[Si]
KmS i
−Vmax
bwd ∏j
[Pj]
KmP j
Vmax
fwd ∏i
1+[Si]
KmS i
+Vmax
bwd ∏j
1+[Pj]
KmP j
−1

Formal unit testing procedure
1) a unit test model for each component C using
- Example of a small model employing C
- Equivalent model implemented only by equations
2) Execute all unit tests by *.mos & OMShell
- after each significant modification
3) Investigate errors , if any
Most useful when the component checks but
the UT does not translate or simulate
A library is shipped with its unit tests
All unit tests are within a package called UnitTests
Currently about 40 unit tests distributed GenKinetics

Case Study 2 ADGenKinetics Library
A = product({KA[i] / (KA[i] + mc_A[i].c) for i in 1:NA});
OMC 1.12.0
A = product({ (KA .+ mc_A.c) ./ KA for i in 1:NA});
OMC 1.13.0
2012
2019

Case Study 3 GenKinetics / Biochem
Libraries
der(c) = if
(c < tolerance) then 0 else r_net;
Numerical Problem
der(c) = if
(c < tolerance and r_net < 0) then 0 else r_net;
Solution

Advantages
●Simplicity
●Purely Modelica
●Suitability
–individual developers of Modelica libraries
●GenKinetics https://github.com/AtiyahElsheikh/GenKinetics
●PSTools
–small-scale developments
–Minimum constraints get done
–Shipping Unit Tests with the product

Outlook
●a user experience report from GenKinetics
●Combine Relative Error w. Absolute Error
●Continuous Integration Solution
Asghar, Adeel, Andreas Pfeiffer, Arunkumar Palanisamy, Alachew Mengist,
Martin Sjölund, Adrian Pop, and Peter Fritzson.
"Automatic Regression Testing of Simulation Models and Concept for
Simulation of Connected FMUs in PySimulator."
InProceedings of the 11th International Modelica Conference, Versailles. 2015.