ArticlePDF Available

State-preserving nonlinear model reduction procedure

Authors:

Abstract and Figures

Model reduction has proven to be a powerful tool to deal with challenges arising from large-scale models. This is also reflected in the large number of different reduction techniques that have been developed. Most of these methods focus on minimizing the approximation error; however, they usually result in a loss of physical interpretability of the reduced model. A new reduction technique, which preserves a non-prescribed subset of the original state variables in the reduced model, is presented in this work. The technique is derived from the Petrov–Galerkin projection by adding constraints on the projection matrix. This results in a combinatorial problem of which states need to be selected. A sequential algorithm has been developed based on the modified Gram–Schmidt orthogonalization procedure. The presented technique is applied to two examples where the reduction error is found to be comparable to the traditional POD method. At the same time, the technique has the advantage that the physical interpretation of the remaining states is retained.
Content may be subject to copyright.
1
State-preserving nonlinear model reduction procedure
Yunfei Chu, Mitchell Serpas and Juergen Hahn
*
Artie McFerrin Department of Chemical Engineering
Texas A&M University
College Station, TX 77843-3122, USA
Tel: +1-979-845-3568
Fax: +1-979-845-6446
hahn@tamu.edu (J. Hahn)
ABSTRACT
Model reduction has proven to be a powerful tool to deal with challenges arising from large-
scale models. This is also reflected in the large number of different reduction techniques that
have been developed. Most of these methods focus on minimizing the approximation error,
however, they usually result in a loss of physical interpretability of the reduced model. A new
reduction technique which preserves a non-prescribed subset of the original state variables in
the reduced model is presented in this work. The technique is derived from the Petrov-
Galerkin projection by adding constraints on the projection matrix. This results in a
combinatorial problem of which states need to be selected. A sequential algorithm has been
developed based on the modified Gram-Schmidt orthogonalization procedure. The presented
technique is applied to two examples where the reduction error is found to be comparable to
the traditional POD method. At the same time, the technique has the advantage that the
physical interpretation of the remaining states is retained.
Keywords:
Model reduction, Nonlinear dynamics, Petrov-Galerkin projection, Proper-orthogonal
decomposition, Mathematical modeling, Systems engineering
2
1. Introduction
Model-based techniques have become increasingly popular in process control, monitoring,
and optimization. Part of this trend is that an increasing number of detailed models have been
built in a variety of different areas, e.g. chemical processes (Skogestad and Morari, 1988;
Maria 2004), biochemical networks (Dano et al., 2006; Conzelmann et al., 2008), or
combustion systems (Wang and Frenklach, 1997; Lu and Law, 2009). Furthermore, modeling
large-scale reaction systems is a subject that has been and will continue to be an active
research area (Ho, 2008). However, it is not always possible to verify all aspects of these
detailed models or to make use of these models in an online environment. Model reduction is
one approach to deal with these problems as the complexity of the model is reduced to a level
that is appropriate for a specific application at hand (Moore, 1981; Okino and Mavrovouniotis,
1998; Marquardt, 2001; Ryckelynck et al., 2006; Nagy and Turanyi, 2009).
It is the purpose of this work to introduce a model reduction technique that reduces the
model size while at the same time retains some of the original state variables of the model.
The advantage of such an approach is that the physical interpretation of the remaining states is
unchanged, unlike results that are obtained by many other model reduction techniques (Vajda
et al., 1985; Deane et al., 1991; Hahn and Edgar, 2002a, 2002b; Lall et al., 2002).
The presented technique makes use of a constrained Petrov-Galerkin projection. Similar to
proper-orthogonal decomposition (POD) techniques, the projection matrix is computed from
simulation data which allows the method to be applied to nonlinear models. However, unlike
regular POD algorithms, constraints are added for the computation of the projection matrix to
ensure that a subset of the original state variables are preserved in the reduced model. This
results in a combinatorial problem where states are selected based upon optimizing a least
squares criterion. A sequential algorithm that is based upon a modified version of the Gram-
3
Schmidt orthogonalization procedure is developed in this work to solve the selection problem
with a low computational burden.
The paper is structured as follows: Section 2 provides background information on the
Petrov-Galerkin projection and POD. Section 3 presents the model reduction technique and
some representative results returned by the method are shown in Section 4. Some concluding
remarks are given in Section 5.
2. Petrov-Galerkin projection and proper orthogonal decomposition
Model reduction aims to replace a complex model with one with fewer differential
equations and state variables. The original model is assumed to be of the form
( ) ( ) ( )
( )
,
d
t t t
dt
=x f x u (1)
where
x
n
x
R
is the state vector and
n
u
R
is the input vector. A common approach for
model reduction is the Petrov-Galerkin projection (Antoulas, 2005), which derives a reduced
model based upon truncation of the model after a linear transformation has been performed
( ) ( ) ( )
( )
T
,
d
t t t
dt
=z L f Kz u (2)
where
z
n
z
R
is the state vector of the reduced model, i.e.
n
z
<
n
x
.
The transformation matrices K and L need to satisfy several conditions: (1) The matrices
have full column rank, as it would otherwise be possible to further reduce the model by
eliminating linear dependent columns, and (2) the matrix L
T
is a left inverse of the matrix K,
satisfying
T
z
n
=
L K I
(3)
4
where I is an identity matrix of dimension
n
z
. This property ensures that the matrix given by
KL
T
is a projection matrix. Though the left inverse matrix is not unique for a given K, it is
often chosen to be the Moore-Penrose pseudo inverse of K (Li and Rabitz, 1990;
Dokoumetzidis and Aarons, 2009)
(
)
1
T T T
+
= =
L K K K K
(4)
which results in a unique choice of L
T
for a given K of full column rank. Using such a
procedure, the model reduction task can be viewed as the computation of a transformation
matrix K so that the reduced model provides an acceptable approximation of the dynamics of
the original model.
Proper orthogonal decomposition (POD) (Sirovich, 1987; Berkooz et al., 1993; Willcox
and Peraire, 2002; Rathinam and Petzold, 2003; Rowley et al., 2004; Romijn et al., 2008)
provides one approach to determine the transformation matrix K via principal component
analysis (PCA) of a matrix of snapshots, i.e. a matrix that contains data points of the states
taken at different points in time derived for a variety of stimulation profiles. In order to
generate a snapshot matrix, different model inputs such as the initial conditions of the state
vector x
0
and the input signals
u
(
t
) need to be determined. The input signal can often be
expressed as a function of a finite set of parameters, e.g., a piecewise constant function, a
polynomial derived from a truncated Taylor series, or a truncated Fourier series expression.
The different input parameters can be concatenated into a vector w and the set of all inputs
under consideration is denoted by . Then, the original model is simulated, generating state
trajectories x(
t
,w) for different conditions w and different points in time
t
. A snapshot matrix
consists of sampling points from the simulated state trajectories, denoted by
( )
(
)
(
)
(
)
1 1 1 1
, , , , , , , , , ,
t w t w
n n n n
t w t w t w t w
 
=
 
X x x x x
L L L
(5)
5
where the values of the states are samples at the sampling points
t
j
and for a set of sampling
parameters w
k
corresponding to an input signal.
If POD is used then the transformation matrix K is determined from the optimal data
compression of the snapshot matrix. The state trajectories in
x
n
R
which are recorded in the
snapshot matrix X, can be compressed by a projection onto a subspace of
z
n
R
, expressed as
PX where P is a projection matrix of rank
n
z
. The projection matrix is determined from the
following optimization problem
( )
2
2
F
min
ε
=
= −
P P
P X PX
(6)
where
ε
is the data compression error evaluated by the Frobenius norm,
(
)
2
T
F
trace=
X XX
.
The optimal solution of the data compression can be obtained using PCA. The cross product
matrix of the data matrix is computed to be
( ) ( )
T
T
1 1
, ,
t w
n n
j k j k
j k
t t
= =
= =
W XX x w x w (7)
which has an eigenvalue decomposition
T
=W V
Λ
V
(8)
where V is an orthogonal matrix with
T
x
n
=
V V I
and
(
)
2 2 2
1 2
diag , , ,
x
n
λ λ λ
=
Λ
L
. The diagonal
entries are ordered such that
2 2 2
1 2
x
n
λ λ λ
≥ ≥L
. Let
n
V
denote the submatrix of
V
which
contains the columns from 1 to
n
z. Then the projection matrix is given by
T
z z
n n
=
P V V
(9)
and the transformation matrix
K
can be found from
T
z
n
=
K V
(10)
6
It can be seen that the key component of POD involves solving a data compression problem
by projecting the state trajectory data on to a linear subspace.
3. Model reduction by state selection
One drawback of POD-based model reduction procedures is that the states of the reduced
model are a linear combination of the states of the original model, i.e., the physical
interpretation of individual states is lost in the process. While it is possible to reconstruct the
original states from the states of the reduced model via a matrix multiplication, the techniques
based upon POD are generally not designed to preserve properties of the original states. This
drawback forms the motivation behind the technique introduced in this work, as the presented
method results in models that retain the most important states of the original model.
3.1. Problem formulation
The presented technique is derived from the Petrov-Galerkin projection as well as POD.
However, additional constraints are placed on the projection matrix to ensure that the states of
the reduced model retain their physical interpretation.
To simplify the notation used for the state selection procedure, the state variables can be
classified into two sets: a set of states which will be retained in the reduced model,
x
r, and a
set of states which will be eliminated in the process,
x
d. It is desired to retain states such that
the trajectories of the states,
z
, of the reduced model shown in Eq. (2) are the same as the
trajectories of the corresponding states of the original model:
r
=
x z
(11)
An approximation of the states that were reduced in the process can be computed by a linear
combination of the retained states:
7
d r
= =
x Tz Tx
(12)
Combining Eq. (11) and Eq. (12) results in the following structure of
K
used in the Petrov-
Galerkin projection
z
n
 
=
 
 
I
K
T
(13)
The order of the states can be changed via multiplication with a permutation matrix
Π
such that
r
d
 
=
 
 
x
Πx
x
(14)
where
z
n
r
x
R
are the states that are retained,
x z
n n
d
xR are the states that will be reduced,
and the number of states in the reduced order model is
n
z
. The state variables retained in the
reduced model are directly given by the permutation matrix.
Appropriate choices for
Π
in Eq. (14) and T in Eq. (12) can be computed by solving the
following data compression problem
(
)
( )
2
F
,
1
T T T
T
min ,
s.t.
z
n
ε
= −
 
=
 
 
=
=
ΠT
ΠPΠX PΠX
I
KT
L K K K
P KL
(15)
where X is the snapshot data matrix calculated according to Eq. (5). This problem is derived
from Eq. (6) by adding the constraints (13) on the projection matrix and including a state
selection procedure via the permutation matrix
Π
.
8
The optimization problem is a mixed integer nonlinear programming (MINLP) which is
nontrivial to solve. However, for a given permutation matrix
Π
, the optimal value of the
transformation matrix T can be calculated analytically by minimizing the objective function
2
F
2
F
2
F
r r
d r
d r
 
 
= −
 
 
 
 
= −
ΠX PΠX
X X
X TX
X TX
(16)
where X
r
is the data matrix of the retained states while X
d
is the data matrix of the reduced
states, the solution of which is given by
(
)
1
T T
d r r r
=T X X X X (17)
Combining Eqs. (15)-(17) results in the following optimization problem
( )
( )
(
)
1
T T
min
t w
d n n r r r r
ε
×
= −
Π
ΠX I X X X X (18)
where only the permutation matrix
Π
needs to be determined, as it directly affects X
r
and X
d
.
3.2. Computation of data compression error
While it is possible to compute the data compression error
(
)
ε
Π
directly via matrix
inversions and multiplications, this procedure can be computationally expensive for large data
matrices. One alternative is to use a modification of the Gram-Schmidt orthogonalization
procedure which avoids computation of the inverse of a matrix:
Algorithm: Compute the data compression error using a modified Gram-Schmidt
orthogonalization procedure
Step 0. Suppose the selected state variables are indexed by
1 2
, , ,
z
n
i i i
L
. Set the initial
value of the projected row vectors
(
)
0
j j
i i
=
x x
, for
1, ,
x
j n
=
L
and the iteration
9
index
k
= 0.
Step 1. Set
k
=
k
+ 1 and
(
)
( ) ( )
( )
1
T
1
T
2
2
1
, for 1, ,
k
j j
x
j
k
k i
k k k k
i i x
k k
n
k
k i
j k
j k n
ε
= +
=
 
= = +
 
 
=
q x
q q
x x I q q
x
L
Step 2. If
k
<
n
z
, return to Step 1.
Step 3. Return the data compression error
z
n
ε ε
=
.
The orthogonalization procedure sequentially constructs a set of orthogonal bases. One
new vector is added to the set of bases at each step and then all remaining vectors are
projected on to the subspace which is orthogonal to the current bases. The projection ensures
that the next basis will be orthogonal to all already computed bases. The value of
k
ε
denotes
the sum of the squared 2-norm of all remaining vectors after the projection.
An important property of the orthogonalization procedure is that the subspace spanned by
the constructed orthogonal bases
1
, ,
k
q q
L
is identical to that spanned by the original vectors
1
, ,
k
i i
x x
L
. One result of this is that the value of
k
ε
is equal to the data compression error
when the states
1
, ,
k
i i
L
are selected. The modified Gram-Schmidt orthogonalization
procedure not only returns the final value of the data compression error but also the values at
each intermediate steps, which will be used to derive a sequential selection procedure.
3.3. Sequential solution of the state selection problem
As has been pointed out, the solution to the combinatorial problem shown in Eq. (18) is
non-trivial. It is generally not possible to perform an exhaustive search as the number of
possible combinations to select
n
z
states for the reduced model from
n
x
states of the original
10
model is
(
)
(
)
! ! !
x z x z
n n n n
, which increases dramatically as the number of states increases.
Instead, a sequential selection algorithm is presented here which is based on the modified
Gram-Schmidt orthogonalization procedure. This procedure transforms the problem of
selecting multiple states simultaneously into a problem of sequentially selecting individual
states. The state variables are selected one at a time and, at the
k
-th step, the state to be
included is the one which results in the smallest data compression error
k
ε
. While
sequentially determining states to be included cannot ensure that the entire set of states will
result in the smallest overall data compression error, this approach has the advantage that only
(
)
2 1 2
z x z
n n n− +
evaluations are required to obtain a solution.
A diagram summarizing the sequential state selection algorithm is shown in Fig. 1. One
difference between the orthogonalization procedure of the algorithm and the one shown in Fig.
1 is that the projection of the data vectors and the calculation of the data compression error are
conducted using the cross product matrix
W
from Eq. (7) instead of the data matrix
X
from
Eq. (5). The reason for this modification is that the data matrix has a large dimension
(
n
x
×
n
t
×
n
w
), whereas only information about the inner product of the data vectors is required to
perform the state selection procedure. The cross product matrix has a significantly smaller
dimension given by (
n
x
×(
n
x
+1)/2). Additionally, only the upper-triangular entries of the
matrix are needed as
W
is a symmetric matrix.
The vector
s
in the selection algorithm in Fig. 1 records the sequence of selected states.
The procedure in Box 1 moves the entries corresponding to the selected state variables to the
top left corner of
W
. After a state variable is added to the selected set, the data vectors of the
unselected states are projected on to the subspace orthogonal to the one spanned by the data
vectors of the selected state. Since the selection and projection operations, performed at each
11
iteration, will be conducted based on the inner product of the projected vectors, the matrix
W
is updated via the procedure shown in Box 2. The inner product of the projected vectors can
be calculated from the current values as follows: Suppose two data vectors
x
i
and
x
j
are
projected on to the subspace orthogonal to another data vector
x
k
. The inner product of
x
i
and
x
j
after projection is given by
( )( )
T
T T
T T
T T
T
T
k k k k
i j
k k k k
i k j k
i j
k k
 
 
− −
 
 
 
 
= −
x x x x
x I x I
x x x x
x x x x
x x x x
(19)
It should be noted that only the values of the inner products before the projection are required
to update the values of the inner products after the projection (see Box 2). In other words,
each entry of W with indices
ij
after the projection, is updated from the
ij
,
ik
,
jk
, and
kk
entries in W before the projection was performed.
As the data compression error, after the selection has been made, is equal to the sum of the
squared norm of all projected vectors, the compression error is calculated by adding the
corresponding diagonal entries in W via the procedure shown in Box 3 in Fig. 1. The
procedure then determines the addition of which state will result in the farthest reduction of
the compression error, as is shown in Box 4 and the selected state and the corresponding error
value are recorded in Box 5.
The algorithm finally returns the sequence of selected states recorded in the vector
[
]
1:
z
n
s
as well as the corresponding data compression errors recorded in the vector
ε
. The
unselected states are recorded in the vector
[
]
1:
z x
n n
+s
.
12
The state selection algorithm determines which states should be retained in the reduced
model from the information of the inner product matrix W and this algorithm forms the core
component of the presented model reduction technique, shown in Fig. 2.
The first part of the procedure generates a data set from simulations for the state selection
algorithm. A set of input signals,
{
}
k
w
, is used to excite the model around its steady state.
The set can be chosen to include specific signals of interest or can be randomly sampled from
a region under consideration. For example, if the upper bound and the lower bound of w are
available, then the values of w
k
can be generated by uniform sampling within this region.
After the set of input signals has been determined, the model is simulated for each input signal
and the state trajectories recorded. The time interval used for the simulation should be
sufficiently long to capture the dynamic behavior of the system, while the sampling interval
should be chosen small enough such that the sampling points represent fairly continuous state
trajectories.
The collected data is used to compute the cross product matrix W according to Eq. (7) and
the states to be retained in the reduced model are selected according to the state selection
algorithm based on the calculated W. As only the cross product matrix W is stored, instead of
the original data matrices, the matrix T is computed by Eq. (20).
[ ] [ ] [ ] [ ]
(
)
1
1: : 1: 1: : 1:
z x z z z
n n n n n
 
= +
 
T W s s W s s
(20)
The final step of the reduction procedure is to determine the number of states,
n
z
, of the
reduced model. It is desirable to set the number as small as possible while retaining a
reasonably accurate model. As it is not possible to determine the number of states
a priori
for
a nonlinear model, a trial-and-error procedure needs to be applied. Fortunately, the sequential
state selection procedure is computationally inexpensive, which facilitates the use of such a
13
trial-and-error procedure. After determining the number of retained states it is straightforward
to calculate the transformation matrices and conduct the Petrov-Galerkin projection to
compute the reduced model.
However, similar to POD, the data compression error is not identical to the actual model
approximation error. The data compression error is the difference between the state trajectory
data and the projection of these data onto a subspace. Even though the transformation
matrices calculated from data compression are used in the Petrov-Galerkin projection to
derive the reduce model, the state trajectories of the reduced model are not identical to the
projection of the state trajectories from the original model. One result of this is that the data
compression error is just an
a priori
estimate of the reduction error and the actual error, called
the
a posteriori
error in the following, will still need to be computed.
The
a posteriori
model approximation error is
2
F
e
= −
X KZ
(21)
where Z is the data matrix calculated from state trajectories of the reduced model, recorded in
the same way as X. The transformation KZ denotes the computed state values of both the
retained states and the reduced states of the reduced model. If the
a posteriori
error is not
acceptable then the number of selected states needs to be enlarged. On the other hand, if the
a
posteriori
error is small then it may be possible to further reduce the model. As the model
reduction scheme selects states sequentially, it is straightforward to either add another state to
the already selected list of states or to delete one of the states. The changes made to the model
can then be evaluated by recomputing the
a posteriori
error.
14
4. Case studies
4.1. Linear 4-compartment model
The first example deals with a linear 4-compartment model (Sherwin et al., 1974). A
diagram of the model is shown in Fig. 3(a). The differential equations describing the model
are
(
)
( )
10 12 13 14 21 31 41
1 1
12 21
2 2
4
13 31
3 3
14 40 41
4 4
0
0 0
0
0 0
0
0 0
1
k k k k k k kC C
k kC C
d
I
k k
C C
dt
k k k
C C
+ + + 
   
 
 
   
 
 
   
 
= +
 
   
 
 
   
 
− +
 
 
   
 
(22)
where the state variables C
i
correspond to the concentration in the i-th compartment and the
values of the kinetic parameters are shown in Fig. 3(a).
The system is stimulated by raising the inflow I
4
from zero to 1.00 mU/min. The system is
simulated from t = 0 to t = 120 min and the states are sampled every 5 minutes. Since the
model is linear, only one input signal is required to compute the cross product matrix W:
40.95 14.54 53.58 106.11
14.54 5.17 19.23 37.55
53.58 19.23 79.24 136.73
106.11 37.55 136.73 275.94
 
 
 
=
 
 
 
W (23)
Applying the state selection algorithm to W, the states are selected in the order C
1
, C
3
, C
4
and C
2
. For example, if one wants to reduce the model from four to three states then C
2
is
reduced in the process and the resulting differential equations of the simplified model are
given by
1 1
3 3
4 4
0.320 0.012 0.099 0.07
0.041 0.020 0.000 0.00
0.250 0.000 0.384 0.99
C C
dC C
dt C C
 
 
= − +
 
 
 
(24)
15
The diagram representing the reduced model from Eq. (24) is shown Fig. 3(b). Similarly, if
only two states are to be retained then the reduced model consists of the states C
1
and C
3
.
When the 3-state model (Fig. 3(b)) is constructed the reduced state is the concentration of
compartment 2. However, the reduced model is not simply obtained by removing the
compartment 2 as the values of the rate constants are also changed, due to the Petrov-Galerkin
procedure (2). It can be seen from the input vector in Eq. (24) that the inflow rate to
compartment 4 is decreased while the inflow into compartment 1 is slightly increased and a
sink term is added to compartment 3. Similar changes occur if the model is further reduced,
(Fig. 3(c) and Fig. 3(d)).
To check the model approximation error the a priori error and the a posteriori errors are
displayed in Fig. 4 along with the state trajectories for each reduced model. From the
reduction error it can be seen that the reduced models with 3 states or 2 states approximate the
original model well, however, the model with only 1 state results in significantly larger errors.
4.2. Binary distillation column
The second example deals with a distillation column that has 30 trays for the separation of
a binary mixture. The state variables are given by the liquid compositions on the 30 trays, in
the reflux drum, and in the reboiler. The column is assumed to have a constant relative
volatility of 1.6, and symmetric product compositions. The feed stream is introduced at the
column on stage 17 and has a composition of 0.5. The reflux ratio is the adjustable input, u,
with a nominal value of 3.0 and a range from 1.0 to 5.0. A detailed description of the model
can be found in the literature (Benallou et al., 1986; Hahn and Edgar, 2002a).
A data set is created by simulating the model for different values of the input u ranging
from 1.0 to 5.0 in 0.04 increments. The matrix W is computed according to Eq. (7) from the
16
generated data sets. The a priori approximation error (data compression error) and the a
posteriori approximation error (model approximation error) are listed in Table 1 for different
numbers of retained states. The threshold value of the reduction error, used to determine the
number of states retained in the reduced model, is set to 2%. If 3 states are retained in the
model then the a priori is equal to 0.65%. However, the a posteriori error for this case is
2.20% according to Eq. (21) and it requires 4 states to have an a posteriori error of 1.03%.
The selected 4 states for the reduced model are the concentration of one component in the
reflux drum, and on the 3
rd
, 8
th
, 22
nd
tray. All other 28 states can be adequately expressed by
linear combinations of these 4 states. To visualize the performance, Fig. 5 shows the
difference between the original model and the reduced model when the input is changed from
the original value to 2.0 in one case and to 4.0 in a second simulation. In both cases, the
reduced model is able to accurately approximate the original model.
For comparison purposes, the approximation errors returned by this model reduction
technique are compared to those for a model that is reduced by POD. The results are
summarized in Table 1. Not surprisingly POD performs slightly better as it represents the
optimal solution to the data compression problem given by Eq. (6). However, the differences
between the model derived by the presented technique and the one reduced by POD are very
small for this case. Furthermore, the presented technique has the advantage that the physical
interpretation of all the states is retained, while this is not the case when POD is used. Though
not shown in this work, the same conclusion can be drawn for the previous example.
5. Conclusion
This paper presents a model reduction technique that is based upon selecting some of the
states of the model to be retained in the reduced model, while the remaining states can be
17
expressed as a linear combination of these retained states. The technique is based on the
Petrov-Galerkin projection and the projection matrix is computed from data derived from the
original model. However, unlike what is done using POD, some constraints are added to the
projection matrix to preserve a subset of original state variables, resulting in a combinatorial
problem. This combinatorial problem is solved sub-optimally using a sequential algorithm
based upon a modified version of the Gram-Schmidt orthogonalization procedure. The
advantage of using a sub-optimal solution is that the solution is computationally inexpensive.
Two examples are shown to illustrate the presented reduction technique.
Acknowledgment
The authors gratefully acknowledge partial financial support from the National Science
Foundation (Grant CBET# 0941313) and the ACS Petroleum Research Fund (Grant PRF#
48144-AC9).
18
References
Antoulas, A.C., 2005. Approximation of large-scale dynamical systems. Society for Industrial
and Applied Mathematics, Philadelphia.
Benallou, A., Seborg, D.E., Mellichamp, D.A., 1986. Dyanmic compartmental models for
separation processes. AIChE Journal 32 (7), 1067-1078.
Berkooz, G., Holmes, P., Lumley, J.L., 1993. The Proper Orthogonal Decomposition in the
Analysis of Turbulent Flows. Annual Review of Fluid Mechanics 25, 539-575.
Conzelmann, H., Fey, D., Gilles, E.D., 2008. Exact model reduction of combinatorial reaction
networks. BMC Systems Biology 2, 78.
Dano, S., Madsen, M.F., Schmidt, H., Cedersund, G., 2006. Reduction of a biochemical
model with preservation of its basic dynamic properties. FEBS Journal 273 (21), 4862-
4877.
Deane, A.E., Kevrekidis, I.G., Karniadakis, G.E., Orszag, S.A., 1991. Low-dimensional
models for complex geometry flows: Application to grooved channels and circular
cylinders, Physics of Fluids. A, Fluid Dynamics 3 (10), 2337-2354.
Dokoumetzidis, A., Aarons, L. 2009. Proper lumping in systems biology models. IET
Systems Biology 3 (1), 40-51.
Hahn, J., Edgar, T.F., 2002a. An improved method for nonlinear model reduction using
balancing of empirical gramians. Computers & Chemical Engineering 26 (10), 1379-1397.
Hahn, J., Edgar, T.F., 2002b. Balancing approach to minimal realization and model reduction
of stable nonlinear systems. Industrial & Engineering Chemistry Research 41 (9), 2204-
2212.
Ho, T.C. 2008. Kinetic modeling of large-scale reaction systems. Catalysis Reviews-Science
and Engineering 50 (3), 287-378.
19
Lall, S., Marsden, J.E., Glavaski, S., 2002. A subspace approach to balanced truncation for
model reduction of nonlinear control systems. International journal of robust and
nonlinear control 12 (6), 519-535.
Li, G., Rabitz, H., 1990. A general analysis of approximate lumping in chemical kinetics.
Chemical Engineering Science 45 (4), 977-1002.
Lu, T.F., Law, C.K., 2009. Toward accommodating realistic fuel chemistry in large-scale
computations. Progress in Energy and Combustion Science 35 2, 192-215.
Maria G., 2004. A review of algorithms and trends in kinetic model identification for
chemical and biochemical systems. Chemical and Biochemical Engineering Quarterly 18
(3): 195-222.
Marquardt, W., 2001. Nonlinear Model Reduction for Optimization Based Control of
Transient Chemical Processes. Proceedings of the CPC VI, 30–60. Tucson, Arizona.
Moore, B., 1981. Principal component analysis in linear systems: Controllability,
observability, and model reduction. IEEE Transactions on Automatic Control 26 (1), 17 -
32.
Nagy, T., Turanyi, T., 2009. Reduction of very large reaction mechanisms using methods
based on simulation error minimization. Combustion and Flame 156 (2), 417-428.
Okino, M.S., Mavrovouniotis, M.L., 1998. Simplification of mathematical models of
chemical reaction systems. Chemical Reviews 98 (2), 391-408.
Rathinam, M., Petzold, L.R., 2003. A new look at proper orthogonal decomposition. SIAM
Journal on Numerical Analysis 41 (5), 1893-1925.
Romijn, R., Ozkan, L., Weiland, S., Ludlage, J., Marquardt, W., 2008. A grey-box modeling
approach for the reduction of nonlinear systems. Journal of Process Control 18 (9), 906-
914.
20
Rowley, C.W., Colonius, T., Murray, R.M., 2004. Model reduction for compressible flows
using POD and Galerkin projection. Nonlinear Phenomena 189 (1-2), 115-129.
Ryckelynck, D., Chinesta, F., Cueto, E., Ammar, A., 2006. On the a priori model reduction:
Overview and recent developments. Archives of Computational Methods in Engineering
13 (1): 91-128.
Sherwin, R.S., Kramer, K.J., Tobin, J.D., Insel, P.A., Liljenquist, J.E., Berman, M., Andres,
R., 1974. A model of the kinetics of insulin in man. Journal of Clinical Investigation 53
(5), 1481-1492.
Sirovich, L. 1987. Turbulence and the dynamics of coherent structures. I-coherent structures.
Quarterly of Applied Mathematics 45, 561-571.
Skogestad, S., Morari, M., 1988. Understanding the dynamic behavior of distillation columns.
Industrial & Engineering Chemistry Research 27 (10), 1848-1862.
Vajda, S, Valko, P., Turanyi, T., 1985. Principal component analysis of kinetic models.
International Journal of Chemical Kinetics 17 (1), 55-81.
Wang, H., Frenklach, M., 1997. A detailed kinetic modeling study of aromatics formation in
laminar premixed acetylene and ethylene flames. Combustion and Flame 110 (1-2), 173-
221.
Willcox, K., Peraire, J., 2002. Balanced model reduction via the proper orthogonal
decomposition. AIAA Journal 40 (11), 2323-2330.
21
Table 1
The a priori approximation error
ε
, and the a posteriori approximation error, e, returned by
the state selection method and POD. The values are listed for several different number of
states preserved in the reduced model and normalized by the norm of all state trajectories of
the original model.
number of states 1 2 3 4 5 6 7
state
selection
ε
% 15.76 3.42 0.65 0.18 0.11 0.07 0.04
e % 19.66 8.81 2.20 1.03 0.95 0.79 0.29
POD
ε
% 15.62 2.81 0.47 0.13 0.08 0.05 0.02
e % 19.75 8.67 2.07 1.03 0.90 0.86 0.33
22
[
]
[
]
[ ] [ ]
[ ] [ ]
, ,
, 1: 1 1 : 1,
, 1: , 1:
l l
l l
l x l x
k k l l
k k l k l l
k l n l l n
+ − + −
+ ↔ +
W W
W W
W W
[ ] [ ]
[
]
[
]
[ ]
, ,
, , ,
l l
l l
l
k i k j
i j i j k k
= W W
W W W
[ ]
1
,
x
n
l l
i k
r i i
= +
=
W
(
)
*
*
arg min ,
k
l
l
l
l r= =
W W
[
]
[
]
*
*
,
l
k l k r
 
↔ =
 
s s ε
Fig. 1. Diagram of the sequential state selection algorithm. A subset of n
z
state variables is
selected from all n
x
state variables based on W, the cross product matrix of the data matrix.
The notation W[i, j] returns the entry in the i-th row and j-th column of W. The notation ‘
refers to the values of two vectors being switched.
23
Fig. 2. Outline of model reduction algorithm
24
(
)
(
)
(
)
(
)
2 1 3 4
0.741 0.010 0.144
C t C t C t C t
= − −
(a) (b)
(
)
(
)
(
)
( ) ( ) ( )
2 1 3
4 1 3
0.325 0.023
2.895 0.232
C t C t C t
C t C t C t
= +
= −
(
)
(
)
( ) ( )
( ) ( )
2 1
3 1
4 1
0.325
1.309
2.591
C t C t
C t C t
C t C t
=
=
=
(c) (d)
Fig. 3. Diagram of compartment models with different number of states; (a): original 4-state
model, (b) reduced model with 3 states, (c) reduced model with 2 states, and (d) reduced
model with 1 state.
25
0 20 40 60 80 100 120
0
1
2
3
4
5
C
1
C
2
C
3
C
4
time, min
concentration, mU
model : 4-state : 3-state : 2- state : 1-state
a priori error 0 0.0002 0.4965 10.120
a pos teriori error 0 0.0169 0.1809 11.727
Fig. 4. Comparison of the states described by the original model and 3 reduced-order models.
26
0 20 40 60 80 100 120
0
0.2
0.4
0.6
0.8
1
time, min
composition
(a)
0 20 40 60 80 100 120
0
0.2
0.4
0.6
0.8
1
time, min
composition
(b)
Fig. 5. State trajectories of the original model (solid line) and the ones recovered from the
reduced model (dash line) in the case of (a) u = 2 and (b) u = 4 (u
ss
= 3).
... For unsteady flow problems, Sirisup & Karniadakis proposed a modified POD model by introducing penalty functions [11]. Most of these work focus on the FSI problem with a moving fluid structure interface and therefore the deformation of the solid domain is still lacking of research via the reduced order modelling approach [15][16][17][18][19]. ...
... In the Petrov-Galerkin projection approach, the original equations describing the system are projected onto the basis in order to determine the reduced-order system, which can be solved for the coefficients. However, this approach is code intrusive (requires extensive source code modifications) and cannot be used with black-box solvers Intrusive ROM approaches can suffer from inefficiencies dealing with nonlinear problems and instability issues [11][12][13] remedies for these issues also being proposed in literature [14][15][16][17]. An alternative nonintrusive approach is to project only the snapshots onto the basis and determine a finite set of POD coefficients corresponding to those snapshots, and then use other surrogate techniques such as Kriging, artificial neural networks (ANNs) or radial basis functions (RBFs) as closures to model the full POD coefficient space based on the available set [18,19]. ...
Article
A non-intrusive reduced-order model for nonlinear parametric flow problems is developed. It is based on extracting a reduced-order basis from high-order snapshots via proper orthogonal decomposition and using multi-layered feedforward artificial neural networks to approximate the reduced-order coefficients. The model is a generic and efficient approach for the reduction of time-dependent parametric systems, including those described by partial differential equations. Since it is non-intrusive, it is independent of the high-order computational method and can be used together with black-box solvers. Numerical studies are presented for steady-state isentropic nozzle flow with geometric parameterisation and unsteady parameterised viscous Burgers equation. An adaptive sampling strategy is proposed to increase the quality of the neural network approximation while minimising the required number of parameter samples and, as a direct consequence, the number of high-order snapshots and the size of the network training set. Results confirm the accuracy of the non-intrusive approach as well as the speed-up achieved compared with intrusive hyper reduced-order approaches.
... Intrusive ROMs have been reported to suffer from instability, and when applied to non-linear problems, the resulting ROMs can be inefficient [16,17,18,19,20]. To tackle these issues, a number of stabilisation methods have been developed, for example; Petrov-Galerkin methods [21,22], calibration methods [23,24,25], regularisation methods [26] and Fourier expansion methods [27]. Also, various non-linearity model reduction methods have been proposed such as the empirical interpolation method (EIM) [28], discrete EIM (DEIM) [19], residual DEIM (RDEIM) [29], least squares Petrov−Galerkin projection methods [24], a Gauss−Newton method with approximated tensors (GNAT) [30] and the quadratic expansion method [31]. ...
Article
Full-text available
In this paper we present a new domain decomposition non-intrusive reduced order model (DDNIROM) for the Navier–Stokes equations. The computational domain is partitioned into subdomains and a set of local basis functions is constructed in each subdomain using Proper Orthogonal Decomposition (POD). A radial basis function (RBF) method is then used to generate a set of hypersurfaces for each subdomain. Each local hypersurface represents, not only the fluid dynamics over the subdomain to which it belongs, but also the interactions with the surrounding subdomains. This implicit coupling between the subdomains provides the global coupling necessary to enforce incompressibility and is a means of providing boundary conditions for each subdomain. The performance of this DDNIROM is illustrated numerically by three examples: flow past a cylinder, and air flow over 2D and 3D street canyons. The results show that the DDNIROM exhibits good agreement with the high-fidelity full model while the computational cost is reduced by several orders of magnitude. The domain decomposition (DD) method provides the flexibility to choose different numbers of local basis functions for each subdomain depending on the complexity of the flow therein. The fact that the RBF surface representation takes input only from its current subdomain and the surrounding subdomains, means that, crucially, there is a reduction in the dimensionality of the hypersurface when compared with a more traditional, global NIROM. This comes at the cost of having a larger number of hypersurfaces.
... However, this method suffers from numerical instability (Franca and Frey, 1992). Various methods have been proposed to overcome or improve the stability issue of the POD/Galerkin projection method, including nonlinear Petrov-Galerkin Xiao et al., 2013), regularisation (Jafarpour and Sabetghadam, 2012), sub grid-scale modelling, calibration (Bou-Mosleh et al., 2011;Serpas et al., 2011) and Fourier expansion (Willcox and Megretski, 2003). Another issue that arises in the ROMs is the efficient treatment of nonlinear terms in the partial differential equations (PDEs). ...
Article
Full-text available
A non-intrusive model reduction computational method using hypersurfaces representation has been developed for reservoir simulation and further applied to 3D fluvial channel problems in this work. This is achieved by a combination of a radial basis function (RBF) interpolation and proper orthogonal decomposition (POD) method. The advantage of the method is that it is generic and non-intrusive, that is, it does not require modifications to the original complex source code, for example, a 3D unstructured mesh control volume finite element (CVFEM) reservoir model used here. The capability of this non-intrusive reduced order model (NIROM) based on hypersurfaces representation has been numerically illustrated in a horizontally layered porous media case, and then further applied to a 3D complex fluvial channel case. By comparing the results of the NIROM against the solutions obtained from the high fidelity full model, it is shown that this NIROM results in a large reduction in the CPU computation cost while much of the details are captured.
... In addition, the IROM has instability and non-linear inefficiency issues [33,28,26]. Those instability and non-linear inefficiency issues could affect the accuracy and efficiency of generating ensembles although some efforts have been made to improve the stability and the non-linearity efficiency of ROMs [22,41,34,15,16,38,6,3,10,13,37,5]. In order to tackle these issues in intrusive ROMs, recently, non-intrusive reduced order models (NIROMs) have been introduced [36,39,35]. ...
Article
Full-text available
This paper presents a novel Ensemble Kalman Filter (EnKF) data assimilation method based on a parameterised non-intrusive reduced order model (P-NIROM) which is independent of the original computational code. EnKF techniques involve the expensive calculations of ensembles. In this work, the recently developed P-NIROM Xiao et al. [40] is incorporated into EnKF to speed up the ensemble simulations. A reduced order flow dynamical model is generated from the solution snapshots, which are obtained from a number of the high fidelity full simulations over the specific parametric space RP. The varying parameter is the background error covariance σ ∈ RP. Using the Smolyak sparse grid method, a set of parameters in the Gaussian probability density function is selected as the training points. The proposed method uses a two-level interpolation method for constructing the P-NIROM using a Radial Basis Function (RBF) interpolation method. The first level interpolation approach is used for generating the solution snapshots and POD basis functions for any given background error covariance while the second level interpolation approach for forming a set of hyper-surfaces representing the reduced system. The EnKF in combination with P-NIROM (P-NIROM-EnKF) has been implemented within an unstructured mesh finite element ocean model and applied to a three dimensional wind driven circulation gyre case. The numerical results show that the accuracy of ensembles and updated solutions using the P-NIROM-EnKF is maintained while the computational cost is significantly reduced by several orders of magnitude in comparison to the full-EnKF.
... However, this method suffers from numerical instability (Franca and Frey, 1992). Various methods have been proposed to overcome or improve the stability issue of the POD/Galerkin projection method, including nonlinear Petrov-Galerkin Xiao et al., 2013), regularisation (Jafarpour and Sabetghadam, 2012), sub grid-scale modelling, calibration (Bou-Mosleh et al., 2011;Serpas et al., 2011) and Fourier expansion (Willcox and Megretski, 2003). Another issue that arises in the ROMs is the efficient treatment of nonlinear terms in the partial differential equations (PDEs). ...
... However, this method suffers from numerical instability (Franca and Frey, 1992). Various methods have been proposed to overcome or improve the stability issue of the POD/Galerkin projection method, including nonlinear Petrov-Galerkin Xiao et al., 2013), regularisation (Jafarpour and Sabetghadam, 2012), sub grid-scale modelling, calibration (Bou-Mosleh et al., 2011;Serpas et al., 2011) and Fourier expansion (Willcox and Megretski, 2003). Another issue that arises in the ROMs is the efficient treatment of nonlinear terms in the partial differential equations (PDEs). ...
... A further problem is that most are derived using POD and/or Galerkin methods. These approaches tend to suffer from non-linear inefficiency and instability issues ( [16,17,18,19,20,21,22,23,24,25,26,27]). ...
Preprint
Full-text available
A novel variable-material non-intrusive reduced order model (NIROM) based on a Smolyak sparse grid interpolation method, a radial basis function (RBF) method and proper orthogonal decomposition (POD) has been developed for oil reservoirs with uncertain rock properties. This novel NIROM is constructed by using a two level interpolation method. The first level interpolation is constructed for the rock properties, and the second level is for the fluid dynamics. The NIROM is independent of governing equations, therefore, this method is easy to implement and easy to be extended for other applications as it does not require modifications to the source code. The novelty of this work is the use of the presented Smolyak gird and RBF interpolation based NIROM for oil reservoirs with uncertain rock properties. Another novelty is the use of Smolyak sparse grid to reduce the NR realisations of the reservoir simulator for constructing the NIROM and reduce the computational effort involved in creating the hyperspace of basis functions and coefficients from the snapshots of all the different realisations. The capability of this new NIROM has been numerically illustrated in two multiphase flows in porous media: a reservoir with eight baffles case and a 3D fluvial channel case with 22 uncertainties. By comparing the results of the novel NIROM against the solutions obtained from the high fidelity full model, it is shown that this model can result in a large reduction in the CPU cost (by a factor of about three orders) while much of the details of multiphase flow in porous media are captured.
... Various linear and non-linear versions and generalisations of the principal component analysis are applied for dimensionality reductions of snapshots [68]. In particular, a modification of the POD with preservation of some original state variables is proposed [69]. ...
Article
To help create a comfortable and healthy indoor and outdoor environment in which to live, there is a need to understand turbulent air flows within the urban environment. To this end, building on a previously reported method [1], we develop a fast-running Non-Intrusive Reduced Order Model (NIROM) for predicting the turbulent air flows found within an urban environment. To resolve larger scale turbulent fluctuations, we employ a Large Eddy Simulation (LES) model and solve the resulting computational model on unstructured meshes. The objective is to construct a rapid-running NIROM from these results that will have ‘similar’ dynamics to the original LES model. Based on Proper Orthogonal Decomposition (POD) and machine learning techniques, this Reduced Order Model (ROM) is six orders of magnitude faster than the high-fidelity LES model and we demonstrate how ‘similar’ it can be to the high-fidelity model by comparing statistical quantities such as the mean flows, Reynolds stresses and probability densities of the velocities. We also include validation of the high-fidelity model against data from wind tunnel experiments. This paper represents a key step towards the use of reduced order modelling for operational purposes with the tantalising possibility of it being used in place of Gaussian plume models, and the potential for greatly improved model fidelity and confidence.
Article
Full-text available
Simulation of complex (bio)chemical reactions plays an important role in a process kinetics characterization. However, detailed kinetic modelling is a difficult task because the model has to reflect the process complexity under variate operating conditions, starting from a limited number of observed variables, (non-)conventional data recorded with a limited sampling frequency, and often with a low reproducibility. Extensive investigations can lead to structured models of complexity depending on the utilisation scope. To overcome weak results, the identification problem must be well formulated, data consistent, numerical estimation appropriate and effective, and the estimate quality analysis ad- equate. While statistical estimation theory has been extensively developed in terms of objective function choice and solution analysis, numerical algorithm application to (bio)chemical kinetic systems presents particularities and difficulties. The present paper aims to review the main steps and trends in solving the kinetic model identification problem. Rules for a suitable problem formulation vs. modelling objectives, advanced numerical algorithms for obtaining a reliable solution, and an estimate of suitable analysis are shortly summarised.
Article
Two-dimensional unsteady flows in complex geometries that are characterized by simple (low-dimensional) dynamical behavior are considered. Detailed spectral element simulations are performed, and the proper orthogonal decomposition is applied to the resulting data for two examples: the flow in a periodically grooved channel and the wake of an isolated circular cylinder. Low-dimensional dynamical models for these systems are obtained using the empirically derived global eigenfunctions in the spectrally discretized Navier-Stokes equations. The short- and long-term accuracy of the models is studied through simulation, continuation, and bifurcation analysis. Their ability to mimic the full simulations for Reynolds numbers beyond the values used for eigenfunction extraction is evaluated.
Article
Two-dimensional unsteady flows in complex geometries that are characterized by simple (low-dimensional) dynamical behavior are considered. Detailed spectral element simulations are performed, and the proper orthogonal decomposition or POD (also called method of empirical eigenfunctions) is applied to the resulting data for two examples: the flow in a periodically grooved channel and the wake of an isolated circular cylinder. Low-dimensional dynamical models for these systems are obtained using the empirically derived global eigenfunctions in the spectrally discretized Navier-Stokes equations. The short- and long-term accuracy of the models is studied through simulation, continuation, and bifurcation analysis. Their ability to mimic the full simulations for Reynolds numbers (Re) beyond the values used for eigenfunction extraction is evaluated. In the case of the grooved channel, where the primary horizontal wave number of the flow is imposed from the channel periodicity and so remains unchanged with Re, the models extrapolate reasonably well over a range of Re values. In the case of the cylinder wake, however, due to the significant spatial wave number changes of the flow with the Re, the models are only valid in a small neighborhood of the decompositional Reynolds number.
Article
In today's technological world, physical and artificial processes are mainly described by mathematical models, which can be used for simulation or control. These processes are dynamical systems, as their future behavior depends on their past evolution. The weather and very large scale integration (VLSI) circuits are examples, the former physical and the latter artificial. In simulation (control) one seeks to predict (modify) the system behavior; however, simulation of the full model is often not feasible, necessitating simplification of it. Due to limited computational, accuracy, and storage capabilities, system approximation—the development of simplified models that capture the main features of the original dynamical systems—evolved. Simplified models are used in place of original complex models and result in simulation (control) with reduced computational complexity. This book deals with what may be called the curse of complexity, by addressing the approximation of dynamical systems described by a finite set of differential or difference equations together with a finite set of algebraic equations. Our goal is to present approximation methods related to the singular value decomposition (SVD), to Krylov or moment matching methods, and to combinations thereof, referred to as SVD-Krylov methods. Part I addresses the above in more detail. Part II is devoted to a review of the necessary mathematical and system theoretic prerequisites. In particular, norms of vectors and (finite) matrices are introduced in Chapter 3, together with a detailed discussion of the SVD of matrices. The approximation problem in the induced 2-norm and its solution given by the Schmidt—Eckart—Young—Mirsky theorem are tackled next. This result is generalized to linear dynamical systems in Chapter 8, which covers Hankel-norm approximation. Elements of numerical linear algebra are also presented in Chapter 3. Chapter 4 presents some basic concepts from linear system theory. Its first section discusses the external description of linear systems in terms of convolution integrals or convolution sums. The section following treats the internal description of linear systems. This is a representation in terms of first-order ordinary differential or difference equations, depending on whether we are dealing with continuous- or discrete-time systems. The associated structural concepts of reachability and observability are analyzed. Gramians, which are important tools for system approximation, are introduced in this chapter and their properties are explored. The last section of Chapter 4 is concerned with the relationship between internal and external descriptions, which is known as the realization problem. Finally, aspects of the more general problem of rational interpolation are displayed.
Article
This review gives a brief account of basic ideas underlying approaches to analysis and modeling of large‐scale reaction systems. The emphasis is on model simplification and mechanism/dimension reduction via heuristic concepts and formal mathematical techniques. Among the key topics discussed are: top‐down and bottom‐up modeling approaches, graph/matrix representation of chemical reactions, mechanistic vs. pathways models, quantitative structure‐reactivity relationships, mathematical reduction of dimensionality, high‐fidelity surrogate models, continuum approximation, lumping of nonlinear kinetics, overall behavior/kinetics of many reactions, effect of pore diffusion, steady state multiplicity and stability. Some common features of dimension reduction methodologies are noted. Areas where further work will be valuable are identified.