PreprintPDF Available

Reduced model-error source terms for fluid flow

Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

It is well known that the wide range of spatial and temporal scales present in geophysical flow problems represents a (currently) insurmountable computational bottleneck, which must be circumvented by a coarse-graining procedure. The effect of the unresolved fluid motions enters the coarse-grained equations as an unclosed forcing term, denoted as the 'eddy forcing'. Traditionally, the system is closed by approximate deterministic closure models, i.e. so-called parameterizations. Instead of creating a deterministic parameterization, some recent efforts have focused on creating a stochastic, data-driven surrogate model for the eddy forcing from a (limited) set of reference data, with the goal of accurately capturing the long-term flow statistics. Since the eddy forcing is a dynamically evolving field, a surrogate should be able to mimic the complex spatial patterns displayed by the eddy forcing. Rather than creating such a (fully data-driven) surrogate, we propose to precede the surrogate construction step by a procedure that replaces the eddy forcing with a new model-error source term which: i) is tailor-made to capture spatially-integrated statistics of interest, ii) strikes a balance between physical insight and data-driven modelling , and iii) significantly reduces the amount of training data that is needed. Instead of creating a surrogate for an evolving field, we now only require a surrogate model for one scalar time series per statistical quantity-of-interest. Our current surrogate modelling approach builds on a resampling strategy, where we create a probability density function of the reduced training data that is conditional on (time-lagged) resolved-scale variables. We derive the model-error source terms, and construct the reduced surrogate using an ocean model of two-dimensional turbulence in a doubly periodic square domain.
Content may be subject to copyright.
3rd ECCOMAS Thematic Conference on
Uncertainty Quantification in Computational Sciences and Engineering
M. Papadrakakis, V. Papadopoulos, G. Stefanou (eds.)
Crete, Greece, 24-26 June 2019
Wouter Edeling1and Daan Crommelin 1,2
1Centrum Wiskunde & Informatica, Scientific Computing Group
Science Park 123, 1098 XG Amsterdam, The Netherlands
e-mail: {Wouter.Edeling, Daan.Crommelin}
2Korteweg-de Vries Institute for Mathematics, University of Amsterdam
Science Park 105-107, 1098 XG Amsterdam, The Netherlands
Keywords: Model error, data-driven surrogate models, ocean flow
Abstract. It is well known that the wide range of spatial and temporal scales present in
geophysical flow problems represents a (currently) insurmountable computational bottleneck,
which must be circumvented by a coarse-graining procedure. The effect of the unresolved fluid
motions enters the coarse-grained equations as an unclosed forcing term, denoted as the ’eddy
forcing’. Traditionally, the system is closed by approximate deterministic closure models, i.e.
so-called parameterizations. Instead of creating a deterministic parameterization, some recent
efforts have focused on creating a stochastic, data-driven surrogate model for the eddy forcing
from a (limited) set of reference data, with the goal of accurately capturing the long-term flow
statistics. Since the eddy forcing is a dynamically evolving field, a surrogate should be able to
mimic the complex spatial patterns displayed by the eddy forcing. Rather than creating such a
(fully data-driven) surrogate, we propose to precede the surrogate construction step by a proce-
dure that replaces the eddy forcing with a new model-error source term which: i) is tailor-made
to capture spatially-integrated statistics of interest, ii) strikes a balance between physical in-
sight and data-driven modelling , and iii) significantly reduces the amount of training data that
is needed. Instead of creating a surrogate for an evolving field, we now only require a surrogate
model for one scalar time series per statistical quantity-of-interest. Our current surrogate mod-
elling approach builds on a resampling strategy, where we create a probability density function
of the reduced training data that is conditional on (time-lagged) resolved-scale variables. We
derive the model-error source terms, and construct the reduced surrogate using an ocean model
of two-dimensional turbulence in a doubly periodic square domain.
Wouter Edeling and Daan Crommelin
In the numerical simulation of coarse-grained turbulent flow problems one has to cope with
small-scale processes which cannot be resolved directly on the numerical grid. The effect of the
unresolved eddy field enters the resolved-scale equations as an unclosed forcing term, denoted
as the eddy forcing, which is highly complex, dynamic, and shows intricate spatio-temporal
correlations. Traditionally, the eddy forcing is approximated by deterministic closure models,
i.e. so-called parameterizations. In the context of geophysical flows, such parameterizations are
based on e.g. the work of Gent-McWilliams [6], or through the inclusion of a tunable (hyper)
viscosity term meant to damp the smallest resolved scales of the model [11].
It is well known that no parameterization scheme is perfect, and attempts have been made to
improve their performance. For instance, the authors of [15] analysed the transfer of energy and
enstrophy in spectral space for a number of parameterizations, and compared their performance
to a high-fidelity reference solution of a two-dimensional turbulent flow case. They proposed a
deterministic ’energy fixer’ scheme, based on adding a weighted vorticity pattern to the com-
puted vorticity field. Recently, data-driven techniques have been applied as well. For instance
the recent work of [10] used artificial neural networks to learn the eddy forcing from a set of
high-fidelity snapshots.
However, a general limitation of such deterministic approaches is their inability to repre-
sent the strong non-uniqueness of the unresolved scales with respect to the resolved scales
[1, 16, 12]. Since the resolved scales are generally defined as the convolution of the full-scale
solution with some filter, multiple unresolved states can correspond to the same resolved solu-
tion. Thus, in general there is no one-to-one correspondence between the resolved-scale state
and the unresolved-scale state, and yet deterministic parameterizations do assume such corre-
spondence. As a result, stochastic methods for representing the unresolved scales have received
an increasing amount of attention. Early contributions to this topic in the context of ocean mod-
elling includes the work of [1], where the eddy-forcing is replaced by a space-time correlated
random-forcing process. Other notable examples include the work of [9, 20, 7], who construct
probability density functions (pdfs) of the eddy forcing using a reference solution.
In this study, we also consider a stochastic surrogate method [17, 16], and as a performance
indicator we use the degree by which it is able to capture energy and enstrophy statistics. How-
ever, we refrain from an approach that is purely data-driven, i.e. one which attempts to learn the
eddy forcing directly from reference data. Instead, we replace the eddy forcing with a simpler
’model-error’ source term, which we parameterize based on physical arguments. Specifically,
we use the energy and enstrophy transport equations to derive a source term which tracks our
chosen target statistics. The only remaining unclosed part of our model-error term is repre-
sentative of the magnitude of these target statistics, i.e. scalars. As a result, the corresponding
surrogate model needs to represent only one (or a few) scalar quantities rather than the full eddy
forcing field. This amounts to a large dimension reduction (in this study, a reduction by four
orders of magnitude), and as a consequence a large reduction in the amount of required training
data, while retaining accuracy in the statistics.
The article is organised as follows. In Section 2 we describe the governing equations and
multiscale decomposition. The model-error source term derivation and the surrogate method
are outlined in Section 3. Initial results are shown in Section 4, and finally the conclusion and
outlook are given in Section 5.
Wouter Edeling and Daan Crommelin
We study the same model as in [18], i.e. the forced-dissipative vorticity equations for two-
dimensional incompressible flow. The governing equations read
∂t +J, ω) = ν2ω+µ(Fω),
2Ψ = ω. (1)
Here, ωis the vertical component of the vorticity, defined from the curl of the velocity field V
as ω:= e3·∇×V, where e3:= (0,0,1)T. The stream function Ψrelates to the horizontal
velocity components by the well-known relations u=Ψ/∂y and v=Ψ/∂x. As in [18],
the forcing term is chosen as the single Fourier mode F= 23/2cos(5x) cos(5y). The system
is fully periodic in x and y directions over a period of 2πL, where Lis a user-specified length
scale, chosen as the earth’s radius (L= 6.371 ×106[m]). The inverse of the earth’s angular
velocity 1is chosen as a time scale, where Ω = 7.292 ×105[s1]. Thus, a simulation time
period of a single ’day’ can now be expressed as 24 ×602×6.3non-dimensional time
units. Given these choices, (1) is non-dimensionalized, and solved using values of νand µ
chosen such that a Fourier mode at the smallest retained spatial scale is exponentially damped
with an e-folding time scale of 5 and 90 days respectively. For more details on the numerical
setup we refer to [18]. Furthermore, our Python source code for (1) can be downloaded from
Finally, the key term in (1) is the Jacobian, i.e. the nonlinear advection term defined as
J, ω) := Ψ
∂y Ψ
∂x .(2)
It is this term that leads to the need for a closure model when (1) is discretized on a relatively
coarse grid which lacks the resolution to capture all turbulent eddies.
2.1 Discretization
We solve (1) by means of a spectral method, where we apply a truncated Fourier expansion:
ωk(x, y, t) = X
Ψk(x, y, t) = X
The sum is taken over the components k1and k2of the wave number vector k:= (k1, k2)T,
and K0kjK0,j= 1,2. These decompositions are inserted in (1), and solved for the
Fourier coefficients ˆωk,ˆ
Ψkby means of the real Fast Fourier Transform. To avoid the aliasing
problem in the nonlinear term (2), we use the pseudo spectral method, such that in practice the
maximum resolved wave number is K, where K2K0/3[14]. 1
To advance the solution in time we use the second-order accurate AB/BDI2 scheme, which
results in the following discrete system of equations [14]
k+ ˆωi1
2∆t+ 2 ˆ
k= 0.(4)
1We use N×Ngrids, with an even N= 2p(e.g. p= 7), such that N= 2K0[14].
Wouter Edeling and Daan Crommelin
Here, t= 0.01 and ˆ
kis the Fourier coefficient of the Jacobian at time level i, computed with
the pseudo spectral technique, and k2:= k2
2.2 Multiscale decomposition
As in [18], we apply a spectral filter in order to decompose the full reference solution into a
resolved (R) and an unresolved component (U), i.e. we use
where the projection operators PRand PUare depicted in Figure 1. Note that the full projection
operator P:= PR+PUalso removes wave numbers due to the use of the pseudo spectral
0 20 40 60
0 20 40 60
0 20 40 60
Figure 1: The spectral filter (black=1, white=0) of the full, resolved and unresolved solutions. Due to the fact that
we use the real FFT algorithm, only part of the spectrum is computed, as Fourier coefficients with opposite values
of kare complex conjugates in order to enforce real ωand Ψfields [14].
Applying the resolved projection operator to the governing equations (1) results in the fol-
lowing resolved-scale transport equation
∂t +PRJ, ω) = ν2ωR+µFRωR(6)
As mentioned, the key term is the Jacobian (2), since due to its non linearity, PRJ, ω)6=
PRJΨR, ωR. We therefore write
J, ω)JΨR, ωR=: r, (7)
such that ris the exact subgrid-scale term, commonly referred to as the ’eddy forcing’ [1]. The
resolved-scale equation (6) can now be written as
∂t +PRJΨR, ωR=ν2ωR+µFRωRr. (8)
We use the notation r:= PRrfor the sake of brevity. A snapshot of the resolved vorticity
ωRand corresponding resolved eddy forcing ris depicted in Figure 2. Notice the fine-grained
character of the eddy forcing compared to the vorticity field.
Wouter Edeling and Daan Crommelin
Figure 2: A snapshot of the exact, reference vorticity field ωRand the corresponding eddy forcing.
2.3 Prediction of climate statistics
Ultimately, our goal is to integrate (8) in time, such that we can compute the long-term
climate statistics of the energy ERand enstrophy ZRdensities, defined as
ER:= 1
2ψR, ωR,(9)
ZR:= 1
2ωR, ωR.(10)
Here VRis the two-dimensional vector of the resolved velocity components in xand ydirec-
tion. For conciseness, we use the short-hand notation
(α, β) = 1
αβ dxdy, (11)
to denote the integral of the product αβ normalized by the area of the flow domain. The deriva-
tion of the last equality of (9) can be found in Appendix A.
We cannot integrate (8) since it is still unclosed (due to the ωand Ψdependence of (7)), a
problem which we aim to solve by creating a data-driven surrogate of r, denoted by er. For our
present purpose, we define an ’ideal’ surrogate erfor the eddy forcing as one which satisfies the
following set of requirements:
1. Data-driven: In absence of a single ’best’ deterministic parameterization of r, we opt for
a model inferred from a pre-computed database of high-fidelity reference data.
2. Stochastic: In general, the resolved scales are defined as a convolution of the full solution
with some (spatial/spectral) filter. As a result there is no longer just a single unresolved-
scale field that is consistent with the resolved-scale solution. This ambiguity provides us
with the motivation for a stochastic model for the unresolved, small-scale fields.
3. Correlated in space and time: As demonstrated by Figure 2, the reference eddy forcing
shows complex spatial structures. A surrogate of the full eddy forcing would ideally
reflect these as well.
Wouter Edeling and Daan Crommelin
4. Conditional on the resolved variables: The resolved and unresolved scales are in reality
two-way coupled. Hence, the eddy-forcing surrogate should not be independent from the
resolved solution.
5. Pre-computed & cheap: While the reference database can be computationally expensive
to compute, the resulting data-driven surrogate must be cheap.
6. Extrapolates well: To justify the cost of creating the reference database in the first place,
the data-driven model must be able to predict the chosen quantity of interest well, sub-
stantially beyond the (time) domain of the data.
As mentioned, we will measure the performance of a surrogate model by its ability to accu-
rately represent the statistics of (9)-(10). Thus, we do not expect from the resolved-scale model
forced by the surrogate the ability to produce individual flow fields which are in absolute lock-
step with the high-fidelity data, especially considering the stochastic nature of the surrogate.
One possible course of action, explored in e.g. [17, 10], is to directly create a full-field
surrogate er(x, y;t)RN×N, using a database reference snapshots in time of the exact eddy
forcing (7). Here, Nis the number of grid points in one spatial direction, typically 27,28or
higher. Constructing a full-field, dynamic surrogate of a quantity as complex as the eddy forcing
is a challenging task, and storing a potentially large amount of reference snapshots can lead to
high memory requirements [17]. We therefore propose to precede the surrogate construction
step with a procedure that significantly compresses the training data.
3.1 Reduced surrogate
Note that our statistical quantities of interest (9) and (10) are scalars. Instead of creating
a full-field N×Nsurrogate er(x, y;t), we will first replace the exact rin (8) with a simpler
alternative, where the unclosed component is reflective of the size of the statistical quantities
we aim to approximate in the first place. A simple option is to specify
r(x, y;t) = τ(t)ωR(x, y;t),(12)
where τ(t)is an unknown, time-varying scalar. Clearly, this choice is arbitrary, and (12) will
not match the eddy forcing (7). Instead, we think of (12) as an example of a ’model-error term’,
meant to correct the unparameterized (r= 0) model in some sense. In our case, a deviation
from the exact eddy forcing does not pose a problem because of the freedom that integrated
quantities-of-interest give us, such that we only need our ωRand ΨRfields to approximate the
truth in the weak sense of (9) and (10). We can examine the effect of (12) on the evolution equa-
tions of ERand ZR, and subsequently combine physical insight with a data-driven approach
to find the time series of τthat constrains their evolution to the reference values. A reduced
surrogate now only needs to be constructed from this scalar time series, instead of from the
full-field evolution of (7).
The evolution equation of ER(see Appendix A) satisfies
∂t =2νZR2µUR2µER+ψR, r,(13)
where we denote the integral ΨR, F /2as UR. If we insert (12) into (13), the last term on the
right-hand side becomes
ψR, r=τψR, ωR= 2τ ER.(14)
Wouter Edeling and Daan Crommelin
Figure 3: The pdfs of the energy (left) and enstrophy (right), of the reduced (r=τ ωR), reference (rgiven by (7))
and unparameterised (r= 0) solution.
The last equality follows from the definition (9). Thus, the physical insight is that (12) leads
to the additional term 2τER, which either acts to produce or dissipate ERdepending on the
sign of τ. Let us denote the difference between the projected reference energy and ERas
E:= EER, where E:= PRΨ, ω/2. Any quantity without superscript, e.g. Eor
ω, is a reference quantity computed from (1). Now, for the data-driven determination of the τ
time series, we require τto be positive when E > 0, i.e. to increase production when ERis
too low, and to dissipate energy when E < 0. We parameterize τvia an analytic relationship
which reflects this property:
τ:= τmax tanh E
Here, τmax is a user-specified constant, which we set to one for now. During the training period,
we can compute (15) every t, building up a reference time series.
To test the validity of our approach, we run the system (8) for a simulation period of 8 years.
Besides τ, at every twe also sample the energy and enstrophy of the reference, reduced and
unparameterised solution, i.e. using rgiven by (7), (12) and zero respectively2. The energy and
enstrophy probability density functions (pdfs) generated from those samples can be found in
Figure 3. By virtue of (15), the energy pdfs of the reference and the reduced solution practically
overlap. This demonstrates that it is possible to obtain statistically-equivalent energy solutions
using training data reduced by a factor of N2compared to the full-field surrogate case3.
However, we have two quantities of interest, and (12) also has an effect on the ZRequation
(a term 2τZRappears). Since we train τto track PRE, we cannot expect a perfect ZRpdf, and
in fact, Figure 3 shows that the situation does not improve upon the unparameterised model,
which displays a large bias in ZRvalues. Rather than trying to construct a different τwhich is
some compromise between accuracy in ERand ZR, we opt for two separate time series, each
of which acts on either the energy or enstrophy evolution equation alone.
2Note that no surrogate is used yet, we are generating a large set of training data.
3In the example of Figure 3, N2= 1282= 16384.
Wouter Edeling and Daan Crommelin
3.2 Orthogonal patterns
We replace our initial simple choice (12) with
where Ψ0and ω0are patterns of the resolved vorticity and stream function. We choose Ψ0such
that τEΨ0only acts on the ERequation, and produces no additional source term in the enstrophy
equation. The converse must be true for the τZω0term. This will allow us to train τEon E
alone, and τZonly on Z:= ZZR. Since the ERand ZRevolution equations are forced
by ΨR, ∂ωR/∂tand ωR, ∂ ωR/∂trespectively (see (13) and appendix A), this suggests
a Gram-Schmidt type of approach to make Ψ0orthogonal to ωR,·and likewise for ω0and
ΨR,·. Setting:
Ψ0=ψRψR, ωR
(ωR, ωR)ωRand ω0=ωRψR, ωR
(ψR, ψR)ψR,(17)
ωR, τEΨ0= 0 and ψR, τZω0= 0.(18)
The additional source term in the ERequation now becomes
ψR, τEΨ0=τEψR, ψR+τEψR, ωR2
(ωR, ωR)= 2τE"ER2
ZRSR#:= 2τES0(19)
Here, we defined the integrated square stream function as SR:= ψR, ψR/2. Since ER2/ZR
SRhas the dimension of the squared stream function, we introduce the final shorthand notation
S0:= ER2/ZRSRin (19). In a similar vein, (16) produces the following source term in
the ZRequation:
2τZZ0with Z0:= ZRER2
We parameterise τEand τZusing the same procedure as in Section 3.1, only now we need to
incorporate the sign of S0and Z0to correctly activate either the production or dissipation of ER
and ZR, i.e.
τE:= τE,max tanh E
ER·sgn(S0) and τZ:= τZ,max tanh Z
Again, we leave the proper estimation of parameters for a later study, and simply set τE,max =
τZ,max = 1. Furthermore, sgn(X)=1when X1and 1otherwise. Repeating the sim-
ulation of Section 3.1, inserting (16) in (8) yields the results depicted in Figure 4. Now, both
pdfs match the reference well. Only a very small discrepancy in the ERpdf can be observed,
which might fixed by tuning τE,max. The corresponding τE,τZreference time series are shown
in Figure 5.
Wouter Edeling and Daan Crommelin
Figure 4: The pdfs of the energy (left) and enstrophy (right), of the reduced (r=τEΨ0+τZω0), reference (rgiven
by (7)) and unparameterised (r= 0) solution.
Figure 5: Training time series of τEand τZover 500 days. Note that there seems to be a negative correlation
between the two time series.
Wouter Edeling and Daan Crommelin
3.3 Surrogate construction
We will build on the resampling stategies as developed by [16, 2]. In general, these methods
model the unresolved term at time ti+1 by sampling from the conditional probability distribution
of the reference data. In our case, we keep the functional forms of (21), such that Eand Z
can be chosen as the unresolved terms in need of a surrogate model:
Ei+1 Ei+1 | Ei,Ei1,· · ·
Zi+1 Zi+1 | Zi,Zi1,· · · (22)
Here, g
Ei+1 denotes the data-driven resampling surrogate at time ti+1, whereas as Ei+1 rep-
resent actual reference data from the training run, and likewise for g
Zi+1. The set of ’condi-
tioning variables’ Ei,Zietc contain variables from the resolved model. They can be (functions
of) ER,S0or any other (scalar) quantity, as long as we also have access to it outside the training
period. Examples of these conditional distributions are Ei+1 |ER
iand Zi+1 |g
Zi, ZR
We could assume a Markov property (Ei+1 | Ei), or build in a larger memory. Note that by
design, (22) already satisfies many of the properties listed in Section 3, e.g. it is data-driven,
stochastic and conditioned on resolved variables.
The main challenges with this approach are twofold. Clearly, the first challenge concerns
the actual formation of the conditional distribution, i.e. how to map the observed conditioning
variables to plausible subsets of Ei+1 and Zi+1 samples from which g
Ei+1 and g
can be randomly sampled. The second challenge concerns the proper choice of conditioning
variables, which is somewhat reminiscent of the choice of ’features’ in a machine-learning
3.4 Building the distribution
We will illustrate the approach using E, the same procedure applies for Z. To map Eito
some subset of plausible Ei+1 values we use the so-called ’binning’ approach of [16]. First,
consider a snapshot sequence of E
1={E1,E2,· · · ,Ei,· · · ,ES},(23)
where iis the time index. In addition, we also have snapshots of corresponding conditioning
1={E1,E2,· · · ,ES}.(24)
Let Cbe the total number of time-lagged conditioning variables used in (22). We then proceed
by creating C-dimensional disjoint bins4, each bin spanning a unique conditioning variable
range, and containing a number of associated Evalues, see Figure 6. Note that not all bins
may contain samples, especially if two or more conditioning variables are used. If during pre-
diction an empty bin is sampled, the data of the nearest bin (in Euclidean sense) is used instead.
Once a bin is selected by Ei, the resulting subset of Evalues can be sampled randomly, or one
might sample from the local bin average instead, leading to a deterministic prediction.
4We used equidistant bins, but this is not a hard requirement.
Wouter Edeling and Daan Crommelin
(a) Low correlation between Ei+1 and Ei. (b) High correlation between Ei+1 and Ei.
Figure 6: Two binning objects, with the reference Ei+1 data on the vertical axis and the conditioning variable Ei
on the horizontal axis. Vertical lines separate the different bins, and the black dots represent the local bin means.
3.5 Choice of conditioning variables
Ideally we would like the conditioning variables of (22) to display some correlation with
Ei+1 and Zi+1. In this case, the range of plausible reference values in the selected subset is
smaller. Consider the two bins depicted in Figure 6, each with 1 conditioning variable (Ei+1 |
Ei). The binning object of Figure 6(a) shows considerable less correlation between Eiand
Ei+1 than its counterpart in Figure 6(b). As a result, each bin contains a larger spread in
possible Evalues, leading to more noisy g
Ei+1 predictions.
We continue by drawing up a list of candidate conditioning variables, and computing the
temporal correlation coefficients
ρ(∆Ei+1,Ei) = Cov [∆Ei+1,Ei]
σ(∆Ei+1)σ(Ei)and ρ(∆Zi+1,Zi) = Cov [∆Zi+1,Zi]
from a reference time series of 500 days. Here Cov (·,·)is the covariance operator and σ(·)
is the standard deviation. Specifically, we will select individual source terms from the ERand
ZRequations as candidate Eiand Zi, the rationale being that these will also (in part) drive the
evolution equations of Eand Z. The complete list, including the correlation coefficient
values, is shown in Table 1. Previously undefined conditioning variables (occurring in the ZR
equation), are VR:= ωR, F /2and OR:= 2ωR, ωR/2. This strategy for selecting
candidate conditioning variables is reasonable, as many show substantial correlation with the
reference data, hovering around the ±0.5mark. Clear exceptions are ER(which correlates
much less), and τES0,τZZ0, which show very high correlation.
This section contains the initial exploratory results of the methodology outlined in the pre-
ceding sections. For validation and training purposes we ran the reference model (1) for a
simulation period of 8 years, storing reference data and conditioning variables every t. Here,
is amounts to roughly 1.8×106snapshots per variable. When predicting, the training data
must be stored in memory to allow for fast resampling. If the reference snapshots are full field,
this can lead to high memory requirements [17]. Subsampling the reference data reduces the
memory constraints, although this leads to a surrogate with an intrinsic time step that is larger
Wouter Edeling and Daan Crommelin
ZR:ωR, ωR/20.4017 0.336
ER:ψR, ωR/20.1401 0.0951
UR:ψR, F /20.5497 0.598
SR:ψR, ψR/2-0.5091 -0.4857
VR:ωR, F /2-0.5467 -0.5965
OR:2ωR, ωR/2-0.4993 -0.4394
τES0:τEER2/ZRSR0.9484 0.8876
τZZ0:τZZRER2/SR0.8915 0.999
Table 1: Correlation coefficients.
than the tof (4), and thus can only be updated after a certain number of ttime cycles [2]. A
clear advantage of our current surrogate approach, is that we can store the full 8 year reduced
training set in memory, without the need for subsampling.
We subdivide the results into tests of increasing complexity:
T1: A one-way coupled simulation where the resolved equation (8) provides the conditioning
variables, without replacing r=τE(∆E) Ψ0+τZ(∆Z)ω0in (8) with the surrogate
Z)ω0. The surrogates g
Eand g
Zare not extrapolated, i.e. they
are constructed using the full 8 year reference data set, so no simulation outside the time
period of the training data takes place.
T2: A two-way coupled simulation, still without surrogate extrapolation.
T3: A two-way coupled simulation with surrogate extrapolation.
4.1 Results T1
T1 serves as a verification of our code, as in this case the exact Eand Zare still used
in (21) to compute τEand τZ. Now, if implemented correctly, surrogates such as g
Ei+1 |(τES0)iand g
Zi+1 Zi+1 |(τZZ0)i, must follow the reference data closely, given
the high correlations displayed in Table 1. This is confirmed by the results of Figure 7.
4.2 Results T2
T2 is the first real test of the surrogate method due to its two-way coupled nature. As a
result, trajectories of g
Eand g
Zcan no longer be expected to follow the reference data.
Discrepancies between the exact (reduced) eddy forcing (16) and its surrogate will cause the
model forced by the surrogate to develop its own dynamics. We reiterate here that our goal
is to predict the time-averaged flow statistics, which might still be feasible if we are not in
absolute lockstep with Eand Z. Even two full-scale simulations with slightly different
initial conditions will diverge from each other (due to their turbulent nature), yet can converge
in a statistical sense.
We tested a variety of surrogates, which differed through the set of selected conditioning
variables. All were Markovian in character, conditioned on variables from the previous time
step alone. Thus far, almost all considered surrogates improved upon the ZRbias of the un-
parameterized model, although they showed some varying performance amongst each other.
Wouter Edeling and Daan Crommelin
Figure 7: T1 time series for Eand Zand their corresponding surrogates over a 50 day period. The g
surrogate is noisier due to the lower correlation with its conditioning variable (see Table 1).
Figure 8: The pdfs of the energy (left) and enstrophy (right), of the reduced surrogate (er=τEg
Zω0), reference (rgiven by (7)) and unparameterised (r= 0) solution. The surrogates were both condi-
tioned on ZR, ER, U R, SRof the previous time step.
For brevity, we only show a representative sample of results. Consider the results of Fig-
ure 8, which shows the pdfs obtained using the surrogates Ei+1 |ZR
i, ER
i, UR
i, SR
Zi+1 |ZR
i, ER
i, UR
i, SR
i, with 10 bins per conditioning variable. As expected, the pdfs do
not show the same (near) perfect overlap with the reference compared to the training case
of Figure 4, but the match is still accurate. Surrogates conditioned on e.g. ZR, ER, URor
ZR, UR, SRshowed fairly similar results. Somewhat degraded performance (although over-
all still better than r= 0), is obtained when conditioning on ER, U R, SR, see Figure 9. While
the ZRbias is still corrected for, the pdfs of the surrogate underestimate the variance. The only
exception, which did not improve upon the unparameterized model, was when conditioning on
τES0and τZZ0, despite the high correlations of Table 1. A possible cause is that, when predict-
ing, we are forced to condition on τE(g
E)S0instead of τE(∆E)S0, as the latter is not available
outside the training period. Perhaps using conditioning variables such as τES0and τZZ0should
be viewed as some form of overfitting, leading to a surrogate which is unlikely to generalize
well beyond the training set. A possible remedy might be to increase the time lag [17].
Wouter Edeling and Daan Crommelin
Figure 9: The pdfs of the energy (left) and enstrophy (right), of the reduced surrogate (er=τEg
Zω0), reference (rgiven by (7)) and unparameterised (r= 0) solution. The surrogates were both condi-
tioned on ER, U R, SRof the previous time step.
Figure 10: The pdfs of the energy (left) and enstrophy (right), of several extrapolated reduced surrogates (er=
Zω0), reference (rgiven by (7)) and unparameterised (r= 0) solution. The surrogates
were both conditioned on ZR, ER, U R, SRof the previous time step.
4.3 Results T3
Predictive capability outside the training set should be the goal of any data-informed numer-
ical simulation tool. In our case, this goal concerns prediction outside the time interval covered
by the training set. We take tentative steps in this direction by incrementally reducing the time
interval of the training set for the Ei+1 |ZR
i, ER
i, UR
i, SR
iand Zi+1 |ZR
i, ER
i, UR
i, SR
surrogates, while keeping the simulation time Tsim fixed to 8 years. Figure 10 shows the
resulting pdfs, obtained using a training set spanning the first Ttrain =αTsim years, with
α∈ {0.9,0.8,0.7,0.6,0.5}. No significant deviation from the unextrapolated T2 test case is
observed, which demonstrates the predictive capability of the surrogate method.
Finally, we note that all results can replicated via the source code and corresponding input
files, available for download at [3].
Wouter Edeling and Daan Crommelin
We presented a method to create a stochastic surrogate model, conditioned on time-lagged
observable variables, from a set of training data of a multiscale dynamical system. The novelty
of our approach is found in the derivation of model-error source terms designed to track chosen
spatially-integrated statistics of interest. We denote these as ’reduced’ model error terms, as
they lead to a significant reduction in the amount of training data. Although using less data
might seem counter productive, we argue that this leads to an easier surrogate construction.
Furthermore, our reduced framework allows us to step away from a fully-data driven, physics-
blind, surrogate, and inform part of our model-error term based on the transport equations of
the target statistics.
Future work includes further testing the extrapolative capability of the method. Another in-
teresting research option would be to contrast the performance of our conditional time-lagged
surrogate with machine-learning alternatives, such as random forests or neural nets. Recent
relevant work also considered a combination of both approaches[13]. Finally, a further inter-
esting avenue of future research is the a-priori incorporation of constraints from mathematical
physics. For instance, when rewriting the eddy forcing in tensor format, certain constraints on
the tensor shape can be found [19]. Such an approach opens up the possibility for efficient,
physics-constrained uncertainty quantification, see e.g. [5] for examples in steady flow prob-
lems or [8] for large-eddy simulations.
This research is funded by the Netherlands Organization for Scientific Research (NWO)
through the Vidi project ”Stochastic models for unresolved scales in geophysical flows”, and
from the European Union Horizon 2020 research and innovation programme under grant agree-
ment #800925 (VECMA project).
We also thank W.T.M. Verkley for making his vorticity equation source code available to us.
[1] P.S. Berloff. Random-forcing model of the mesoscale oceanic eddies. Journal of Fluid
Mechanics, 529:71–95, 2005.
[2] D. Crommelin and E. Vanden-Eijnden. Subgrid-scale parameterization with conditional
markov chains. Journal of the Atmospheric Sciences, 65(8):2661–2675, 2008.
[3] W.N. Edeling. Tau ez - uncecomp branch (github repository).
wedeling/TAU_EZ/tree/uncecomp, 2019.
[4] W.N. Edeling. vorticity-solver (github repository).
wedeling/vorticity-solver, 2019.
[5] W.N. Edeling, G. Iaccarino, and P. Cinnella. Data-free and data-driven rans predictions
with quantified uncertainty. Flow, Turbulence and Combustion, 100(3):593–616, 2018.
[6] P.R. Gent and J.C. Mcwilliams. Isopycnal mixing in ocean circulation models. Journal of
Physical Oceanography, 20(1):150–155, 1990.
[7] I. Grooms and L. Zanna. A note on ’toward a stochastic parameterization of ocean
mesoscale eddies’. Ocean Modelling, 113:30–33, 2017.
Wouter Edeling and Daan Crommelin
[8] L. Jofre, S.P. Domino, and G. Iaccarino. A framework for characterizing structural uncer-
tainty in large-eddy simulation closures. Flow, Turbulence and Combustion, 100(2):341–
363, 2018.
[9] P. Mana and L. Zanna. Toward a stochastic parameterization of ocean mesoscale eddies.
Ocean Modelling, 79:1–20, 2014.
[10] R. Maulik, O. San, A. Rasheed, and P. Vedula. Subgrid modelling for two-dimensional
turbulence using neural networks. Journal of Fluid Mechanics, 858:122–144, 2019.
[11] J.C. McWilliams. The emergence of isolated coherent vortices in turbulent flow. Journal
of Fluid Mechanics, 146:21–43, 1984.
[12] T. Palmer and P. Williams. Stochastic physics and climate modelling. Cambridge Univer-
sity Press Cambridge, UK, 2010.
[13] S. Pan and K. Duraisamy. Data-driven discovery of closure models. SIAM Journal on
Applied Dynamical Systems, 17(4):2381–2413, 2018.
[14] R. Peyret. Spectral methods for incompressible viscous flow, volume 148. Springer Sci-
ence & Business Media, 2013.
[15] J. Thuburn, J. Kent, and N. Wood. Cascades, backscatter and conservation in numerical
models of two-dimensional turbulence. Quarterly Journal of the Royal Meteorological
Society, 679(140):626–638, 2014.
[16] N. Verheul and D. Crommelin. Data-driven stochastic representations of unresolved fea-
tures in multiscale models. Commun. Math. Sci, 14(5):1213–1236, 2016.
[17] N. Verheul, J. Viebahn, and D. Crommelin. Covariate-based stochastic parameterization of
baroclinic ocean eddies. Mathematics of Climate and Weather Forecasting, 3(1):90–117,
[18] W.T.M. Verkley, P.C. Kalverla, and C.A. Severijns. A maximum entropy approach to the
parametrization of subgrid processes in two-dimensional flow. Quarterly Journal of the
Royal Meteorological Society, 142(699):2273–2283, 2016.
[19] S. Waterman and J.M. Lilly. Geometric decomposition of eddy feedbacks in barotropic
systems. Journal of Physical Oceanography, 45(4):1009–1024, 2015.
[20] L. Zanna, P. Mana, J. Anstey, T. David, and T. Bolton. Scale-aware deterministic and
stochastic parametrizations of eddy-mean flow interaction. Ocean Modelling, 111:66–80,
For convenience, we reproduce certain relevant derivations regarding the ERand ZRtrans-
port equations from [18]. The energy (density) is defined as
ER:= 1
VR·VRdxdy, (26)
Wouter Edeling and Daan Crommelin
where VRis the vector containing the velocity components in x and y direction. It can be
rewritten as ER=ψR, ωR/2via
VR·VR=ψR· ∇ψR=∇ · ψRψRψR2ψR=∇ · ψRψRψRωR(27)
The first equality follows from the definition VR:= ∂ψR/∂y, ∂ψR/∂xT, while the second
stems from the product rule of a scalar (ψR) and a vector (ψR):
∇ · ψRψR=ψR· ∇ψR+ψR2ψR.(28)
Finally, the last equality of (27) simply follows from the governing equations (1). The term
∇ · ψRψRdisappears when integrated over the spatial domain, after application of the
divergence theorem in combination with the doubly periodic boundary conditions. This leaves
ER=ψR, ωR/2. To obtain the energy equation, start with
∂t 1
∂t dxdy.
Similar to the analysis above, we use the relation VR·VR
t=∇ · ψRψR
the subscript tdenotes ∂/∂t) to obtain
∂t =
ψR, P RJψR, ωRνψR,2ωRµψR, F ωR+ψR, r(30)
Using integration by parts and the periodic boundary conditions it can be shown that the first
term on the right-hand side satisfies ψR, P RJψR, ωR=JψR, ψR, ωR= 0, since the
Jacobian of two equal arguments is zero [18]. Furthermore, using the self-adjoint nature of the
Laplace operator, we have ψR,2ωR=2ψR, ωR=ωR, ωR. This leads to
dt=νωR, ωRµψR, F +µψR, ωR+ψR, r,(31)
which equals (13). Using the same procedure, the evolution equation for the enstrophy reads
∂t =νωR,2ωR+µωR, F µωR, ωRωR, r.(32)
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
Derivation of reduced order representations of dynamical systems requires the modeling of the truncated dynamics on the retained dynamics. In its most general form, this so-called closure model has to account for memory effects. In this work, we present a framework of operator inference to extract the governing dynamics of closure from data in a compact, non-Markovian form. We employ sparse polynomial regression and artificial neural networks to extract the underlying operator. For a special class of non-linear systems, observability of the closure in terms of the resolved dynamics is analyzed and theoretical results are presented on the compactness of the memory. The proposed framework is evaluated on examples consisting of linear to nonlinear systems with and without chaotic dynamics, with an emphasis on predictive performance on unseen data.
Full-text available
In this study we investigate a covariate-based stochastic approach to parameterize unresolved turbulent processes within a standard model of the idealised, wind-driven ocean circulation. We focus on vertical instead of horizontal coarse-graining, such that we avoid the subtle difficulties of horizontal coarsegraining. The corresponding eddy forcing is uniquely defined and has a clear physical interpretation related to baroclinic instability.We propose to emulate the baroclinic eddy forcing by sampling from the conditional probability distribution functions of the eddy forcing obtained from the baroclinic reference model data. These conditional probability distribution functions are approximated here by sampling uniformly from discrete reference values. We analyze in detail the different performances of the stochastic parameterization dependent on whether the eddy forcing is conditioned on a suitable flow-dependent covariate or on a timelagged covariate or on both. The results demonstrate that our non-Gaussian, non-linear methodology is able to accurately reproduce the first four statistical moments and spatial/temporal correlations of the stream function, energetics, and enstrophy of the reference baroclinic model.
Full-text available
Motivated by the sizable increase of available computing resources, large-eddy simulation of complex turbulent flow is becoming increasingly popular. The underlying filtering operation of this approach enables to represent only large-scale motions. However, the small-scale fluctuations and their effects on the resolved flow field require additional modeling. As a consequence, the assumptions made in the closure formulations become potential sources of incertitude that can impact the quantities of interest. The objective of this work is to introduce a framework for the systematic estimation of structural uncertainty in large-eddy simulation closures. In particular, the methodology proposed is independent of the initial model form, computationally efficient, and suitable to general flow solvers. The approach is based on introducing controlled perturbations to the turbulent stress tensor in terms of magnitude, shape and orientation, such that propagation of their effects can be assessed. The framework is rigorously described, and physically plausible bounds for the perturbations are proposed. As a means to test its performance, a comprehensive set of numerical experiments are reported for which physical interpretation of the deviations in the quantities of interest are discussed.
Full-text available
For the purpose of estimating the epistemic model-form uncertainty in Reynolds-Averaged Navier-Stokes closures, we propose two transport equations to locally perturb the Reynolds stress tensor of a given baseline eddy-viscosity model. The spatial structure of the perturbations is determined by the proposed transport equations, and thus does not have to be inferred from full-field reference data. Depending on a small number of model parameters and the local flow conditions, a ’return to eddy viscosity’ is described, and the underlying baseline state can be recovered. In order to make predictions with quantified uncertainty, we identify two separate methods, i.e. a data-free and data-driven approach. In the former no reference data is required and computationally inexpensive intervals are computed. When reference data is available, Bayesian inference can be applied to obtained informed distributions of the model parameters and simulation output.
Full-text available
Porta Mana & Zanna (2014) recently proposed a subgrid-scale parameterization for eddy-permitting quasigeostrophic models. In this model the large-scale fluid is represented as a non-Newtonian viscoelastic medium, with a subgrid-stress closure that involves the Lagrangian derivative of large-scale quantities. This note derives this parameterization, including the nondimensional proportionality coefficient, using only two statistical assumptions: that the subgrid-scale term is locally homogeneous and decorrelates rapidly in space. The parameterization is then verified by comparing against eddy-resolving quasigeostrophic simulations, independently reproducing the results of Porta Mana and Zanna in a simpler model.
Full-text available
The role of mesoscale eddies is crucial for the ocean circulation and its energy budget. The sub-grid scale eddy variability needs to be parametrized in ocean models, even at so-called eddy permitting resolutions. Porta Mana and Zanna (2014) propose an eddy parametrization based on a non-Newtonian stress which depends on the partially resolved scales and their variability. In the present study, we test two versions of the parametrization, one deterministic and one stochastic, at coarse and eddy-permitting resolutions in a double gyre quasi-geostrophic model. The parametrization leads to drastic improvements in the mean state and variability of the ocean state, namely in the jet rectification and the kinetic-energy spectra as a function of wavenumber and frequency for eddy permitting models. The parametrization also appears to have a stabilizing effect on the model, especially the stochastic version. The parametrization possesses attractive features for implementation in global models: very little computational cost, it is flow aware and uses the properties of the underlying flow. The deterministic coefficient is scale-aware, while the stochastic parameter is scale- and flow-aware with dependence on resolution, stratification and wind forcing.
Full-text available
A subgrid-scale form for mesoscale eddy mixing on isopycnal surfaces is proposed for use in non-eddy-resolving ocean circulation models. The mixing is applied in isopycnal coordinates to isopycnal layer thickness, or inverse density gradient, as well as to passive scalars, temperature and salinity. The transformation of these mixing forms to physical coordinates is also presented.
Full-text available
In this study, we investigate how to use sample data, generated by a fully resolved multiscale model, to construct stochastic representations of unresolved scales in reduced models. We explore three methods to model these stochastic representations. They employ empirical distributions, conditional Markov chains, and conditioned Ornstein-Uhlenbeck processes, respectively. The Kac-Zwanzig heat bath model is used as a prototype model to illustrate the methods. We demonstrate that all tested strategies reproduce the dynamics of the resolved model variables accurately. Furthermore, we show that the computational cost of the reduced model is several orders of magnitude lower than that of the fully resolved model.
In this investigation, a data-driven turbulence closure framework is introduced and deployed for the subgrid modelling of Kraichnan turbulence. The novelty of the proposed method lies in the fact that snapshots from high-fidelity numerical data are used to inform artificial neural networks for predicting the turbulence source term through localized grid-resolved information. In particular, our proposed methodology successfully establishes a map between inputs given by stencils of the vorticity and the streamfunction along with information from two well-known eddy-viscosity kernels. Through this we predict the subgrid vorticity forcing in a temporally and spatially dynamic fashion. Our study is both a priori and a posteriori in nature. In the former, we present an extensive hyper-parameter optimization analysis in addition to learning quantification through probability-density-function-based validation of subgrid predictions. In the latter, we analyse the performance of our framework for flow evolution in a classical decaying two-dimensional turbulence test case in the presence of errors related to temporal and spatial discretization. Statistical assessments in the form of angle-averaged kinetic energy spectra demonstrate the promise of the proposed methodology for subgrid quantity inference. In addition, it is also observed that some measure of a posteriori error must be considered during optimal model selection for greater accuracy. The results in this article thus represent a promising development in the formalization of a framework for generation of heuristic-free turbulence closures from data.
In numerical models of geophysical fluid systems, parametrization schemes are needed to account for the effect of unresolved processes on processes that are resolved explicitly. Usually, these parametrization schemes require tuning of their parameters to achieve optimal performance. We propose a new type of parametrization that requires no tuning, as it is based on an assumption that is not specific to any particular model. The assumption is that the unresolved processes can be represented by a probability density function that has maximum information entropy under the constraints of zero average time derivatives of key integral quantities of the unresolved processes. In the context of a model of a simple fluid dynamical system, it is shown that this approach leads to definite expressions for the mean effect of unresolved processes on processes that are resolved. The merits of the parametrization, regarding both short-range forecasting and long-term statistics, are demonstrated by numerical experiments in which a low-resolution version of the model is used to simulate the results of a high-resolution version of the model. For the fluid dynamical system that is studied, the proposed parametrization turns out to be related to the anticipated potential vorticity method with definite values of its parameters.