ArticlePDF Available

Comparison of high-order numerical methodologies for the simulation of the supersonic Taylor–Green vortex flow

AIP Publishing
Physics of Fluids
Authors:

Abstract and Figures

This work presents a comparison of several high-order numerical methodologies for simulating shock/turbulence interactions based on the supersonic Taylor–Green vortex flow, considering a Reynolds number of 1600 and a Mach number of 1.25. The numerical schemes considered include high-order Finite Difference, Targeted Essentially Non-Oscillatory, Discontinuous Galerkin, and Spectral Difference schemes. The shock capturing methods include high-order filtering, localized artificial diffusivity, non-oscillatory numerical fluxes, and local low-order switching. The ability of the various high-order numerical methodologies to both capture shocks and represent accurately the development of turbulent vortices is assessed.
This content is subject to copyright. Terms and conditions apply.
Comparison of high-order numerical methodologies for the simulation of the
supersonic Taylor-Green Vortex flow
Jean-Baptiste Chapelier,
1
David J. Lusher,
2
William Van Noordt,
3
Christoph Wenzel,
4
Tobias Gibis,
4
Pascal Mossier,
4
Andrea Beck,
4
Guido Lodato,
5
Christoph Brehm,
6
Matteo
Ruggeri,7Carlo Scalo,7and Neil Sandham8
1)
ONERA - Department of Aerodynamics, Aeroelasticity and Acoustics - 92322 Chˆatillon,
France; jean-baptiste.chapelier@onera.fr - corresponding author
2)
University of Southampton - Aerodynamics and Flight Mechanics Group - Southampton
SO17 1BJ, United Kingdom; d.lusher@soton.ac.uk
3)
University of Oxford - Department of Engineering Science - Wellington Square,
Oxford OX1 2JD, United Kingdom; william.vannoordt@eng.ox.ac.uk
4)
University of Stuttgart - Institute of Aerodynamics and Gas Dynamics - 70569 Stuttgart,
Germany; {wenzel,gibis,pascal.mossier,andrea.beck}@iag.uni-stuttgart.de
5)
INSA Rouen Normandie - CORIA-CNRS - 76800 Saint-Etienne-du-Rouvray,
France; guido.lodato@insa-rouen.fr
6)
University of Maryland at College Park - Aerospace Engineering,
Alfred Gessow Rotorcraft Center - College Park, Maryland 20742,
USA; cbrehm1@umd.edu
7)
Purdue University - School of Mechanical Engineering - West Lafayette, Indiana 47906,
USA; {mruggeri,scalo}@purdue.edu
8)
University of Southampton - Aerodynamics and Flight Mechanics Group - Southampton
SO17 1BJ, United Kingdom; n.sandham@soton.ac.uk
1
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
Abstract
This work presents a comparison of several high-order numerical methodologies
for simulating shock/turbulence interactions based on the supersonic Taylor-Green
vortex flow, considering a Reynolds number of 1600 and a Mach number of 1.25. The
numerical schemes considered include high-order Finite Differences, Targeted Es-
sentially Non-Oscillatory, Discontinuous Galerkin and Spectral Difference schemes.
The shock capturing methods include high-order filtering, localized artificial diffu-
sivity, non-oscillatory numerical fluxes and local low-order switching. The ability
of the various high-order numerical methodologies to both capture shocks and
represent accurately the development of turbulent vortices is assessed.
Keywords: Supersonic Taylor-Green Vortex; Turbulence; High-order; Targeted
Essentially Non-Oscillatory; Discontinuous Galerkin; Spectral Difference; Shock
capturing
2
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
I. Introduction
The accurate representation of turbulent scales using high-fidelity modeling approaches
such as Direct Numerical Simulation or Large-Eddy Simulation is particularly challenging
in engineering practice due to the wide range of scales considered and their interaction
with numerical errors stemming from the discretization method. For such problems, it is
customary to employ high-order numerical methods that provide low levels of numerical
dissipation and dispersion, enabling an accurate representation of the small scales of
turbulence. However, high-order numerical schemes are often less robust than their lower
order counterparts, posing problems when considering stiff problems such as supersonic
flows and shock waves. A number of industrial applications involve shock/turbulence
interactions, such as cruising aircraft configurations or engines. In this context, it is of
the utmost importance to establish numerical schemes that are robust regarding shock
capturing and retain a high accuracy for the representation of the turbulent scales.
In the present paper, several of those methodologies are compared based on a canonical
shock/turbulence interaction flow problem, namely the supersonic Taylor-Green vortex.
The incompressible - or low Mach - Taylor-Green vortex flow problem
1
has become a
widely used test case for assessing numerical flow solvers, and in particular their ability
to characterize precisely the development of turbulence scales
2–5
. More recently, a su-
personic version of this flow problem has been introduced
6
, which features turbulence
scales progressively breaking down and their interactions with shocks, allowing as as-
sessment of the ability of numerical methods to capture shocks, whilst still accurately
representing the turbulence cascade. This case is of great interest for comparing numerical
methods, as its initial condition is analytical and can be exactly reproduced in different
flow solvers. The supersonic Taylor-Green vortex case has been introduced recently and,
to the authors’ knowledge, this is the first time a cross-comparison involving several
numerical methodologies is performed for this test case. In terms of physical phenomena,
it features transition, fully developed turbulence, strong shocks and shock-turbulence
interaction. The closest configuration in terms of flow physics would be compressible
isotropic turbulence. This flow features small-amplitude shocks (or shocklets) interacting
with turbulence and might resemble the later stages of evolution of the supersonic Taylor
Green vortex case used in the present work7.
3
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
In this work, established Finite Difference schemes and modern variants for shock
and turbulence capturing are considered, as well as more recent Discontinuous Finite
Element methods for which the treatment of such problems is still an open topic of in-
vestigation. Among the numerical methods considered, we consider Targeted Essentially
Non-Oscillatory (TENO)
8
, Discontinuous Galerkin (DG)
9
, Spectral Difference (SD)
10,11
,
Flux Reconstruction (FR)
12
, and compact Finite Difference (FD)
13,14
schemes. Without any
specific treatment of the discontinuities, high-order solvers tend to produce oscillations
around shocks, which eventually lead to non-physical states, such as negative values of
density or pressure. In this study, shock capturing is handled either by non-oscillatory nu-
merical fluxes, high-order filters, Localized Artificial Diffusivity (LAD) or local switching
to a lower order, more robust scheme (i.e. an embedded Finite Volume TVD scheme). A
mesh convergence study is conducted and the ability of each method to capture large and
small turbulent scales, as well as shocks depending on the resolution, is assessed from
select quantities of interest.
The paper is organized as follows. First, the governing equations and description of the
Taylor-Green Vortex flow case are described in section II, as well as the reference solutions
and quantities of interest. Section III presents the various numerical methodologies
considered in the present work. Section IV concerns the description and analysis of the
results. Finally, conclusions are drawn in Section V.
II. Governing equations and flow problem specification
A. Compressible Navier-Stokes equations
The compressible Navier-Stokes equations governing compressible fluid flow motion
are considered:
∂ρ
t+ · (ρu) = 0 (1)
∂ρu
t+ · (ρuu+pIτ) = 0 (2)
∂ρE
t+ · (u(ρE+p)τ·u+q)=0 (3)
4
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
where
ρ
is the density,
E
the total energy,
u
is the velocity vector,
q=λT
is the heat
flux and
τ=µ(
T
)u+uT2
3( · u)I
is the viscous stress tensor. The dynamic
viscosity is expressed as a function of temperature following Sutherland’s law:
µ(T) = 1.4042(T/Tref )1.5
T/Tref +0.4042 µref (4)
The thermal conductivity is defined as
λ=µCp
Pr
, with
Pr
the Prandtl number set to the
value 0.71. The pressure is a function of the conservative variables following the ideal gas
equation of state:
p= (γ1)ρE1
2u·u(5)
where γis the specific heat ratio set to the value 1.4 suitable for air.
B. Taylor-Green Vortex flow setup
The Taylor-Green vortex problem features the analytical initialization of large vortices
in a cubic, periodic computational domain and the subsequent transition and breakdown
of the initial vortices towards a turbulent state. The computational domain is a cubic box
= [πL
,
πL]3
with periodic boundary conditions. In terms of primitive variables, the
initialization features a constant temperature field and reads:
u(x,t=0) =
U0sin x
Lcos y
Lcos z
L
U0cos x
Lsin y
Lcos z
L
0
(6)
p(x,t=0) = p0+ρ0U2
0
16 cos 2x
L+cos 2y
L2+cos 2z
L (7)
T(x,t=0) = T0(8)
T0
is set to the reference temperature
Tref
of the Sutherland law. The Mach and Reynolds
numbers are defined as
M0=U0qρ0
γp0
and
Re =ρ0U0
µ0
, respectively. This particular
initialization features an isothermal flow field, with variations of the density and pressure
in the domain. Figures 1 illustrates the vortices and density gradients relative to this flow
5
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
FIG. 1. TGV flow at
Re =
1600 and
M0=
1.25:
Q
-criterion iso-surfaces colored by the density
gradient magnitude, extracted from a 256
3
degrees of freedom simulations with 4th order of
accuracy using the CODA DG solver. Left: initial condition; Right: field at t=10.
problem at the initial time and the time
t=
10 for which the turbulent structures have
developed in the computational domain.
C. Flow diagnostics and reference solution
As flow diagnostics, we consider the time evolution of three quantities of interest,
spatially integrated over the computational domain
. For each quantity described,
we also present a mesh convergence study of the
M0=
1.25/
Re =
1600 case obtained
with the OpenSBLI
15
solver using a 6th order TENO scheme for consistency with the
original work on this benchmark problem
6
. TENO schemes
8
are a fairly recent (2016)
addition to the family of Essentially Non-Oscillatory (ENO) shock-capturing schemes that
have been in use for several decades in high-speed CFD research. They improve upon
previous ENO and Weighted Essentially Non-Oscillatory (WENO) schemes by having
considerably lower numerical dissipation, while still retaining robust shock-capturing
capability
6
. These characteristics make them ideal for simulating compressible turbulence
with shockwaves and, therefore, a suitable choice for providing the benchmark solutions
in this work. TENO schemes achieve this behavior via a modified ENO-like staggered
6
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
stencil layout, and a strong scale-separation procedure for controllable low dissipation.
TENO schemes have already been applied to a wide range of complex fluid flow problems.
A comprehensive review on the different variants and their applications was given in16.
The extremely fine meshes used for the reference solutions in this study required
substantial computational resources on distributed-memory supercomputing clusters.
For the finest mesh solution of 2048
3
DoFs provided by the OpenSBLI solver with a
6th-order TENO scheme, a total of 57,600 ARM-based Fujitsu A64FX CPU cores (1200
nodes, 48 CPU cores/node) were utilised. In addition to the compute requirements, the
reference solutions require large amounts of disk storage and memory. With 2048
3
DoFs,
each three-dimensional snapshot of the flow-field requires 520GB of storage in double
precision.
While OpenSBLI is more frequently used on GPU-based machines, the flexibility of the
OPS parallel library allowed us to explore a hybrid MPI+OpenMP CPU-based approach
for this study instead. In total, 4800 MPI ranks were used at
N=
2048
3
(4 ranks per node,
1 distributed to each Core Memory Group (CMG)), with 12 OpenMP threads on each rank.
For a non-dimensional time-step of
t=
2.5
×
10
4
, 80,000 time-steps were required for
a non-dimensional integration period of
t=
20. For an average iteration time of around
1.53 seconds, the total runtime was 34 hours. On the requested resources this equates to
40,800 node hours, or, equivalently, 1.96 million core hours.
The first quantity is the kinetic energy, which is representative of the large scale motion
in the flow:
Ek=1
2ρ0U2
0||Zρu·ud(9)
In terms of engineering practice, this quantity is of importance as it characterizes the
energy carried by the turbulent motion, and is therefore the target quantity of interest to
be captured accurately in turbulence simulations. It is carried mainly by the large vortices
in the flow. At sufficiently high Reynolds numbers, the kinetic energy is conserved in the
early stages of the large scale evolution, and starts being dissipated when the turbulence
cascade generates smaller scales, that are impacted by molecular viscous effects. As a
result, relatively coarse grids are able to capture this quantity, as it is carried by the large
scales in the flow that, depending on the accuracy of the scheme considered, require few
points per vortex to be precisely described numerically. Figure 2 displays the kinetic
7
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
0 5 10 15 20
t
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Ek
643
1283
2563
5123
10243
20483
FIG. 2. Mesh convergence study for the TGV flow at
Re =
1600 and
M0=
1.25: Time evolution of
kinetic energy using OpenSBLI/6th order TENO scheme.
energy evolution for a 6th order TENO scheme and resolutions ranging from 64
3
to 2048
3
spatial degrees of freedom (DoFs). We clearly see a fast mesh convergence of this quantity,
as the 128
3
resolution is sufficient to capture accurately the kinetic energy carried by large
scales in the flow. Thus, this quantity will be of interest to assess the resolution needed for
a given numerical scheme to capture accurately large-scale vortices. The decaying part
is also relevant to evaluate how the energy transfers from large-scales into smaller-scale
turbulence are handled by the scheme.
The second quantity of interest is the solenoidal part of the kinetic energy dissipation,
which is directly related to the vortical motion in the flow:
ϵs=L2
ReU2
0||Z
µ(T)
µ0
ω·ωd(10)
where
ω
is the vorticity vector. This quantity is sensitive to the development of small scales
in the flow, which carry a significant vortical intensity. The magnitude of this quantity
increases significantly as the large scales break into smaller structures, and therefore
provides a good diagnostic to assess the ability of a given numerical scheme to represent
8
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
0 5 10 15 20
t
0.000
0.002
0.004
0.006
0.008
0.010
0.012
ǫs
643
1283
2563
5123
10243
20483
FIG. 3. Mesh convergence study for the TGV flow at
Re =
1600 and
M0=
1.25: Time evolution of
solenoidal dissipation using OpenSBLI/6th order TENO scheme.
accurately the breakdown mechanism and small-scale structures dynamics. Compared to
the kinetic energy, the solenoidal dissipation is more challenging to capture numerically,
and finer resolutions/more accurate schemes are required to represent accurately this
quantity. Figure 3 displays the mesh convergence for this quantity, showing here that with
the 6th order TENO scheme, a 512
3
resolution is required for converging towards the exact
values and capture all turbulent scales. In this study, the amplitude of enstrophy will
be directly correlated to the ability of a given numerical scheme to represent accurately
small-scale turbulence.
The third quantity is the dilatational component of the kinetic energy dissipation, and
is related to compressibility effects:
ϵd=4L2
3ReU2
0||Z
µ(T)
µ0
( · u)2d(11)
This quantity is strongly impacted by the onset of shocks in the flow, which are char-
acterized by peaks in the dilatational dissipation evolution. Figure 4 displays a mesh
convergence of the dilatational dissipation time evolution using the 6th order TENO
9
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
0 5 10 15 20
t
0.0000
0.0002
0.0004
0.0006
0.0008
0.0010
0.0012
ǫd
643
1283
2563
5123
10243
20483
FIG. 4. Mesh convergence study for the TGV flow at
Re =
1600 and
M0=
1.25: Time evolution of
dilatational dissipation using OpenSBLI/6th order TENO scheme.
scheme. First, two peaks are clearly identified, the first being linked to shocks develop-
ing with little interaction with the vortices, while the second corresponds to the onset of
shock/turbulence interaction. It is interesting to see that mesh convergence is not achieved
on the first peak, as the shocks are becoming sharper corresponding to an increasing mag-
nitude of the divergence of velocity. Ultimately, a mesh convergence of velocity gradients
should be reached when the molecular viscosity operator is able to smooth the shocks,
which is not yet observed even with a 2048
3
resolution with 8 billion DoFs. The second
peak seems to be mesh converging faster, possibly due to the presence of multiple shock
systems, being less intense than the shocks present in the earlier flow development steps.
The dilatational dissipation is therefore a meaningful quantity as its amplitude allows for
assessing the sharpness of shocks developing in the flow, and oscillations in this quantity
are also a good diagnostic to detect the presence of Gibbs phenomena around the shocks,
and possible flaws in the numerical strategies for shock capturing.
10
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
III. Numerical methodologies
A. Summary of numerical methodologies
This study involves different type of numerical discretization and shock capturing
approaches for the simulations of the supersonic turbulent flows. Mainly two classes of
solvers are considered, high-order finite difference solvers and high-order Discontinuous
Finite Element solvers. In the following, a brief description of the two families is provided:
High-Order Finite Difference approaches: FD techniques are particularly suited
for turbulence simulation, as very high-orders of accuracy can be achieved on
structured grids. However, shock capturing coupled with such schemes can prove
difficult, as those schemes can become unstable in the presence of discontinuities.
Several stabilizing techniques can be employed. The NS3D
17
solver utilizes the
high resolution of compact finite differences combined with high-order filtering,
for this study it was chosen to refrain from additional targeted shock-capturing
approaches to stabilize the simulations. SPADE
18
uses a kinetic energy/entropy
preserving formulation coupled with a local switching to WENO flux reconstruction
near shocks to provide stable and accurate simulations. OpenSBLI
15
uses high-order
TENO schemes with the shock capturing embedded in the definition of numerical
fluxes while providing an accurate representation of turbulence.
Discontinuous Finite element approaches: the solvers SD3DvisP
19
(abbreviated
SD3D), CODA
20
, FLEXI
21
and H3AMR
22
solve the Navier-Stokes equations using
piecewise polynomial approximations of arbitrary order of accuracy per mesh el-
ement. The polynomial bases are local, meaning the continuity of the solution is
not enforced at element interfaces. The elements are connected via numerical fluxes
defined for the computation of interface integrals. Typically, the element sizes em-
ployed for DG or SD are larger compared to FV or FD methods due to the subcell
variations of the numerical solution. This is problematic for shock capturing as a
shock located inside a cell is represented by a polynomial approximation which is
likely to display strong oscillations. The oscillations are controlled via a localized dif-
fusion operator for the DG (CODA) and SD (SD3D) codes or via an embedded Finite
11
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
Volume solver with a limiter (FLEXI). DG and SD methods provide accurate shock
sensors, based on the amplitude of the highest polynomial modes of a given quantity
(typically the density or pressure), which provide an estimation of the smoothness
of the solution. SD3D features a Spectral Difference scheme which belongs to the
category of nodal DFEM, for which the DoFs are the conservative variables values
at given quadrature points inside the mesh elements and the Lagrange polynomial
basis is considered. CODA features a modal DG scheme for which the solution is
expressed as a superposition of spatial modes, the DoFs being the magnitude of
each mode (i.e. the weighting factors of the constant, linear, quadratic or higher
degree modes) and the basis functions being orthonormal and hierarchical poly-
nomials. FLEXI features a nodal DG scheme on a tensor product formulation of
Lagrange basis functions and a collocation of interpolation and integration nodes on
Legendre-Gauss-family sets.
All shock capturing approaches considered in this study, excluding the one of NS3D,
rely on a specific numerical treatment around shocks which are detected using discon-
tinuity or smoothness sensors, therefore it is expected that the orders of accuracy of the
schemes are preserved away from discontinuities. As regards NS3D, the simulations are
stabilized using a filter which order is greater than the order of accuracy of the scheme,
therefore it is expected as well to recover the nominal order of accuracy away from shocks.
As this study focuses on spatial accuracy, robustness and shock capturing assessment,
the time advance schemes considered for all flow solvers rely on classical third or fourth
order Runge-Kutta methods, which is the standard practice for the simulation of unsteady
compressible flows7.
In order to verify the correct implementation of the numerical set-up for each solver,
computations of the Taylor-Green vortex at low Reynolds and subsonic conditions have
been carried out at fine resolution and compared to ensure that all solvers provide similar
solutions in the absence of shocks, see Appendix A. The following sections describe in
more detail the flow solvers considered in the present study.
12
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
B. Flow solvers description
a. Discontinuous Galerkin Solver - CODA
CODA
20
is the CFD software jointly owned and developed by ONERA, DLR and
Airbus with the purpose of applied research and aerodynamic design in the aeronautic
industry. CODA features several numerical schemes for solving Navier-Stokes and RANS
equations, including Finite Volumes and high-order Discontinuous Galerkin schemes
tailored for complex geometries and mixed-element unstructured grids (featuring hexahe-
dra, tetrahedra, pyramids or prisms). In this work, we are interested in the DG scheme
in CODA for which the numerical solution is expressed as a polynomial expansion of
degree
p
in each of the mesh elements. The polynomial basis is constructed to verify the
properties of hierarchy and orthonormality for any mesh elements
23
. The size of the basis
is determined using Pascal triangle products, yielding
(p+
1
)(p+
2
)(p+
3
)/
6 basis func-
tions per element in three spatial dimensions. Additional details about the formulation
can be found in
24
. The numerical flux chosen for the integration of the convective face
fluxes is the Roe flux with an entropy fix, the entropy fraction is set to a standard value 0.1.
The viscous fluxes are discretized using the BR1 approach by Bassi and Rebay
25
, while the
time integration is performed using a third-order explicit Runge-Kutta scheme. As regards
the BR1 approach, it is chosen over the BR2 approach
26
as it avoids tuning the penalty
parameter for the BR2 gradient reconstruction on faces. The volume and face integrals
appearing in the variational formulation of the DG discretization are computed using
tensor-product Gauss-Legendre quadrature rules with
(p+
1
)3
and
(p+
1
)2
points, respec-
tively. The shock capturing technique is based on the Persson and Peraire sensor
27
with the
formulation and calibration of Glaubitz et al.
28
. The idea is to identify troubled cells where
shocks are present based on the amplitude of highest density polynomial modes, build an
artificial viscosity scaled by the maximum wave-speed in the computational domain and
apply a localized artificial dissipation in those cells. The operator chosen for the diffusion
is a Laplacian acting on all conservative variables. To enable an accurate representation
of turbulence near shocks, the artificial viscosity is multiplied on each quadrature points
by the Ducros function
29
, which reduces its amplitude in vorticity-dominated regions.
Additionally, in order to enhance robustness of the flow solver, the positivity preserving
limiter of Wang et al.
30
is applied whenever a negative pressure or density is detected at
13
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
an element or face quadrature point of the DG discretization.
b. Conservative Finite Difference Solver - SPADE
Static Polymorphic Algorithms for Differential Equations (SPADE) is a newly-developed
library for the solution of spatially-varying differential equations using finite-volume and
finite-difference methods. SPADE is written in native C++20 and makes heavy use of
metaprogramming to provide a device-agnostic framework for the implementation and
optimization of arbitrary numerical kernels. The solver used in this study is a prototype
for a new version of the CHAMPS (Cartesian High-fidelity Adaptive Multi-Physics Solver)
code from University of Maryland31,32.
For this study, a conservative finite-difference formulation is employed. The viscous
fluxes are computed using a standard 2
nd
-order scheme. To compute the inviscid fluxes,
a hybrid scheme is employed as detailed in
18
: two fluxes are computed, one of which
is done using a centered kinetic energy and entropy preserving scheme
33,34
(centered
schemes of order 2, 4, 6, and 8 are considered in this work), and the other is a 3
rd
-order
upwind flux reconstruction. The two schemes are combined with a linear homotopy using
the shock sensor from Ducros et al.
29
as a parameter. There is no model introduced to
account for subgrid scales.
SPADE is developed jointly between the University of Oxford and the University of
Maryland.
c. High-order Finite Difference Solver - OpenSBLI
OpenSBLI
15
is an open-source high-order compressible multi-block flow solver on struc-
tured curvilinear meshes, developed at the University of Southampton and JAXA. Written
in Python, OpenSBLI utilizes symbolic algebra to automatically generate a complete
finite-difference CFD solver in the Oxford Parallel Structured (OPS)
35
Domain-Specific
Language (DSL). Users can define systems of partial differential equations to solve, which
are expanded and discretized symbolically to create a simulation code tailored to the
problem specified. The OPS library is embedded in C/C++ code, enabling massively-
parallel execution of the code on a variety of high-performance-computing architectures
via source-to-source translation, including GPUs. OpenSBLI is explicit in both space and
time, with a range of different discretization options available to users.
For smooth flows, spatial discretization is performed by arbitrary order central dif-
ferences written in quadratic and cubic split forms
36
. Central schemes are also used for
14
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
computing diffusive and heat flux terms. Shock-capturing is performed via Weighted
Essentially Non-Oscillatory (WENO) and Targeted Essentially Non-Oscillatory (TENO)
schemes
8
of arbitrary order. The shock-capturing schemes can either be used to solve
the convective terms directly, or, as a filter step to stabilize the non-dissipative central
schemes. The shock-capturing schemes use a Local Lax-Friedrich (LLF) method to build
the flux reconstruction. Time-advancement is performed by 3
rd
or 4
th
order low-storage
Runge-Kutta schemes
37
. The efficacy of the shock-capturing schemes in OpenSBLI was
assessed for the compressible Taylor-Green vortex case in
6
and for compressible wall-
bounded turbulence in
38
. OpenSBLI also contains adaptive-TENO schemes, which further
lower numerical dissipation via tuning with the addition of a shock sensor. For simplicity,
however, the original 6th order standard TENO formulation
8
is used in this work. A
TENO cut-off threshold value of
CT =
1
×
10
6
is used here for a good balance between
low numerical dissipation and robust shock-capturing.
d. High-order Finite Difference Solver - NS3D
NS3D is a compressible high-order DNS code being continuously developed at the IAG
of the University of Stuttgart. NS3D is written in FORTRAN 2008 and solves the three-
dimensional, unsteady, compressible Navier–Stokes equations in conservative formulation
and Laplace formulation of the viscous terms. NS3D features fully three-dimensional
domain decomposition
17
; communication along the internal boundaries of the resulting
blocks is handled by MPI routines; in addition, the code is hybrid parallelized by means
of OpenMP directives. For spatial discretization, multiple high-order finite-difference
approaches are implemented; for all present computations, subdomain tridiagonal 6
th
-
order compact finite differences are used employing several ghost derivatives outside
the subdomain ends, see
39,40
. Time advancement is performed by the classical explicit
4
th
-order Runge-Kutta scheme, which can be coupled with unconditionally alternating
forward- and backward-biased finite differences for the convective terms, see
14,39
. In
addition, a compact 10
th
-order filter is used every full time step to stabilize the simulation
41
(filter equal in all spatial directions,
α=
0.4), further allowing to treat the convective terms
with purely central schemes for all present cases. For all NS3D simulations of the Taylor-
Green test case presented in the following, stable results were already obtained without
the additional use of shock capturing. Therefore, as a valuable comparison reference for
cases with active shock-capturing, it was decided to calculate all NS3D results without
15
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
additional shock-capturing. Obviously, however, this approach only gives high-quality
results for cases with adequate DNS resolution but leads to strong oscillations in e.g. the
later shown Mach number profiles in the shock regions for the under-resolved cases. In
nondimensionalized units, the time steps for the four simulations have been chosen to
be
t=
0.00125, 0.0025, 0.005 and 0.01 for the 512
3
, 256
3
, 128
3
and 64
3
resolution cases,
respectively. Post-processing is performed with 6
th
order compact finite differences and
thus with the same schemes as the simulations were run. For lower orders, e.g. explicit 4
th
order finite differences, deviations in
ϵs
of up to 5% have been found for the supersonic
case.
e. High-order Spectral Difference Solver - SD3DvisP
The SD3DvisP solver, originally developed by Antony Jameson’s group at Stanford
University, is an MPI parallelized FORTRAN 90 code for compressible viscous flows
based on the high-order spectral difference (SD) scheme for unstructured hexahedral
elements.
10,19,42,43
Inviscid numerical fluxes at element interfaces are computed via the
Roe’s Riemann solver with entropy fix
44
. In particular, the first Harten and Hyman
entropy correction is considered
45,46
. As regards the viscous fluxes, the centered flux
introduced by Sun, Wang, and Liu
42
is considered. The time integration is done explicitly
with a third-order, three-stage, total variation diminishing (RK33-TVD) Runge-Kutta
scheme.
47
The shock capturing technique considered for SD3DvisP is described in detail in
Lodato
48
and is based on the sub-cell shock capturing method
27,49
with the shock sensor
based on density modes and the artificial viscosity parameters (including the ramp and
threshold parameters) being set using a self-calibration procedure. The amplitude of
artificial viscosity
Cε
is set to 0.5 instead of 1 to avoid too strong CFL restrictions. The
artificial viscosity is first computed as a single value in each mesh element, then made
C0
continuous via linear interpolation. To do so, an average of the artificial viscosity values is
computed at each node of the mesh from the values in the surrounding elements, then
local linear interpolations weights are found using the 8 nodal values per hexahedral
element corresponding to the elements vertices. The artificial viscosity values are then
interpolated at volume quadrature points. The artificial diffusion operator considered
is the one presented in Tonicello, Lodato, and Vervisch
50
, where the artificial diffusion
acts as a bulk viscosity on the momentum equations, and as a thermal diffusivity on the
energy equation. No artificial diffusion is considered for the mass conservation equation.
16
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
A positivity preserving procedure is also employed to enhance the robustness of the
simulations.48,51
f. Discontinuous Galerkin Spectral Element solver - FLEXI
FLEXI
52
is an open source framework for solving hyperbolic - parabolic systems of
equations on unstructured grids via a high order Discontinuous Galerkin Spectral Element
Method (DGSEM)
21
. The main areas of application of FLEXI are multiscale / multiphase /
multiphysics problems of compressible aerodynamics
53
. FLEXI is developed and main-
tained by the Numerics Research Group at the IAG of the University of Stuttgart. The
framework consists of the open source high order preprocessor HOPR
54
for handling
and generation of curved, unstructured grids, the PDE solver FLEXI and a high order
postprocessing suite with ParaView plugin. All parts are written in FORTRAN 2008
and parallelized with MPI3.0; strong scaling on up to 262.000 CPUs shows superlinear
behaviour down to one element per processor55.
In this study, FLEXI discretizes the compressible Navier-Stokes equations by a fourth
order accurate DGSEM scheme in space, optionally combined with a subcell finite volume
method to capture regions with shocks. Temporal integration is performed by a fourth
order accurate low storage, explicit Runge-Kutta scheme
37
. In the DGSEM formulation,
the solution is approximated by tensor products of 1D-Lagrange polynomials of arbitrary
degree
N
on a reference element. The local polynomial degree can be adapted dynamically
to the solution. The nodes for this basis are chosen as Legendre-Gauss or Legendre-
Gauss-Lobatto points. This collocation of interpolation and integration points transfers
the tensorproduct structure to the DG operator and results in dimension-by-dimension
operations, as opposed to volume operations in other DG variants. As an inviscid numeri-
cal flux function, the approximate Riemann solver by Roe with an entropy fix is used. The
second entropy fix of Harten and Hyman was used
46
, which has the benefit of not having
a user-defined constant entropy-fix threshold. For the viscous fluxes, the first method of
Bassi and Rebay is chosen
25
. Various subgrid scale closure models for LES computations
are available, however in this study, no explicit modelling of the unclosed terms is present.
Shock capturing is based on a hybrid Finite Volume / DGSEM scheme. In grid elements
in which a suitable sensor detects the occurence of a shock wave, the DG solution is
projected onto a compatible FV subgrid (Cartesian in reference space) that shares the same
data-structure as the DG solution. On this element-local subgrid, instead of the DGSEM
17
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
scheme, a second order TVD FV scheme with a generalized minmod limiter
56
is solved. To
detect discontinuous solution features, an indicator based on the modal decomposition of
the polynomial solution ansatz is employed. It is evaluated on the pressure and infers the
smoothness of the element local solution from the exponential decay rate of the modes
57,58
.
The combination of a high order DG scheme with a local and robust FV formulation
allows an efficient resolution of smooth regions and a sharp capturing of discontinuities
through an adaptation of the approximation space to the underlying solution. More details
on this hybrid DGSEM / FV scheme, its implementation and validation for single- und
multiphase flows can be found in57,59,60.
g. Flux Reconstruction solver - H3AMR
H3AMR (HySonic, High-Order, Hybrid Adaptive Mesh Refinement)
22,61,62
is an in-
house code developed by HySonic Technology, LLC. The code solves the compressible
Navier-Stokes equations using the flux reconstruction
12
numerical scheme for unstruc-
tured meshes. The solver is block-spectral based
10
therefore the simulation domain is
divided into blocks - or mesh elements - and inside each element, the solution is stored
at
N
Gauss-Legendre quadrature points in each direction,
N
corresponding here also
to the spatial order of accuracy. Hence, the number of elements is equal to the number
of DoFs divided by
N3
. The code has several capabilities from different LES and shock
capturing methods to
h
and
p
refinement capabilities. For the purpose of this work, we are
going to explain only the functionality used for this simulation. Time advancement is a
third-order Runge-Kutta method. The flux reconstruction method relies on an average of
different orders of Radau polynomials to compute the correction functions and the fluxes
are updated at the faces using the Rusanov method. The shock capturing method is a new
numerical method developed in Carlo Scalo’s group at Purdue University called Block
Spectral Stresses (BSS)
63
, able to do shock-capturing and turbulence modeling at the same
time. The modeling is specifically developed for the flux reconstruction method and it
relies on the spectra of the velocity gradient to compute the sub-filter stresses, heat flux,
and pressure-work.
h. Summary of shock capturing approaches
In this paragraph, the various shock capturing strategies considered for the present
study are briefly reviewed. All of them introduce additional numerical dissipation or
limiting with the aim of smoothing the numerical solution around shocks and mitigate the
18
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
Gibbs oscillations.
To do so, SD3D, CODA and H3AMR use a dissipation operator explicitly added to
the Navier-Stokes equations. For SD3D and CODA, it takes the form of a Laplacian
operator impacting all conservative variables, while for H3AMR the dissipative operator
mimicks a molecular dissipation, impacting the momentum and total energy variables.
The strategies in FLEXI, SPADE and OpenSBLI aim at switching locally to a lower-order
scheme featuring improved shock-capturing properties. FLEXI uses a second-order FV
scheme with the total variation diminishing property, while SPADE uses a third-order
weighted-essentially-non-oscillatory (WENO) scheme. As regards OpenSBLI, the switch
to lower-order is part of the design of the TENO method. NS3D considers a filtering of
the numerical solution using a high-order finite difference operator, which effectively
smoothes the solution at all time steps.
A second important ingredient common to all present methodologies, except NS3D,
is the sensor driving the local activation of the specific shock-capturing treatment. The
discontinuous finite element-based schemes SD3D, CODA, FLEXI and H3AMR exploit
the element-wise polynomial information to devise such sensor. SD3D and CODA use the
sensor introduced by Persson and Peraire
27
which isolates the highest density polynomial
mode in the element, providing a local estimation of the smoothness of the numerical
solution. FLEXI uses a similar idea by estimating the exponential decay of the polynomial
modes magnitude in the elements
58
. H3AMR also uses the energy related to the high-
est polynomial mode of momentum to activate locally the shock-capturing treatment
63
.
SPADE uses the well known Ducros function
29
- which tends to 0 when the vorticity
magnitude locally dominates the divergence of velocity magnitude and to 1 otherwise - to
blend the high-order and diffusive spatial schemes. The TENO scheme in OpenSBLI relies
as well on smoothness indicators based on the polynomials used for the reconstruction of
the numerical solution.
19
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
TABLE I. Summary of numerical methods employed in the present study
Solver Numerical method Order of accuracy Shock capturing
CODA20 Modal DG 4 LAD
SPADE18 Central FD 8 Local upwinding
OpenSBLI15 FD-TENO 6 TENO
NS3D17 Central FD 6 HO filter
SD3DvisP19 SD 4 LAD
FLEXI21 DGSEM 4 subgrid FV
H3AMR22 FR 2 LAD
TABLE II. Number of mesh elements and DoFs for Discontinuous Finite element discretization
Solver Order of accuracy #Mesh elements #DoFs
CODA20 4 233- 473- 943- 1893243,340 - 2,076,460 -
16,611,680 - 135,025,380
SD3DvisP19 4 163- 323- 643- 1283643- 1283- 2563- 5123
FLEXI21 4 163- 323- 643- 1283643- 1283- 2563- 5123
H3AMR22 2 323- 643- 1283- 2563643- 1283- 2563- 5123
IV. Supersonic Taylor-Green vortex simulations: Results and discussion
A. Integrated quantities
This section presents the results obtained with the various flow solvers introduced
in Section III for the simulation of the supersonic Taylor-Green vortex at Mach 1.25 and
Reynolds number 1600. A summary of the solvers, numerical methods, order of accuracy
and type of shock capturing is displayed in Table I. All solvers are considering high orders
of accuracy except for H3AMR which runs with a second order of accuracy. This is of
interest to assess how high-order schemes behave compared to standard second order
schemes for such problems. A mesh convergence study is performed considering four
resolution levels with respectively 64
3
, 128
3
, 256
3
and 512
3
degrees of freedom. Regarding
finite difference-based solvers, the number of DoFs is equivalent to the number of mesh
points considered. However, for discontinuous finite element solvers, considering the
20
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
polynomial resolution inside elements, the total number of DoFs is higher than the number
of mesh elements, and summarized in Table II. The 64
3
DoFs resolution is coarse and is
not expected to fully resolve the fine turbulent scales or the shock features. This coarse
resolution might be relevant for Large-Eddy Simulation modeling, however considering
the relatively low Reynolds number of this configuration, we assume in this work that
the inherent dissipative properties of the considered numerical schemes are sufficient
to mimic the subgrid dissipation. For the 512
3
DoFs resolution, we expect the turbulent
scales to be fully or nearly fully resolved, while the shock profiles are expected to become
sharper but not fully resolved.
Figures 5, 6 and 7 display the time evolution of kinetic energy, dilatational dissipation
and solenoidal dissipation, respectively. The results are presented for all solvers and
resolutions.
Regarding the kinetic energy evolution, all solvers are able to capture it accurately with
128
3
DoFs and above. Although the agreement is also good for the 64
3
grid, this resolution
features the most differences between flow solvers and is therefore interesting to discuss
further. An underestimation of kinetic energy levels in the early stages is observed for
the CODA (DG), SD3DvisP (SD) and OpenSBLI (TENO) flow solvers. This behavior is
likely to be related to the intrinsic numerical dissipation of these schemes, stemming from
Riemann-based numerical fluxes defined at the interfaces between elements. On the other
hand, the SPADE and NS3D solvers based on numerical dissipation-free high-order FD
strategies provide a very accurate prediction of kinetic energy on the coarse grid in the
early and intermediate flow regimes (about up to
t=
12), but display slightly excessive
levels in the later stages (
t>
12), for which the turbulence is fully developed and the large-
scale energy cascade communicates kinetic energy towards the smallest scales that can be
represented by the discretization. This behavior could be explained by a slight small-scale
energy pile-up located around the grid cut-off wavenumber and a lack of subgrid scale
closure, as in this case the central FD schemes (unlike upwinded schemes) do not introduce
a sufficient numerical dissipation that could mimic the subgrid dissipation. The results of
FLEXI are more consistent with the FD-based solvers SPADE and NS3D, despite using
the same Riemann-based flux approximations as CODA and SD3DvisP. However, the
shock capturing strategy of FLEXI and thus the introduced numerical diffusion differs
considerably from the LAD approaches, which likely explains these differences. Finally,
21
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
the H3AMR second order 64
3
DoFs simulation displays underestimated kinetic energy
levels representative of an excess of numerical dissipation, typically found for second
order upwind schemes. It is interesting to verify that higher order schemes with the same
number of DoFs have better kinetic energy resolution properties in this case. Overall, the
present results confirm that all schemes considered provide an accurate representation of
the large turbulent scales in the flow and therefore low numerical dissipation levels in the
lower wavenumber range, which is a typical feature of high-order schemes that all solvers
manage to preserve in this compressible/shock-turbulence interaction case.
The dilatational dissipation levels, governed by the magnitude of divergence of veloc-
ity, are mainly driven by strong compression and dilatation effects, which in turn leads
to a significant contribution of shocks that are numerically correlated with important
divergence levels. Low levels of dilatational dissipation will tend to indicate that a given
shock capturing method is strong, and yield smooth and thick shock profiles. Conversely,
methods able to yield sharp shock profiles will display higher levels, with the risk of pro-
ducing oscillations which can also appear in the time evolution of dilatational dissipation.
We can clearly see that NS3D, which features a high-order central FD scheme coupled
with a high-order filtering, yielding overall low levels of shock-dissipation, produces the
highest magnitude of dilatational dissipation among the solvers considered, with oscilla-
tions around the first peak identified for all resolutions. FLEXI results display a slightly
lower dissipation and comparable oscillations for the lower resolutions. This behavior is
due to the switching of the DG operator to a FV operator and back for individual grid
elements as shock appear or decay. Next, CODA (DG) and OpenSBLI (TENO) provide
lower levels, but with reduced oscillations, in particular for OpenSBLI which is clear of
oscillations regardless of the grid resolution. CODA starts displaying slight oscillations
around the first peak with the 256
3
resolution, which could be related to the locality of
artificial diffusivity, which transitions sharply from zero to maximal values in cells where
shocks are localized. SPADE, SD3D and H3AMR appear to be the most impacting solvers
with respect to shock capturing, and display the lower levels of dilatational dissipation on
all grids.
Lastly, the solenoidal dissipation, being correlated to the quality of resolution of the
smaller turbulent scales, is assessed. This quantity is more challenging to capture correctly
compared to the kinetic energy as seen from Figure 7, and is also likely to be impacted
22
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
by the shock capturing strategy, which might dissipate excessively small turbulent scales.
Some discrepancies are observed between flow solvers, in particular considering the
coarser resolutions. The rate of convergence with respect to the resolution also differs
from method to method. CODA, FLEXI and NS3D display the highest levels on all
grids, which could be explained by their relatively lenient shock capturing approach,
which does not interfere much with the onset and dynamics of the small turbulent scales.
It is interesting to note that NS3D provides smooth solenoidal dissipation evolutions,
as well as satisfactory mesh convergence, which indicates that a mild shock capturing
approach does not seem to impact the characterization of small vortices. SPADE displays
lower levels, which in the same fashion can be explained by the interaction of the shock
capturing scheme with the development of small vortices. SD3D starts off with high levels
of solenoidal dissipation on the coarse grid, but falls in the lower range for more refined
resolutions. Here again, its strong shock capturing strategy is probably hindering to some
extent the development of small scales on the finer grids. Finally, OpenSBLI displays
low levels on coarse grids but good convergence properties, as its levels of solenoidal
dissipation move into the higher range among flow solvers when the grid is refined. As
regards the H3AMR results, they clearly show that second order schemes struggle with
capturing accurately fine turbulent scales, as the levels are lower compared to higher-order
solvers for all resolutions. Here again, the interest of increasing the order of accuracy
for an accurate capture of shocks interacting with small-scale turbulence is emphasized.
Overall, all solvers display a comparable and consistent behavior regarding the solenoidal
dissipation, showing that most of the turbulent dynamics is captured by all solvers using
a resolution of 256
3
DoFs or above. For coarser grids, present results emphasize that
high-order schemes are still able to capture a significant part of the small-scale dynamics.
In particular, discontinuous finite elements schemes seem to be efficient in that respect.
For all high-order schemes, the solenoidal dissipation plots show that small turbulent
scales are better represented with respect to a second-order discretization.
23
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
0 5 10 15 20
t
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Ek
CODA
OpenSBLI
NS3D
SD3D
SPADE
FLEXI
H3AMR
Reference
0 5 10 15 20
t
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Ek
0 5 10 15 20
t
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Ek
0 5 10 15 20
t
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Ek
FIG. 5. Code comparison study for the TGV flow at
Re =
1600 and
M0=
1.25: Time evolution of
kinetic energy for the four resolutions considered. Top left: 64
3
DoFs; Top right: 128
3
DoFs; Bottom
left: 2563DoFs; Bottom right: 5123DoFs. Reference: 20483sixth-order TENO simulation.
24
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
0 5 10 15 20
t
0.0000
0.0002
0.0004
0.0006
0.0008
0.0010
0.0012
ǫd
CODA
OpenSBLI
NS3D
SD3D
SPADE
FLEXI
H3AMR
Reference
0 5 10 15 20
t
0.0000
0.0002
0.0004
0.0006
0.0008
0.0010
0.0012
ǫd
0 5 10 15 20
t
0.0000
0.0002
0.0004
0.0006
0.0008
0.0010
0.0012
ǫd
0 5 10 15 20
t
0.0000
0.0002
0.0004
0.0006
0.0008
0.0010
0.0012
ǫd
FIG. 6. Code comparison study for the TGV flow at
Re =
1600 and
M0=
1.25: Time evolution of
dilatational dissipation for the four resolutions considered. Top left: 64
3
DoFs; Top right: 128
3
DoFs;
Bottom left: 2563DoFs; Bottom right: 5123DoFs. Reference: 20483sixth-order TENO simulation.
25
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
0 5 10 15 20
t
0.000
0.002
0.004
0.006
0.008
0.010
0.012
ǫs
CODA
OpenSBLI
NS3D
SD3D
SPADE
FLEXI
H3AMR
Reference
0 5 10 15 20
t
0.000
0.002
0.004
0.006
0.008
0.010
0.012
ǫs
0 5 10 15 20
t
0.000
0.002
0.004
0.006
0.008
0.010
0.012
ǫs
0 5 10 15 20
t
0.000
0.002
0.004
0.006
0.008
0.010
0.012
ǫs
FIG. 7. Code comparison study for the TGV flow at
Re =
1600 and
M0=
1.25: Time evolution of
solenoidal dissipation for the four resolutions considered. Top left: 64
3
DoFs; Top right: 128
3
DoFs;
Bottom left: 2563DoFs; Bottom right: 5123DoFs. Reference: 20483sixth-order TENO simulation.
26
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
B. Mach profiles
In this part, the numerical representation of shocks by the various methods is assessed
from Mach profiles extracted along
x=z=
0
y
lines in the computational domain. The
chosen time is
t=
2.5, which corresponds to the peak of dilatational dissipation, for which
the shocks are the strongest.
The results are displayed in Figure 8 for the four resolutions considered. We can observe
different behavior between the various flow solvers. On the coarser grids featuring 64
3
and 128
3
DoFs, NS3D, which only relies on a high-order filter for stabilization without
any particular treatment of shocks, clearly displays the strongest Gibbs oscillations. On
the other hand, SD3D, which features the strongest shock capturing approach, displays
oscillation-free but thick shock representations. The TENO6 scheme of OpenSBLI displays
sharp shocks and is oscillation-free for all cases considered. CODA also displays sharp
profiles with limited oscillations on all grids, emphasizing the potential of Discontinuous
Galerkin type of discretization to capture shocks efficiently despite the large element sizes.
The shock profiles produced by FLEXI are the sharpest ones due to the embedded FV
scheme. They match the reference solution shock profiles on the 256
3
grid, but show oscil-
lations below this resolution. In summary, while all presented solvers converge towards
the reference solution, this comparison stresses the consequences of different numerical
strategies and highlights the complex, non-linear interplay between discretization proper-
ties, shock capturing and turbulent flows. In particular, this study shows the difficulty
of obtaining a consistent shock capturing technique accross all resolutions considered.
Indeed, several shock capturing techniques are found to produce oscillations on coarse
grids, which vanish when the mesh is refined. TENO schemes and LAD-based schemes
seem to perform better at mitigating shock oscillations for all grid resolutions. This topic
could be investigated in future research to yield further improvements of the proposed
techniques.
27
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
y/L
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
Mach
CODA
OpenSBLI
NS3D
SD3D
FLEXI
SPADE
H3AMR
Ref. TENO6 512
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
y/L
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
Mach
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
y/L
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
Mach
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
y
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
Mach
FIG. 8. Code comparison study for the TGV flow at
Re =
1600 and
M0=
1.25: Mach profiles
extracted from the yline at x=z=0, for the four resolutions considered. Top left: 643DoFs; Top
right: 1283DoFs; Bottom left: 2563DoFs; Bottom right: 5123DoFs.
V. Conclusions
Several high-order spatial schemes were tested for the shock-turbulence interaction
problem stemming from the supersonic Taylor-Green case at Reynolds 1600 and Mach
1.25. The spatial discretization approaches featured both high-order finite element and
high-order finite differences with a variety of shock capturing techniques including LAD,
filtering, upwind numerical fluxes and subcell limiting. From the time evolution of ki-
netic energy, integrated over the computational domain, the large-scale dynamics were
found to be accurately captured by all high-order schemes, even considering coarse grids.
Solenoidal dissipation plots also showed an accurate representation and fast mesh con-
28
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
vergence regarding the small-scale dynamics. Mach profiles and dilatational dissipation
plots showed differences in the treatment of shocks by the various numerical strategies
considered. Strong shock capturing strategies yield thick shock profiles and low levels of
dilatational dissipation, while more lenient strategies tend to display Gibbs oscillations
and high levels of dilatational dissipation.
Overall, this study showed the capability of high-order schemes to represent accu-
rately turbulence dynamics in a compressible setting, with a low impact of the shock
capturing treatment on the representation of turbulent scales. A few notable points can be
highlighted:
In terms of shock capturing methods, all approaches considered were able to provide
a qualitatively similar representation of shocks on fine grids. Most differences were
observed on coarser grids, for which it was found that approaches combining
artificial diffusion yield smooth but thick shock profiles, while approaches relying
on local switching to lower-order schemes tend to provide sharp but oscillating
shocks, except the TENO scheme which displays relatively sharp shocks without
oscillations. The finite difference scheme using only a high-order filter to stabilize
simulations provided the highest oscillations.
In terms of spatial accuracy, all high-order schemes considered provided an accurate
representation of the kinetic energy and large turbulent scales, even considering
coarser grids. As regards solenoidal dissipation and small-scale dynamics, solvers
provided similar results on fine grids but differences were observed on the coarser
grids. Discontinuous finite element schemes - including DG, SD and DGSEM vari-
ants - captured well the small scales on coarser grids, as well as high-order finite
difference with high-order filtering stabilazation. Finite difference with local scheme
switching as well as high-order TENO yield more small-scale dissipation. Over-
all, better large and small-scale turbulence resolution properties were found for
high-order schemes against second-order solutions, even in the present challenging
compressible setting.
The dilatational dissipation quantity provides valuable information on the represen-
tation of shocks by the numerical method. Its amplitude is directly related to the
strength of the shock capturing method and the shock thickness. Strong LAD-based
29
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
shock capturing techniques, such as the one used in the SD3D code, yield the lowest
values and thicker shock profiles while NS3D, which features only a stabilization
technique with no specific shock treatment, yield the highest values and strong shock
oscillations. Oscillations in time of the dilatational dissipation were also found to
be correlated with Gibbs oscillations around shocks. In future studies, additional
quantities characterizing compressibility effects could be considered - such as the
temperature variance
7
- in order to quantify better the quality of resolution of such
effects by the numerical methods.
This study has highlighted some methodological aspects that could benefit from future
improvements: reduce shock oscillations for approaches based on a local switch to a
lower-order scheme; increase shock sharpness while keeping low oscillations for artificial
viscosity-based approaches; pursue the development of shock or smoothness sensor to
maximize the quality of both shock and turbulence resolution for any grid refinement.
This study focused on the assessment of spatial discretization schemes, but future
investigation could also address the assessment of time integration schemes and their
influence on the accuracy and performance of simulations.
The current work showed promising progress in the development of a variety of high-
order schemes - including FD, SD, DG, DGSEM and FR - in order to perform accurate
and robust numerical simulations of complex phenomena involving strong shocks and
turbulence. Applications of interest involve cruising aircraft configurations, atmospheric
re-entry vehicles, supersonic engines or astrophysical fluid dynamics.
Future research will consider similar comparison of numerical approaches for more
complex cases, including shocks interacting with wall-bounded turbulence.
A. Code verification, TGV case at Re =500,M0=0.5
This appendix presents a verification study of the flow solver implementations of the
Taylor-Green vortex case, including the various codes considered in the present study. To
alleviate the computational burden of this verification and the potential variations due
to shock capturing techniques, the flow conditions are set to
Re =
500 and
M0=
0.5,
ensuring a limited turbulent scales development and a subsonic regime. The target
resolution is 256
3
DoFs, which should suffice for most numerical schemes, based on the
30
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
observation that a case at
Re =
500 and
M0=
0.1 was fully resolved in terms of kinetic
energy and enstrophy using a Fourier spectral method and 128
3
DoFs
5
. For the present
simulations, minor offsets were observed for the SPADE and SD3D codes using a 256
3
resolution, hence 512
3
and 384
3
resolution were considered for these codes, respectively,
to ensure that the offsets were due to insufficient mesh convergence rather than problems
in the initialization and discretization. As regards H3AMR, a 320
3
DoFs resolution with
a fourth order of accuracy was considered for this verification run. The simulations are
run up to
t=
10 which is sufficient to assess the code behavior from the initial condition
to the end of the turbulent small-scales build-up process. The results of the verification
runs are shown in Figures 9 and 10. We find that the agreement between codes is perfect
for the time evolution of kinetic energy and solenoidal dissipation, showing an identical
representation of the full extent of turbulent scales by all codes and methods. The same
observation applies for the dilatational dissipation up to
t=
9, after which there are minor
discrepancies between codes, probably due to high resolution needed for capturing short
acoustic waves radiated by the small-scale turbulent structures. Overall, the verification
test is successful in confirming that all codes provide a sound implementation of the
compressible Taylor-Green vortex case considered in the present study.
31
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
0246810
t
0.04
0.05
0.06
0.07
0.08
0.09
0.10
0.11
0.12
0.13
Ek
CODA 256
OpenSBLI 256
NS3D 256
SD3D 384
FLEXI 384
H3AMR 320
SPADE 512
FIG. 9. Code validation for the TGV flow at
Re =
500 and
M0=
0.5: Time evolution of kinetic
energy.
0 2 4 6 8 10
t
0.000
0.002
0.004
0.006
0.008
0.010
0.012
ǫs
0 2 4 6 8 10
t
0
1
2
3
4
5
ǫd
×106
FIG. 10. Code validation for the TGV flow at
Re =
500 and
M0=
0.5: Time evolution of solenoidal
dissipation (left) and dilatational dissipation (right).
32
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
References
1
M. E. Brachet, D. I. Meiron, S. A. Orszag, B. Nickel, R. H. Morf, and U. Frisch, “Small-
scale structure of the Taylor–Green vortex,” Journal of Fluid Mechanics 130, 411–452
(1983).
2
Z. J. Wang, K. Fidkowski, R. Abgrall, F. Bassi, D. Caraeni, A. Cary, H. Deconinck,
R. Hartmann, K. Hillewaert, H. T. Huynh, et al., “High-order CFD methods: current
status and perspective,” International Journal for Numerical Methods in Fluids 72,
811–845 (2013).
3
C. Carton de Wiart, K. Hillewaert, M. Duponcheel, and G. Winckelmans, “Assessment
of a discontinuous Galerkin method for the simulation of vortical flows at high Reynolds
number,” International Journal for Numerical Methods in Fluids 74, 469–493 (2014).
4
G. J. Gassner and A. D. Beck, “On the accuracy of high-order discretizations for under-
resolved turbulence simulations,” Theoretical and Computational Fluid Dynamics 27,
221–237 (2013).
5
J.-B. Chapelier, M. De La Llave Plata, F. Renac, and E. Lamballais, “Evaluation of a
high-order discontinuous Galerkin method for the DNS of turbulent flows,” Computers
& Fluids 95, 210–226 (2014).
6
D. J. Lusher and N. D. Sandham, “Assessment of low-dissipative shock-capturing
schemes for the compressible Taylor–Green vortex,” AIAA Journal 59, 533–545 (2021).
7
E. Johnsen, J. Larsson, A. V. Bhagatwala, W. H. Cabot, P. Moin, B. J. Olson, P. S. Rawat,
S. K. Shankar, B. Sj
¨
ogreen, H. C. Yee, et al., “Assessment of high-resolution methods
for numerical simulations of compressible turbulence with shock waves,” Journal of
Computational Physics 229, 1213–1237 (2010).
8
L. Fu, X. Y. Hu, and N. A. Adams, “A family of high-order targeted ENO schemes for
compressible-fluid simulations,” Journal of Computational Physics 305, 333–359 (2016).
9
B. Cockburn, “Discontinuous Galerkin methods for convection-dominated problems,”
in High-Order Methods for Computational Physics (Springer, 1999) pp. 69–224.
10
D. A. Kopriva and J. H. Kolias, “A conservative staggered-grid Chebyshev multidomain
method for compressible flows,” Journal of Computational Physics 125, 244–261 (1996).
11
Y. Liu, M. Vinokur, and Z. J. Wang, “Spectral difference method for unstructured grids I:
Basic formulation,” Journal of Computational Physics 216, 780–801 (2006).
33
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
12H. T. Huynh, “A flux reconstruction approach to high-order schemes including discon-
tinuous Galerkin methods,” in 18th AIAA computational fluid dynamics conference (2007) p.
4079.
13
S. K. Lele, “Compact finite difference schemes with spectral-like resolution,” Journal of
Computational Physics 103, 16–42 (1992).
14M. J. Kloker, “A robust high-resolution split-type compact FD scheme for spatial direct
numerical simulation of boundary-layer transition,” Applied Scientific Research 59, 25
(1997).
15
D. J. Lusher, S. P. Jammy, and N. D. Sandham, “OpenSBLI: Automated code-generation
for heterogeneous computing architectures applied to compressible fluid dynamics on
structured grids,” Computer Physics Communications 267, 108063 (2021).
16
L. Fu, “Review of the high-order TENO schemes for compressible gas dynamics and
turbulence,” Archives of Computational Methods in Engineering 30, 2493–2526 (2023).
17
C. Wenzel, J. M. F. Peter, B. Selent, M. Weinschenk, U. Rist, and M. J. Kloker, “DNS of
compressible turbulent boundary layers with adverse pressure gradients,” in W.E. Nagel
(ed.) High Performance Computing in Science and Engineering 2018 (Springer, 2018).
18
W. van Noordt, S. Ganju, and C. Brehm, “An immersed boundary method for wall-
modeled large-eddy simulation of turbulent high-mach-number flows,” Journal of
Computational Physics 470, 111583 (2022).
19
J.-B. Chapelier, G. Lodato, and A. Jameson, “A study on the numerical dissipation
of the spectral difference method for freely decaying and wall-bounded turbulence,”
Computers & Fluids 139, 261–280 (2016).
20
P. Stefanin Volpiani, J.-B. Chapelier, A. Schw
¨
oppe, J. J
¨
agersk
¨
upper, and S. Champagneux,
“Aircraft simulations using the new cfd software from onera, dlr, and airbus,” Journal of
Aircraft (2024).
21
N. Krais, A. Beck, T. Bolemann, H. Frank, D. Flad, G. Gassner, F. Hindenlang, M. Hoff-
mann, T. Kuhn, M. Sonntag, et al., “Flexi: A high order discontinuous Galerkin frame-
work for hyperbolic–parabolic conservation laws,” Computers & Mathematics with
Applications 81, 186–219 (2021).
22
C. L. Running, B. L. Bemis, J. L. Hill, M. P. Borg, J. J. Redmond, K. Jantze, and C. Scalo,
“Attenuation of hypersonic second-mode boundary-layer instability with an ultrasoni-
cally absorptive silicon-carbide foam,” Experiments in Fluids 64, 79 (2023).
34
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
23
F. Bassi, L. Botti, A. Colombo, D. A. Di Pietro, and P. Tesini, “On the flexibility of
agglomeration based physical space discontinuous Galerkin discretizations,” Journal of
Computational Physics 231, 45–65 (2012).
24
F. Basile, J.-B. Chapelier, M. de la Llave Plata, R. Laraufie, and P. Frey, “Unstructured
h
-and
hp
-adaptive strategies for discontinuous Galerkin methods based on a posteriori
error estimation for compressible flows,” Computers & Fluids 233, 105245 (2022).
25
F. Bassi and S. Rebay, “A high-order accurate discontinuous finite element method
for the numerical solution of the compressible Navier–Stokes equations,” Journal of
computational physics 131, 267–279 (1997).
26
F. Bassi, S. Rebay, G. Mariotti, S. Pedinotti, M. Savini, et al., “A high-order accurate discon-
tinuous finite element method for inviscid and viscous turbomachinery flows,” in 2-nd
European Conference on Turbomachinery Fluid Dynamics and Thermodynamics (Technologisch
Instituut, Antwerpen, Belgium, 1997) pp. 99–108.
27
P.-O. Persson and J. Peraire, “Sub-cell shock capturing for discontinuous Galerkin meth-
ods,” in 44th AIAA Aerospace Sciences Meeting and Exhibit (2006) p. 112.
28
J. Glaubitz, A. Nogueira, J. L. Almeida, R. Cant
˜
ao, and C. Silva, “Smooth and com-
pactly supported viscous sub-cell shock capturing for discontinuous Galerkin methods,”
Journal of Scientific Computing 79, 249–272 (2019).
29
F. Ducros, V. Ferrand, F. Nicoud, C. Weber, D. Darracq, C. Gacherieu, and T. Poinsot,
“Large-eddy simulation of the shock/turbulence interaction,” Journal of Computational
Physics 152, 517–549 (1999).
30
C. Wang, X. Zhang, C.-W. Shu, and J. Ning, “Robust high order discontinuous Galerkin
schemes for two-dimensional gaseous detonations,” Journal of Computational Physics
231, 653–665 (2012).
31
S. Ganju, W. van Noordt, and C. Brehm, “Progress in the development of an immersed
boundary viscous-wall model for 3d and high-speedflows,” in AIAA Scitech 2021 Forum
(2021) p. 0160.
32
C. Brehm, M. F. Barad, and C. C. Kiris, “Development of immersed boundary computa-
tional aeroacoustic prediction capabilities for open-rotor noise,” Journal of Computa-
tional Physics 388, 690–716 (2019).
33
Y. Kuya, K. Totani, and S. Kawai, “Kinetic energy and entropy preserving schemes for
compressible flows by split convective forms,” Journal of Computational Physics 375,
35
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
823–853 (2018).
34
K. Totani, Y. Kuya, and S. Kawai, “High-order-accurate kinetic energy and entropy
preserving schemes on curvilinear meshes,” in AIAA Scitech 2019 Forum (2019) p. 1403.
35I. Z. Reguly, G. R. Mudalige, M. B. Giles, D. Curran, and S. McIntosh-Smith, “The OPS
Domain Specific Abstraction for Multi-block Structured Grid Computations,” (IEEE
Press, 2014) pp. 58–67.
36
G. Coppola, F. Capuano, S. Pirozzoli, and L. de Luca, “Numerically stable formulations
of convective terms for turbulent compressible flows,” Journal of Computational Physics
382, 86–104 (2019).
37
M. H. Carpenter and C. A. Kennedy, “Fourth-order 2N-storage Runge-Kutta schemes,”
NASA Langley Research Center (1994).
38
A. Hamzehloo, D. J. Lusher, S. Laizet, and N. D. Sandham, “On the performance of
WENO/TENO schemes to resolve turbulence in DNS/LES of high-speed compressible
flows,” International Journal for Numerical Methods in Fluids 93, 176–196 (2021).
39
A. Babucke, Direct Numerical Simulation of Noise-Generation Mechanisms in the Mixing
Layer of a Jet (Verlag Dr. Hut, 2009).
40
M. A. Keller and M. J. Kloker, “Effusion cooling and flow tripping in laminar supersonic
boundary-layer flow,” AIAA Journal 53, 902–919 (2014).
41
M. R. Visbal and D. V. Gaitonde, “On the use of higher-order finite-difference schemes
on curvilinear and deforming meshes,” Journal of Computational Physics 181, 155–185
(2002).
42
Y. Sun, Z. Wang, and Y. Liu, “High-order multidomain spectral difference method for
the Navier-Stokes equations on unstructured hexahedral grids,” Communications in
Computational Physics 2, 310–333 (2007).
43
A. Jameson, “A proof of the stability of the spectral difference method for all orders of
accuracy,” Journal of Scientific Computing 45, 348–358 (2010).
44
P. Roe, “Approximate Riemann solvers, parameter vectors, and difference schemes,”
Journal of Computational Physics 43, 357–372 (1981).
45
A. Harten and J. M. Hyman, “Self adjusting grid methods for one-dimensional hyperbolic
conservation laws,” Journal of computational Physics 50, 235–269 (1983).
46
M. Pelanti, L. Quartapelle, and L. Vigevano, “A review of entropy fixes as applied to
Roe’s linearization,” Teaching material of the Aerospace and Aeronautics Department of
36
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
Politecnico di Milano 31 (2001).
47
S. Gottlieb and C. Shu, “Total variation diminishing Runge-Kutta schemes,” Mathematics
of Computation 67, 73–85 (1998).
48
G. Lodato, “Characteristic modal shock detection for discontinuous finite element meth-
ods,” Computers & Fluids 179, 309–333 (2019).
49
P.-O. Persson, “Shock capturing for high-order discontinuous Galerkin simulation of
transient flow problems,” in 21st AIAA Computational Fluid Dynamics Conference (2013)
p. 9.
50
N. Tonicello, G. Lodato, and L. Vervisch, “Analysis of high-order explicit LES dynamic
modeling applied to airfoil flows,” Flow, Turbulence and Combustion 108, 77–104 (2022).
51
X. Zhang and C.-W. Shu, “On positivity-preserving high order discontinuous Galerkin
schemes for compressible Euler equations on rectangular meshes,” Journal of Computa-
tional Physics 229, 8918–8934 (2010).
52https://github.com/flexi-framework.
53
A. D. Beck, T. Bolemann, D. Flad, H. Frank, G. J. Gassner, F. Hindenlang, and C.-D.
Munz, “High-order discontinuous Galerkin spectral element methods for transitional
and turbulent flow simulations,” International Journal for Numerical Methods in Fluids
76, 522–548 (2014).
54https://github.com/flexi-framework/hopr.
55
M. Blind, M. Gao, D. Kempf, P. Kopper, M. Kurz, A. Schwarz, and A. Beck, “Towards ex-
ascale CFD simulations using the Discontinuous Galerkin solver FLEXI,” arXiv preprint
arXiv:2306.12891 (2023).
56
B. Van Leer, “Towards the ultimate conservative difference scheme. IV. A new approach
to numerical convection,” Journal of Computational Physics 23, 276–299 (1977).
57
P. Mossier, A. Beck, and C.-D. Munz, “A p-adaptive discontinuous Galerkin method
with hp-shock capturing,” Journal of Scientific Computing 91, 4 (2022).
58
C. Mavriplis, “A posteriori error estimators for adaptive spectral element techniques,”
in Proceedings of the Eighth GAMM-Conference on Numerical Methods in Fluid Mechanics,
edited by P. Wesseling (Vieweg+Teubner Verlag, Wiesbaden, 1990) pp. 333–342.
59
M. Sonntag and C.-D. Munz, “Efficient parallelization of a shock capturing for discontin-
uous Galerkin methods using finite volume sub-cells,” Journal of Scientific Computing
70, 1262–1289 (2017).
37
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
60
P. Mossier, D. Appel, A. D. Beck, and C.-D. Munz, “An efficient hp-adaptive strategy for
a level-set ghost-fluid method,” Journal of Scientific Computing 97, 50 (2023).
61
B. L. Bemis, J. L. Hill, M. P. Borg, J. J. Redmond, M. Ruggeri, K. Jantze, C. Scalo, and
C. L. Running, “Effect of porosity on the ability of silicon-carbide foams to attenuate
second-mode boundary-layer instability,” in AIAA AVIATION 2023 Forum (2023) p. 3706.
62
B. L. Bemis, J. L. Brun, C. T. Wanstall, J. J. Hill, M. P. Borg, J. J. Redmond, M. Ruggeri,
K. Jantze, C. Scalo, and C. L. Running, in AIAA SCITECH 2023 Forum (2023) p. 0096.
63
M. Ruggeri, V. C. B. Sousa, and C. Scalo, “Block Spectral Stresses (BSS) estimation for
shock-capturing and turbulent modeling,” arXiv (2024).
38
This is the author’s peer reviewed, accepted manuscript. However, the online version of record will be different from this version once it has been copyedited and typeset.
PLEASE CITE THIS ARTICLE AS DOI: 10.1063/5.0206359
... Therefore, many scholars have developed high-order methods in recent years and verified their performance through TGV flow. For example, weighted essentially nonoscillatory (WENO) schemes [7,19,23,33], multi-resolution WENO (MR-WENO) schemes [35,37], Targeted ENO (TENO) schemes [7,19], discontinuous Galerkin methods [7,11,20], spectral method [7,33], flux reconstruction methods [5,10], gas-kinetic scheme [34], two-stage fourth-order subcell finite volume method [45], weighted compact nonlinear scheme [47], etc. Among them, the finite difference WENO-type scheme is more popular due to its computational efficiency and stability. ...
... Therefore, many scholars have developed high-order methods in recent years and verified their performance through TGV flow. For example, weighted essentially nonoscillatory (WENO) schemes [7,19,23,33], multi-resolution WENO (MR-WENO) schemes [35,37], Targeted ENO (TENO) schemes [7,19], discontinuous Galerkin methods [7,11,20], spectral method [7,33], flux reconstruction methods [5,10], gas-kinetic scheme [34], two-stage fourth-order subcell finite volume method [45], weighted compact nonlinear scheme [47], etc. Among them, the finite difference WENO-type scheme is more popular due to its computational efficiency and stability. ...
... Therefore, many scholars have developed high-order methods in recent years and verified their performance through TGV flow. For example, weighted essentially nonoscillatory (WENO) schemes [7,19,23,33], multi-resolution WENO (MR-WENO) schemes [35,37], Targeted ENO (TENO) schemes [7,19], discontinuous Galerkin methods [7,11,20], spectral method [7,33], flux reconstruction methods [5,10], gas-kinetic scheme [34], two-stage fourth-order subcell finite volume method [45], weighted compact nonlinear scheme [47], etc. Among them, the finite difference WENO-type scheme is more popular due to its computational efficiency and stability. ...
Article
Full-text available
The coexistence and interaction of shock waves and turbulence occur in various applications, such as inertial confinement fusion, scramjet propulsion, and supernova explosions. The supersonic Taylor–Green vortex (TGV) flow is a benchmark example for studying shock/turbulence interaction problems. In this paper, the performance of the interpolation-based weighted essentially non-oscillatory (WENO) schemes for compressible TGV simulation was evaluated, rather than the existing reconstruction-based approach. First, based on the popular unequal-sized WENO (US-WENO) scheme, we developed an interpolation-based US-WENO scheme for simulating three-dimensional inviscid/viscous TGV problems over the Mach number range of 0.1–2.5. Second, a discontinuous sensor based on extremum properties (EPs) of the polynomial was designed, and a corresponding hybrid interpolation-based US-WENO scheme was developed. This EP-based sensor does not contain empirical parameters and can simulate the supersonic TGV problems well while significantly improving the computational efficiency of the original US-WENO scheme. Numerical experiments show that the interpolation-based WENO scheme has smaller numerical dissipation and better performance for compressible TGV problems than the reconstruction-based WENO scheme. However, its computational cost is slightly higher, while the hybrid US-WENO scheme can perform better in terms of both computational accuracy and efficiency.
... To demonstrate the accuracy of PyFR v2.0.3 for high-speed flows, we consider a supersonic Taylor-Green vortex test case at a Mach number of 1.25 and a Reynolds number of 1, 600, which was studied in Lusher and Sandham [65] and used to benchmark the accuracy and shock-resolving capabilities of several solvers in Chapelier et al. [66]. Specifically, we solve the compressible Navier-Stokes equations in a domain − ≤ , , ≤ , subject to the following initial conditions, ...
... where 0 and the reference dynamic viscosity were selected to achieve the desired Mach and Reynolds numbers based on length, velocity and density scales of unity, and a dynamic viscosity computed using Sutherland's law with a reference temperature of 273K [67]. For comparison with the results in Chapelier et al. [66], the simulations were performed using computational meshes consisting of 16 3 , 32 3 , 64 3 and 128 3 hexahedral elements, with a third-order polynomials approximating the solution within each element (corresponding to 64 3 , 128 3 , 256 3 and 512 3 DoFs, respectively). Gauss-Legendre-Lobatto flux and solution points were used, and entropy filtering was employed as a shock capturing approach. ...
... as a function of time, as well as a Schlieren-type representation of the density gradient norm at = 6 for a case computed with = 128 3 hexahedral elements (512 3 DoFs). The predicted enstrophy profiles are found to be in [69] Nodal Discontinuous Galerkin 4 Subcell finite volume SD3D [70] Spectral difference 4 Artificial viscosity OpenSBLI [71] Finite difference 6 Targeted ENO (TENO) excellent agreement with the reference data of Chapelier et al. [66], computed using a highly resolved (2048 3 DoFs) high-order finite difference targeted ENO (TENO) scheme, indicating that PyFR can accurately resolve small-scale turbulent flow structures in supersonic flows. This test case allows for further comparison against the results of various solvers presented in Chapelier et al. [66]. ...
Preprint
Full-text available
PyFR is an open-source cross-platform computational fluid dynamics framework based on the high-order Flux Reconstruction approach, specifically designed for undertaking high-accuracy scale-resolving simulations in the vicinity of complex engineering geometries. Since the initial release of PyFR v0.1.0 in 2013, a range of new capabilities have been added to the framework, with a view to enabling industrial adoption of the capability. This paper provides details of those enhancements as released in PyFR v2.0.3, explains efforts to grow an engaged developer and user community, and provides latest performance and scaling results on up to 1024 AMD Instinct MI250X accelerators of Frontier at ORNL (each with two GCDs), and up to 2048 NVIDIA GH200 GPUs on Alps at CSCS.
... More recently, the TGV case was extended to the compressible regime by increasing the Mach number of the initial flow field [46,43]. A common choice is Ma 0 = 1.25, for which complex shock patterns emerge that interact with the turbulent flow. ...
... The permitted maximum of the blending parameter α is set identically across all investigated resolutions. The results reported by Chapelier et al. [43] serve as the reference solution. The results in Fig. 10 (right) again show that GALAEXI and FLEXI yield identical results for the temporal evolution of the solenoidal dissipation rate. ...
... Temporal evolution of the solenoidal dissipation rate ε S for the incompressible TGV at Ma 0 = 0.1 (left) and the compressible TGV at Ma 0 = 1.25 (right) using between 64 and 512 DOF per spatial direction with a polynomial degree N = 7. The results by DeBonis[42] (left) and Chapelier et al.[43] (left) serve as the reference solution for the incompressible and compressible case, respectively. The results of the CPU implementation are given for reference. ...
Preprint
Full-text available
This work presents GALÆXI as a novel, energy-efficient flow solver for the simulation of compressible flows on unstructured meshes leveraging the parallel computing power of modern Graphics Processing Units (GPUs). GALÆXI implements the high-order Discontinuous Galerkin Spectral Element Method (DGSEM) using shock capturing with a finite-volume subcell approach to ensure the stability of the high-order scheme near shocks. This work provides details on the general code design, the parallelization strategy, and the implementation approach for the compute kernels with a focus on the element local mappings between volume and surface data due to the unstructured mesh. The scheme is implemented using a pure distributed memory parallelization based on a domain decomposition, where each GPU handles a distinct region of the computational domain. On each GPU, the computations are assigned to different compute streams which allows to antedate the computation of quantities required for communication while performing local computations from other streams to hide the communication latency. This parallelization strategy allows for maximizing the use of available computational resources. This results in excellent strong scaling properties of GALÆXI up to 1024 GPUs if each GPU is assigned a minimum of one million degrees of freedom. To verify its implementation, a convergence study is performed that recovers the theoretical order of convergence of the implemented numerical schemes. Moreover, the solver is validated using both the incompressible and compressible formulation of the Taylor-Green-Vortex at a Mach number of 0.1 and 1.25, respectively. A mesh convergence study shows that the results converge to the high-fidelity reference solution and that the results match the original CPU implementation. Finally, GALÆXI is applied to a large-scale wall-resolved large eddy simulation of a linear cascade of the NASA Rotor 37. Here, the supersonic region and shocks at the leading edge are captured accurately and robustly by the implemented shock-capturing approach. It is demonstrated that GALÆXI requires less than half of the energy to carry out this simulation in comparison to the reference CPU implementation. This renders GALÆXI as a potent tool for accurate and efficient simulations of compressible flows in the realm of exascale computing and the associated new HPC architectures.
... More recently, the TGV case was extended to the compressible regime by increasing the Mach number of the initial flow field [59,58]. A common choice is Ma 0 = 1.25, for which complex shock patterns emerge that interact with the turbulent flow. ...
... The permitted maximum of the blending parameter is set identically across all investigated resolutions. The results reported by Chapelier et al. [58] serve as the reference solution. The results in Fig. 10 (right) again show that GALAEXI and FLEXI yield identical results for the temporal evolu-tion of the solenoidal dissipation rate. ...
Article
Full-text available
This work presents GALÆXI as a novel, energy-efficient flow solver for the simulation of compressible flows on unstructured hexahedral meshes leveraging the parallel computing power of modern Graphics Processing Units (GPUs). GALÆXI implements the high-order Discontinuous Galerkin Spectral Element Method (DGSEM) using shock capturing with a finite-volume subcell approach to ensure the stability of the high-order scheme near shocks. This work provides details on the general code design, the parallelization strategy, and the implementation approach for the compute kernels with a focus on the element local mappings between volume and surface data due to the unstructured mesh. The scheme is implemented using a pure distributed memory parallelization based on a domain decomposition, where each GPU handles a distinct region of the computational domain. On each GPU, the computations are assigned to different compute streams which allows to antedate the computation of quantities required for communication while performing local computations from other streams to hide the communication latency. This parallelization strategy allows for maximizing the use of available computational resources. This results in excellent strong scaling properties of GALÆXI up to 1024 GPUs if each GPU is assigned a minimum of one million degrees of freedom. To verify its implementation, a convergence study is performedthat recovers the theoretical order of convergence of the implemented numerical schemes. Moreover, the solver is validated using both the incompressible and compressible formulation of the Taylor–Green-Vortex at a Mach number of 0.1 and 1.25, respectively. A mesh convergence study shows that the results converge to the high-fidelity reference solution and that the results match the original CPU implementation. Finally, GALÆXI is applied to a large-scale wall-resolved large eddy simulation of a linear cascade of the NASA Rotor 37. Here, the supersonic region and shocks at the leading edge are captured accurately and robustly by the implemented shock-capturing approach. It is demonstrated that GALÆXI requires less than half of the energy to carry out this simulation in comparison to the reference CPU implementation. This renders GALÆXI as a potent tool for accurate and efficient simulations of compressible flows in the realm of exascale computing and the associated new HPC architectures.
... These includes the Discontinuous Galerkin (DG) [4,5,6], Flux Reconstruction (FR) [7] and Spectral Difference (SD) [8,9,10] methods. All of these have shown promising results in a large variety of fluid dynamics problems such as compressible flows [11,12,13,14,15], combustion [16,17,18] and multiphase flows [19,20,21,22,23]. ...
Preprint
In this paper, we introduce a data-driven filter to analyze the relationship between Implicit Large-Eddy Simulations (ILES) and Direct Numerical Simulations (DNS) in the context of the Spectral Difference method. The proposed filter is constructed from a linear combination of sharp-modal filters where the weights are given by a convolutional neural network trained to replicate ILES results from filtered DNS data. In order to preserve the compactness of the discretization, the filter is local in time and acts at the elementary cell level. The neural network is trained on the data generated from the Taylor-Green Vortex test-case at Re=1600. In order to mitigate the temporal effects and highlight the influence of the spatial discretization, the ILES are periodically restarted from DNS data for different time windows. Smaller time windows result in higher cross-correlations between ILES and the filtered DNS snapshots using the data-driven filters. The modal decay of the filter for the smallest time window considered aligns with classical eigenanalysis, showing better energy conservation for higher orders of approximation. Similarly, an analysis of the filter's kernel in the Fourier space confirms that higher polynomial orders are less dissipative compared to lower orders. As large time windows are considered, the trained filter encounters difficulties in representing the data due to significant non linear effects. Additionally, the impact of the data-driven filter on the resolved kinetic energy has been assessed through the evaluation of the sub-grid production term which results in both direct and inverse cascades with the former being more likely on average. The presence of backscatter suggests that ILES based on Discontinuous Spectral Element Methods might be equipped with an intrinsic mechanisms to transfer energy in both directions with a predominance of direct kinetic energy cascade.
... The OPS library is embedded in C/C++ code, enabling massively-parallel execution of the code on a variety of high-performance-computing architectures via source-to-source translation, including GPUs. The OpenSBLI code was recently cross-validated against six other independently-developed flow solvers using a range of different numerical methodologies in [52] for a supersonic Taylor-Green vortex case involving shock-waves and transition to turbulence. ...
Preprint
Full-text available
Recent research into buffet in the transonic flow regime has been focused on a limited number of proprietary airfoil geometries and has mainly considered parametric variations in Mach number and angle of attack. In contrast, relatively little is known about the sensitivity of buffet frequencies and amplitudes to geometric properties of airfoils. In the present contribution, an airfoil geometry construction method based on a small number of parameters is developed. The resulting airfoils and computational grids are high-order continuous everywhere except at the trailing edge corners. The effects of four key geometric parameters, defined by local extrema of coordinates on the airfoil at the design condition and denoted as 'crest' points, are studied using large-eddy simulation, considering both free-transitional and tripped boundary layers. For both states of the boundary layer, buffet amplitude and frequency are found to be highly sensitive to the axial and vertical position of the suction-side crest point, while the vertical crest position on the pressure side affects the mean lift. The present work confirms that buffet can appear for free-transitional (laminar buffet) and tripped conditions (turbulent buffet) with similar sensitivities. For test cases near onset conditions intermediate-frequency phenomena have been observed, which were linked to unsteadiness of separation bubbles. Frequencies scaling based on mean-flow properties of the separation bubble were shown to be in good agreement with previous findings on different airfoil geometries in Zauner et al. (Flow Turb. & Comb., Vol. 110, 2023, pp. 1023-1057). Airfoil geometries are provided as open source: https://doi.org/10.5281/zenodo.12204411
... In the present work, we propose to address the problem of shock/turbulence interaction for DG methods by using a combination of numerical ingredients that allow for an accurate and robust representation of such problems: a positivity-preserving limiter that enables robust DG simulations by correcting locally the solution when negative density or pressure values are appearing; a localized diffusivity approach based on a modal shock sensor that allows for eliminating oscillations around discontinuities; a turbulence sensor that removes excesses in artificial dissipation levels when the vorticity magnitude locally dominates the velocity divergence. This strategy is thoroughly tested from the supersonic Taylor-Green vortex (TGV) flow problem [9,10]. This setting is of great interest for testing both the shock capturing properties of numerical schemes and their ability to handle shock/turbulence interactions, as the early flow regime features strong shocks separated from turbulent structures and the later regimes display interactions of turbulent scales with a system of multiple physical discontinuities. ...
Article
Full-text available
This paper presents a thorough comparison between RANS simulations performed with the computational fluid dynamics by ONERA, DLR, and Airbus (CODA) new-generation flow solver and reference legacy codes TAU (DLR) and elsA (ONERA) for high-speed cruising NASA Common Research Model (CRM) configurations that were considered in the context of the 5th, 6th, and 7th Drag Prediction Workshops. The solver’s accuracy is assessed with several meshing strategies, including block-structured, hybrid-structured/unstructured, and fully unstructured tetrahedral meshes. This solver features both a cell-centered finite-volume (FV) scheme suited to arbitrary meshes as well as a modern high-order discontinuous Galerkin (DG) scheme. We show that for all cases considered, the FV component recovers an equivalent accuracy compared to elsA (cell-centered FV) on block-structured meshes and TAU (node-centered FV) on hybrid and unstructured meshes. The high-order DG scheme (third-order accurate) is found to significantly enhance the drag prediction on coarse meshes compared to legacy FV methods, both for structured and unstructured meshes.
Article
Full-text available
We present an hp-adaptive discretization for a sharp interface model with a level-set ghost-fluid method to simulate compressible multiphase flows. The scheme applies an efficient p-adaptive discontinuous Galerkin (DG) operator in regions of smooth flow. Shocks and the phase interface are captured by a Finite Volume (FV) scheme on a h-refined element-local sub-grid. The resulting hp-adaptive scheme thus combines both the high order accuracy of the DG method and the robustness of the FV scheme by using p-adaptation in smooth areas and h-refinement at discontinuities, respectively. For the level-set based interface tracking, a similar hybrid DG/FV operator is employed. Both p-refinement and FV shock and interface capturing are performed at runtime and controlled by an indicator, which is based on the modal decay of the solution polynomials. In parallel simulations, the hp-adaptive discretization together with the costly interface tracking algorithm cause a significant imbalance in the processor workloads. To ensure parallel efficiency, we propose a dynamic load balancing scheme that determines the workload distribution by element-local wall time measurements and redistributes elements along a space filling curve. The parallelization strategy is supported by strong scaling tests using up to 8192 cores. The framework is applied to established benchmarks problems for inviscid, compressible multiphase flows. The results demonstrate that the hybrid adaptive discretization can efficiently and accurately handle complex multiphase flow problems involving pronounced interface deformations and merging interface contours.
Article
Full-text available
Unsteady surface pressure measurements have been carried out on a flat-plate test article in the Air Force Research Laboratory’s Mach-6 Ludwieg Tube at a negative angle of attack using PCB pressure sensors. The acoustic waves of the second-mode boundary-layer instability mechanism were successfully measured as indicated by the fluctuating surface pressure spectra. The effectiveness of a novel porous material (Silicon-Carbide (SiC) foam) at absorbing these fluctuations was investigated by comparing porous-wall and impermeable-wall spectra. The impermeable-wall spectra indicate second-mode peaks in amplitude, which agree with companion computations, while the equivalent porous material showed no signs of the instability in the spectra therefore exhibiting a strong dampening ability. However, the porous-wall spectra demonstrate similar breakdown to turbulence to the impermeable material at higher length Reynolds numbers, which might be attributed to surface-roughness effects. Acoustic-absorption bench tests also indicated high levels of ultrasonic absorption for the SiC foam—the acoustic absorption coefficient was consistently higher than 0.90 for second-mode relevant frequencies. A band-limited amplitude integration of the experimental PCB spectra indicate lower amplitude growth in the porous material compared to the impermeable material; however, beyond a length Reynolds number of ≈4·1064106\approx 4 \cdot 10^{6}, the band-limited amplitude of the porous material approach a similar value to the impermeable one. Linear-stability (LST) computations confirmed the experimentally observed frequencies only when supported by base-flow calculations capturing the leading-edge shock, which entails sudden thickening of the boundary layer at the tip not captured by the Blasius solution. Computed N factors for the porous material were approximately 30% of the associated impermeable values where the material inserts were located, downstream of which, they approached the same value. This study indicates that SiC foam has a strong ability to absorb acoustic waves characteristic of the hypersonic second-mode boundary-layer instability mechanism, and is therefore a candidate material for passive boundary-layer control.
Article
Full-text available
For compressible flow simulations involving both turbulence and shockwaves, the competing requirements render it challenging to develop high-order numerical methods capable of capturing the discontinuities sharply and resolving the turbulence with high spectral resolution. In particular when deployed with the advanced large-eddy simulation (LES) approach, for which the governing equations are solved with coarse meshes, the solution is extraordinarily sensitive to the numerical dissipation resulting in large uncertainties for cross-code comparisons. Similar sensitivities have also been observed for a wide range of complex fluid predictions, e.g., turbulent reacting flows, two-phase flows, and transitional flows. In this paper, the family of high-order targeted essentially non-oscillatory (TENO) schemes on both the Cartesian and unstructured meshes is reviewed for general hyperbolic conservation laws with an emphasis on the high-speed turbulent flows. As a novel variant of popular weighted ENO (WENO) scheme, the TENO scheme retains the sharp shock-capturing capability of WENO and is suitable for resolving turbulence with controllable low numerical dissipation. The key success of TENO relies on a strong scale-separation procedure and the tailored novel ENO-like stencil selection strategy. In addition, the built-in candidate stencils with incremental width facilitate the construction of arbitrarily high-order (both odd- and even-order) schemes featuring superior robustness. Detailed performance comparisons between the WENO and TENO schemes are discussed comprehensively as well as the applications of TENO schemes to challenging compressible fluids. At last, the potential future developments to further boost the performance of TENO schemes from various perspectives are highlighted.
Article
Full-text available
In this work, we present a novel hybrid Discontinuous Galerkin scheme with hp-adaptivity capabilities for the compressible Euler equations. In smooth regions, an efficient and accurate discretization is achieved via local p-adaptation. At strong discontinuities and shocks, a finite volume scheme on an h-refined element-local subgrid gives robustness. Thus, we obtain a hp-adaptive scheme that exploits both the high convergence rate and efficiency of a p-adaptive high order scheme as well as the stable and accurate shock capturing abilities of a low order finite volume scheme, but avoids the inherent resolution loss through h-refinement. A single a priori indicator, based on the modal decay of the local polynomial solution representation, is used to distinguish between discontinuous and smooth regions and control the p-refinement. Our method is implemented as an extension to the open source software FLEXI. Hence, the efficient implementation of the method for high performance computers was an important criterion during the development. The efficiency of our adaptive scheme is demonstrated for a variety of test cases, where results are compared against non adaptive simulations. Our findings suggest that the proposed adaptive method produces comparable or even better results with significantly less computational costs.
Article
Full-text available
In this paper, we present h- and hp-adaptative strategies suited for the discontinuous Galerkin formulation of the compressible laminar and Reynolds-averaged Navier–Stokes equations on unstructured grids, relying on a metric-based simplicial remeshing approach. An a posteriori error estimator, combining the measure of the energy associated with the highest-order modes and the inter-element jumps, is used to build both the metric field and the polynomial degree distribution map. The choice of refining either in h or p is driven by a smoothness indicator based on the decay of modal coefficients in each element. The performance of the developed adaptation algorithms is assessed for the 2D laminar viscous flow past a NACA0012 airfoil, and for the 3D laminar viscous flows past a sphere and past a delta wing. The adaptive hp-strategy is applied to a 3D turbulent jet issued from a nozzle. Finally, the gain in accuracy provided by the adaptive algorithms with respect to uniformly refined simulations, for a given number of degrees of freedom with polynomial degrees p=1,2,3, is demonstrated.
Article
Full-text available
A high-order low dissipative numerical framework is discussed to tackle simultaneously the modeling of unresolved sub-grid scale flow turbulence and the capturing of shock waves. The flows around two different airfoil profiles are simulated using a Spectral Difference discretisation scheme. First, a transitional, almost incompressible, low Reynolds number flow over a Selig-Donovan 7003 airfoil. Second, a high Reynolds number flow over a RAE2822 airfoil under transonic conditions. These flows feature both laminar and turbulent flow physics and are thus particularly challenging for turbulence sub-grid scale modeling. The accuracy of the recently developed Spectral Element Dynamic Model, specifically capable of detecting spatial under-resolution in high-order flow simulations, is evaluated. Concerning the test in transonic conditions, the additional complexity due to the presence of shock waves has been handled using an artificial viscosity shock-capturing technique based on bulk viscosity. To mitigate the impact of the shock-capturing on turbulence dissipation, it was necessary to combine the high-order modal-type shock detection with a usual sensor measuring the local flow compressibility.
Article
The excessive computational resource requirements for direct numerical simulation (DNS) and wall-resolved large eddy simulation (LES) make these methods intractable for simulations concerning turbulent flows over complex, real-world geometries. Wall-modeled LES (WMLES) approaches seek to reduce this cost by applying wall-layer models to the inner-most region of turbulent boundary layers. Immersed boundary methods (IBMs) have grown in popularity over the past few decades since the mesh can be automatically generated for any complex geometry - a potentially major advancement for common industrial CFD simulations. However, immersed boundary approaches suffer from reduced numerical accuracy and, possibly, stability at the wall, primarily attributable to irregular discretization of boundary operators. This is a problem that is especially significant for WMLES of high Mach number flows, where numerical schemes have the conflicting requirements to be both stable at high wavenumbers and non-dissipative. This presentation introduces a coupled IBM-WMLES approach that is tailored to the simulation of high-speed turbulent flows. The basis of this method is to carefully address numerical dissipation and stability away from and close to the wall. This is achieved by using a hybrid central-upwind flux reconstruction scheme and careful application of smooth boundary conditions. The presented test cases evaluate the current scheme for a wide range of physical flow phenomena, and include a turbulent channel for basic validation, a hypersonic transitional boundary layer to investigate high-speed zero-pressure-gradient flows, a hypersonic shock wave boundary layer interaction to investigate adverse pressure gradients, separation, and turbulent heating, and a hypersonic compression ramp which is significantly challenging as it combines all of the features from the aforementioned high-speed flow problems.
Article
OpenSBLI is an open-source code-generation system for compressible fluid dynamics (CFD) on heterogeneous computing architectures. Written in Python, OpenSBLI is an explicit high-order finite-difference solver on structured curvilinear meshes. Shock-capturing is performed by a choice of high-order Weighted Essentially Non-Oscillatory (WENO) or Targeted Essentially Non-Oscillatory (TENO) schemes. OpenSBLI generates a complete CFD solver in the Oxford Parallel Structured (OPS) domain specific language. The OPS library is embedded in C code, enabling massively-parallel execution of the code on a variety of high-performance-computing architectures, including GPUs. The present paper presents a code base that has been completely rewritten from the earlier proof of concept (Jacobs et al, JoCS 18 (2017), 12-23), allowing shock capturing, coordinate transformations for complex geometries, and a wide range of boundary conditions, including solid walls with and without heat transfer. A suite of validation and verification cases are presented, plus demonstration of a large-scale Direct Numerical Simulation (DNS) of a transitional Shockwave Boundary Layer Interaction (SBLI). The code is shown to have good weak and strong scaling on multi-GPU clusters. We demonstrate that code-generation and domain specific languages are suitable for performing efficient large-scale simulations of complex fluid flows on emerging computing architectures. Program summary Program Title: OpenSBLI code-generation framework for compressible fluid dynamics on heterogeneous architectures CPC Library link to program files: https://doi.org/10.17632/3sdb6hck2c.1 Licensing provisions: GPLv3 Programming languages: Python, C/C++, OPS DSL Nature of problem: The compressible 3D Navier-Stokes equations are solved via Implicit Large Eddy Simulation (ILES) or Direct Numerical Simulation (DNS). Solution method: OpenSBLI [1,2] is a Python-based code-generation system that uses symbolic algebra to generate a complete CFD solver in C/C++. The basic algorithm is a stencil-based finite-difference solver on structured curvilinear meshes. Shock-capturing is performed by a selection of high-order Weighted/Targeted Essentially Non-Oscillatory (WENO/TENO) schemes. Explicit low-storage Runge-Kutta schemes are used for time-advancement. Additional comments including restrictions and unusual features: The generated code is compliant with the Oxford Parallel Structured (OPS) [3] software library. OpenSBLI/OPS executables can be generated for the OpenMP, MPI, CUDA, OpenCL, and OpenACC parallel programming paradigms. Multi-GPU support is available via combinations of MPI with CUDA, OpenCL or OpenACC. References [1]D.J. Lusher, S.P. Jammy, N.D. Sandham, Shock-wave/boundary-layer interactions in the automatic source-code generation framework OpenSBLI, Computers & Fluids 173 (2018) 17-21. [2]C.T. Jacobs, S.P. Jammy, N.D. Sandham, OpenSBLI:A framework for the automated derivation and parallel execution of finite difference solvers on a range of computer architectures, Journal of Computational Science 18 (2017) 12-23. [3]I.Z. Reguly, G.R. Mudalige, M.B. Giles, D. Curran and S. McIntosh-Smith, The OPS Domain Specific Abstraction for Multi-Block Structured Grid Computations. In proceedings of the 4th international workshop on Domain-Specific Languages and High-Level Frameworks for High Performance Computing(WOLFHPC ‘14).Held in conjunction with IEEE/ACM Supercomputing 2014(SC’14).