An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method

Full text

Preprint manuscript No.
(will be inserted by the editor)
An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid
Method
Pascal Mossier a,∗·Daniel Appela·Andrea
Beck a·Claus-Dieter Munza
Received: date / Accepted: date
Abstract We present an hp-adaptive discretization for a sharp interface model with a
level-set ghost fluid method to simulate compressible multiphase flows. The scheme
applies an efficient p-adaptive Discontinuous Galerkin (DG) operator in regions of
smooth flow. Shocks and the phase interface are captured by a Finite Volume (FV)
scheme on a h-refined element-local sub-grid. The resulting hp-adaptive scheme thus
combines both the high order accuracy of the DG method and the robustness of the
FV scheme by using p-adaptation in smooth areas and h-refinement at discontinu-
ities, respectively. For the level-set based interface tracking, a similar hybrid DG/FV
operator is employed. Both p-refinement and FV shock and interface capturing are
performed at runtime and controlled by an indicator, which is based on the modal de-
cay of the solution polynomials. In parallel simulations, the hp-adaptive discretization
together with the costly interface tracking algorithm cause a significant imbalance in
the processor workloads. To ensure parallel efficiency, we propose a dynamic load
balancing scheme that determines the workload distribution by element-local wall
time measurements and redistributes elements along a space filling curve. The par-
allelization strategy is supported by strong scaling tests using up to 8192 cores. The
framework is applied to established benchmarks problems for inviscid, compress-
ible multiphase flows. The results demonstrate that the hybrid adaptive discretization
can efficiently and accurately handle complex multiphase flow problems involving
pronounced interface deformations and merging interface contours.
Keywords Discontinuous Galerkin ·High order ·p-Adaptivity ·Shock capturing ·
Finite Volumen ·Level-set ·Multiphase ·Ghost Fluid ·Dynamic load balancing
∗Corresponding author
Pascal Mossier
Pfaffenwaldring 21
70569 Stuttgart
Deutschland
Tel.: +49 711 685-63421
E-mail: pascal.mossier@iag.uni-stuttgart.de
aInstitute of Aerodynamics and Gas Dynamics, University of Stuttgart, Stuttgart, Germany

2 Pascal Mossier et al.
1 Introduction
Compressible multiphase flows are encountered in many technical applications like
fuel injection systems of rocket or jet engines. Due to their multi-scale nature, the
accurate and efficient simulation of such flows poses a challenging task and is still
an active field of research. For the modeling and computation of multiphase flows,
two main approaches can be distinguished in literature: the sharp interface method
(SIM) and the diffuse interface method (DIM). In the DIM, the phase interface is
modelled as a smooth transition layer between two fluids, by introducing a mixture of
the phases. Consequently, the DIM requires a high resolution of the numerical approx-
imation near the smoothed interface to preserve the physical properties. Well-known
implementations of this ansatz comprise the Navier-Stokes-Korteweg equations [1]
and the Baer-Nunziato equations [3]. The SIM, by contrast, models the interface as
a discontinuity separating the bulk phases. This approach is advantageous when the
width of physical interface is infinitesimal with respect to the macroscopic flow scale.
The SIM consists of two major building blocks: An algorithm to track the phase
interface and a physically consistent coupling of the bulk phases.
In this work, we use a SIM with a level-set method [56] to track the phase interface and
a ghost fluid method [22] to provide boundary conditions at the phase interface. The
required ghost states are obtained from a multiphase Riemann solver following the idea
of Merkle and Rohde [41]. The simulation of compressible multiphase flows with the
SIM poses demanding requirements on a numerical scheme. In smooth regions inside
the bulk phases, a high order scheme is beneficial for its accuracy and efficiency. At the
phase interface, a robust, non-oscillatory scheme is required to cope with the strong
discontinuity caused by the sharp interface. Furthermore, for compressible flows, the
discretization needs to handle potential shocks in the bulk phase. There exist a mul-
titude of numerical schemes in literature to cope with the above requirements. From
the class of Finite Volume schemes (FV), the Weighted Essentially Non-Oscillatory
(WENO) [61,13] and the Central Weighted Essentialy Non-Oscillatory (CWENO)
[12] schemes have been successfully applied to compressible multiphase flows e.g. by
Hu et al. [30] and Tsoutsanis et al. [60]. An alternative is the Discontinuous Galerkin
(DG) Method that has gained popularity due to its efficiency and accuracy [8,26]. To
avoid spurious oscillations related to the Gibb’s instabilities at discontinuous solution
features, different stabilization and shock capturing techniques have been developed.
One approach is to smooth out the oscillations by a local switch-on of artificial vis-
cosity [46,49,65]. A different method is to combine the DG method with a WENO
approach to avoid instabilities at shocks [15].
In this work, we follow the idea of Huerta et al. [31] and Persson et al. [50] to intro-
duce piecewise constant ansatz functions and apply them on a sub-cell grid inside the
original DG element. The approach presented in this paper is based on the work of
Sonntag and Munz [53] who introduced an a priori FV sub-cell limiting scheme. If a
shock or strong gradient are observed inside a DG element, the element is subdivided
into sub-cells and discretized by a second-order FV scheme. In a recent publication
[44], we extended this hybrid DG/FV approach by introducing a p-adaptive DG oper-
ator and allowing for a FV sub-cell resolution independent of the element-local DG
degree N. The resulting scheme combines the high order accuracy of the DG scheme

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 3
with the robustness of the FV method by applying p-adaptivity in smooth regions and
a h-refined sub-cell grid to improve shock localization. In this paper, we introduce
this adaptive hybrid scheme [44] to our sharp interface framework [33,45, 20].
Our sharp interface framework solves a level-set transport equation to compute the po-
sition and geometry of the phase interfaces. To provide accurate geometric properties,
high order accuracy is required at the phase interface. However, in case of merging
phases, the level-set field may exhibit discontinuities and kinks. Therefore we employ
a hybrid p-adaptive DG scheme combined with FV sub-cell limiting in analogy to
the proposed fluid discretization. The sub-cell limiting and the p-refinement of both
the fluid and level-set discretization are controlled at runtime by means of an a priori
indicator based on the modal decay of the solution polynomials [39,44].
The adaptive discretization and the interface tracking algorithm cause an inhomoge-
neous distribution of the element-local computational costs, which implies a workload
imbalance across the processor units in parallel simulations. Therefore, a dynamic load
balancing (DLB) is essential to ensure parallel efficiency of the code. Appel et al.
[2] introduced a DLB scheme based on repartitioning along a space filling curve to
treat the imbalance due to the interface tracking. In the present study, we extend this
algorithm to cover imbalances caused by the adaptive discretization and propose an
efficient communication strategy to speedup the data redistribution during rebalanc-
ing.
The paper is organized as follows: In section 2 we introduce the governing equations
for the bulk flow and the interface tracking algorithm. Section 3 provides the numeri-
cal framework to solve the conservation equations and the level-set interface tracking
with an adaptive hybrid DG/FV scheme. The parallelization strategy with the focus
on an efficient dynamic load balancing approach is addressed in section 4. Finally, we
apply the code framework to well-known shock-droplet and shock-bubble interactions
in section 5. We assess the efficiency and accuracy of our scheme by comparing with
numerical and experimental results from literature. We close with a conclusion and a
summary of our findings.
2 Governing Equations
In this work, we investigate compressible multiphase flows with two pure, immiscible
phases separated by a sharp interface of zero thickness. Therefore, we consider a
computational domain Ω, consisting of a liquid part Ωland a vapor part Ωv, separated
by the phase interface Γon a finite time interval [0,T]. The bulk fluid behavior is
governed by the compressible Euler equations
∂Q
∂t
+∇x·F(Q)=0in Ω× [0,T](1)
with the vector of conservative variables Q=[ρ, ρu,E]|and the convective flux
vector F=[ρu, ρu⊗u+pI,u(E+p)]|given in terms of the density ρ, the velocity
vector u=(u1,u2,u3)|, the pressure pand the total energy E. The total energy Eis
defined as the sum of the internal energy and the kinetic energy
E=ρe=ρ +1
2ρu·u.(2)

4 Pascal Mossier et al.
The equation system is closed with an equation of state (EOS), that provides expres-
sions for the pressure pand the specific internal energy :
p=p(ρ, ),  =(ρ, p).(3)
Both algebraic and multi-parameter EOS are implemented in our code framework and
can be efficiently evaluated by the tabulation method introduced by Föll et al. [23].
Here, we model the liquid phase by a stiffend gas EOS and apply the perfect gas law
to model the gaseous phase. Following Le Métayer et al. [43], the stiffened gas EOS
is given as
p=(γ−1)ρ −γp∞(4)
with the ratio of specific heats γand the reference pressure p∞. For p∞=0, the
stiffened gas equation reduces to the perfect gas law.
A necessary building block for the SIM is the tracking algorithm for the phase
interface Γ. Following Sussman et al. [56], we obtain the position of the phase interface
as the root of a level-set function Φthat is advected by a velocity field s. The level-set
transport equation is defined as
∂Φ
∂t
+s· ∇xΦ=0.(5)
An essential feature of the level-set function is the signed distance property. It is
necessary for the accurate derivation of geometric properties such as the unit normal
vector nls and the interface curvature κls. Since the level-set transport equation (5)
does not preserve the signed distance property, a reinitialization procedure is required
to regularize the level-set field. Following Sussman et al. [56], this is accomplished
by solving a Hamilton-Jacobi equation in pseudo time τ:
∂Φ
∂τ +sign (Φ) ( |∇xΦ| − 1)=0(6)
Fast marching [52] or fast sweeping [59] methods could be used as an alternative
here. According to [6], geometric properties of the phase boundary can be expressed
in terms of derivatives of the level-set field, with the normal vector nls given as
nls :=∇xΦ
|∇xΦ|(7)
and the curvature κls computed by
κls :=
Φ2
x1Φx2x2+2Φx1Φx2Φx1x2+Φ2
x2Φx1x1
|∇xΦ|3+
Φ2
x1Φx3x3+2Φx1Φx3Φx1x3+Φ2
x3Φx1x1
|∇xΦ|3+
Φ2
x2Φx3x3+2Φx2Φx3Φx2x3+Φ2
x3Φx2x2
|∇xΦ|3
(8)
As a final building block a velocity field shas to be provided for the level-set transport
equation (5). The velocity sat the phase boundary results from the two-phase Riemann

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 5
problem [41,19,21]. To obtain a velocity field in the volume, the velocities computed
by the Riemann solver are first copied to neighboring cells. These pointwise data
are then extrapolated into the volume by solving in pseudo time a Hamilton-Jacobi
equation for each velocity component si[48]
∂si
∂τ +sign(Φ)nls · ∇xsi=0,with i=1, ..., d(9)
To save computational cost, the level-set transport equation (5), the reinitialization
(6) and the velocity extrapolation (9) are only evaluated in a narrow band around the
phase boundary.
3 Numerical Methods
In this section, we first provide the numerical methods used to approximate the govern-
ing equations of the bulk flow and the level-set interface tracking algorithm. We then
assemble the building blocks to obtain a hp-adaptive level-set ghost fluid framework.
All equations are discretized on a computational domain Ω⊂R3that is subdivided
into K∈Nnon overlapping hexahedral elements Ωesuch that Ω=ÐK
e=1Ωeand
ÑK
e=1Ωe=∅.
We apply a hybrid discretization based on a p-adaptive Discontinuous Galerkin spec-
tral element method (DGSEM) with a FV sub-cell shock capturing scheme for both the
Euler equations (1) and the level-set transport equation (5). The hybrid discretization
is illustrated in figure 1 where a single droplet and the underlying mesh are visualized.
Ωv
Ωl
DG Element NDG
FV Element NFV
Phase boundary
Surrogate phase boundary
Fig. 1: Spatial discretization of a droplet in a vapor environment. Elements in the
bulk region containing a smooth solution are discretized by DG elements with an
element-local degree NDG. Elements containing the phase boundary are subdivided
into NFV equidistant sub-cells per direction to refine the approximation of the phase
boundary by the surrogate surface.
The phase interface is always discretized by FV sub-cells to avoid spurious oscil-
lations due to the discontinuity and to refine the resolution of the surrogate surface.
The surrogate surface is aligned with the FV sub-cell grid and serves as a discrete ap-
proximation of the physical phase boundary. For a detailed derivation of the DGSEM

6 Pascal Mossier et al.
and FV sub-cell methods, the reader is referred to [36,37,53]. The hp-adaptivity is
covered in more depth in [44], while the foundations of the multiphase framework can
be found in [33,45, 20].
3.1 Spatial discretization
3.1.1 DGSEM with p-adaptation
In this section we apply a p-adaptive formulation of the DGSEM in combination
with a FV sub-cell scheme to both the hyperbolic conservation law of the bulk flow
(1) and the level-set transport equation (5), which is a hyperbolic equation with a
non-conservative product.
As a first step, we introduce the mapping from the physical space x=(x1,x2,x3)|
to reference space ξ=(ξ1, ξ2, ξ3)|to transform the physical elements Ωeto a unit
reference element E=[−1,1]3. We thus obtain the conservation law and the level-set
transport equation in reference space as
Jgeo ∂Q
∂t
+∇ξ·F(Q)=0(10)
Jgeo ∂Φ
∂t
+s· ∇ξL=0(11)
with the Jacobi determinant of the mapping Jgeo, the contravariant flux Fand the
contravariant level-set field Lrespectively. Projection onto a space of test functions
ψ∈Pleads to the weak form of equations (10) and (11)
∫E
(Jgeo ∂Q
∂t)ψdΩ+∮∂E
(F·nξ)ψdSξ−∫E
(F· ∇ξ)ψdΩ=0(12)
∫E
(Jgeo ∂Φ
∂t)ψdΩ+∮∂E
(s· ∇ξL)ψdSξ+∫E
(s· ∇ξL)ψdΩ=0(13)
The term F·nξdenotes the contravariant flux across a surface element nξand is
approximated by a numerical flux function (F·nξ)∗, provided either by a HLLC
[58] or HLLE [17] approximate Riemann solver in the present study. To evaluate the
surface integral of the level-set transport equation, a path-conservative jump term
(D·nξ)∗:=s· ∇ξLis introduced. Following [5,14,33], it can be approximated by
a Rusanov-type Riemann solver as
(D·nξ)∗≈1
4(s++s−) − 2smaxΦ+−Φ−(14)
with the maximum signal speed smax :=max |sL·nξ|,|sR·nξ|. The transport
velocities s±are determined from sLand sRin an upwinding procedure to allow for
topological changes like merging interface contours. A detailed explanation of this
procedure is given in [33,64]. The solution vectors Qand Φas well as the contravariant

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 7
flux Fand contravariant level-set Lare approximated as piecewise polynomials
Q(ξ,t) ≈
N
Õ
i,j,k=0
ˆ
Qi jk (t)ζi jk (ξ),Φ(ξ,t) ≈
N
Õ
i,j,k=0
ˆ
Φi jk (t)ζi jk (ξ)(15)
F(ξ,t) ≈
N
Õ
i,j,k=0
Fi jk (ˆ
Qi jk )ζi jk (ξ),L(ξ,t) ≈
N
Õ
i,j,k=0
ˆ
Li jk (t)ζi jk (ξ)(16)
in the space spanned by tensor products of one-dimensional Lagrange polynomials
ζij k (ξ)=`i(ξ1)`j(ξ2)`k(ξ3).(17)
According to the Galerkin ansatz, we use the tensor product Lagrange polynomials
for both the ansatz functions ζand the test functions ψ.
A semi-discrete scheme is eventually obtained by introducing a numeric quadrature
for the integrals in equations (12) and (13). A central idea of the DGSEM is to use the
same node distribution, in our case Legendre-Gauss nodes, for both the quadrature and
the interpolation. This collocation approach results in a tensor product structure of the
DG operator and reduces the number of required operations per degree of freedom
(DOF). A multi-dimensional operator can thus be constructed as a succession of
multiple one-dimensional operations.
The extension to a p-adaptive DGSEM discretization is straightforward, by allow-
ing for a variable element-local degree N∈ [Nmin,Nmax]. The bounds Nmin and Nmax
for the allowed polynomial degree need to be set prior to each computation. In the
general case, the degrees of adjacent elements differ, M,Nwith M>N, such that
the flux computation needs to be adjusted. As depicted in figure 2, the surface solution
Q−
Nof the element with the lower degree Nis interpolated to the higher degree Q−
M.
With surface solutions Q−
Mand Q+
Mof common degree M, the numerical flux F∗
M
can be computed. For the element of degree N, the flux is subsequently projected to
the lower degree F∗
N.
Q−
N
Q−
N7→ Q−
M
F∗
M7→ F∗
N
0
ξ1
ξ2
Q−
MQ+
M
0
ξ1
ξ2
Fig. 2: Flux calculation for two adjacent elements with degree N=3(left) and M=4
(right). The node distribution is indicated by dots in the volume and by squares on the
surface. The numerical flux is computed on the solution representation of the higher
polynomial degree.
We follow an analogous procedure for the level-set transport equation, using the
path-conservative jump term (D·nξ)∗instead of the numerical flux. When the local

8 Pascal Mossier et al.
degree of an element changes during runtime, the solution is either interpolated to a
higher degree or projected to a lower degree. For a more detailed derivation of the
p-adaptive scheme, we refer to [44].
3.1.2 h-refined FV scheme for DGSEM
In the presence of strong gradients or discontinuities within an element, high order
DGSEM schemes produce oscillatory solutions (Gibb’s phenomenon). For the Euler
equations, shock waves and phase boundaries are typical discontinuous solution fea-
tures that necessitate a stabilization technique. The level-set field can develop sharp
gradients for large curvatures, merging interface contours and the cut-off at the edge
of the narrow band. To stabilize the solution, we combine the DGSEM with a robust
FV scheme on an h-refined sub-cell grid. The FV scheme is formally obtained by
introducing piecewise constant test and basis functions and is thus equivalent to a first
order DGSEM. With piecewise constant basis functions, the derivative in the volume
integral vanishes, simplifying the weak formulations (12) and (13) to
∫e
(Jgeo ∂Q
∂t)ψdΩ+∮∂e
(F·nξ)ψdSξ=0(18)
∫e
(Jgeo ∂Φ
∂t)ψdΩ+∮∂e
(s· ∇ξL)ψdSξ=0(19)
The reduced approximation order is compensated by refining an affected DG element
Eto a sub-cell grid of NFV equidistant sub-cells eper direction. This improves the
localization of shocks and increases the resolution of the surrogate phase boundary
significantly. In the previous implementation of our framework [33], a DG operator
with constant degree Nand a sub-cell resolution NFV =N+1were used. With this
choice, both DG and FV elements use the same number of DOFs, which allows a
common data structure and thereby eases the implementation of the sub-cell approach.
Following Dumbser et al. [16], we choose an increased sub-cell resolution of NFV =
2Nmax +1. This particular choice is motivated by the time step restriction of the
DGSEM compared to the FV scheme. In case of an equidistant sub-cell grid, NFV =
2Nmax +1is the maximum sub-cell resolution that does not impose a more restrictive
time step compared to a DG element of degree Nmax. Thus, the increased sub-cell
resolution of the phase boundary can be achieved without compromising the time
step. The accuracy of the FV sub-cell scheme is further improved through a TVD
second-order reconstruction scheme, proposed by [54].
Switching between the DG and FV operators requires transformations between
piecewise polynomial and piecewise constant solution representations. The transfor-
mation DG FV can be performed in a conservative manner by integrating the DG
polynomial over each sub-cell e. Conversely, a polynomial representation FV DG
can be recovered from FV sub-cell data with a discrete projection, equivalent to a
least square approach with integral conservation as a constraint. This transformation
was proposed by Dumbser et al. [14] for the case NFV >N+1, which causes an
over-determined system. Switching between a DG and FV solution is performed as a
matrix vector operation with the transformation matrices VFV DG and VFV DG, for

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 9
which holds
VFV DG ◦VDG FV =I.(20)
Adjacent FV sub-cells are coupled via a numerical flux F∗. Following [54], we use
the FV sub-cell scheme as an a priori limiter to stabilize the bulk fluid flow. Thus,
every element is discretized either by the DG or the FV operator, according to an
indicator evaluated prior to each time step. In case of a mixed interface between a DG
and a FV element, as depicted in figure 3, a common solution representation needs
to be provided for the flux computation. To this end, the DG solution is transformed
to a FV sub-cell representation at the element side. With a consistent FV solution
representation from both sides, the numerical flux can be evaluated as usual and is
subsequently transformed back to a DG discretization for the DG element.
Q−
DG
Q−
DG 7→ Q−
FV
F∗
FV 7→ F∗
DG
0
ξ1
ξ2
Q−
FV Q+
FV
0
ξ1
ξ2
Fig. 3: Flux computation for a mixed interface with a DG element of degree N=3
(left) and a FV sub-cell element with NFV =4sub-cells (right). DG volume nodes
and FV sub-cell centers are indicated by dots. Surface nodes are indicated by squares.
The numerical flux is computed on a piecewise constant FV representation F∗
FV and
projected afterwards to a DG solution F∗
DG to provide the flux for the DG element.
For the level-set transport equation, the FV sub-cell scheme is applied as an a
posteriori limiter. We compute the time update for a pure DG representation as well
as a pure FV representation. The update candidate for each element is then chosen
according to an a priori indicator. This procedure is motivated by the occurrence
of spurious oscillations if adjacent DG and FV elements are coupled via the path-
conservative jump term [14]. It should be noted that evaluating both the DG and the
FV operator for each element has a minor effect on the overall computation time
since the level-set transport equation is only solved in a narrow band around the phase
interface.
3.2 Time Discretization
The Euler equations and the level-set transport are both integrated in time with an
explicit fourth-order low-storage Runge-Kutta scheme [35]. The time integration is
subject to time step restrictions, derived from the CFL condition by Courant, Friedrichs
and Lewy [9]. For the DGSEM and FV discretizations of the Euler equations and the

10 Pascal Mossier et al.
level-set transport, time step restrictions can be formulated as follows
∆tDG =CFL ·αRK(N) · ∆xDG
(2N+1) |λmax |(21)
∆tFV =CFL ·αRK(0) · ∆xFV
|λmax|.(22)
They depend on the maximum signal velocity |λmax|, the grid spacing ∆xDG or ∆xFV
and the empirical scaling factor αRK(N)>1for the Runge-Kutta scheme. The DG
time step ∆tDG depends furthermore on the element-local degree N. In case of the
Euler equations the maximum signal velocity is given by the maximum absolute
eigenvalue λmax =max(|c−u|,|u|,|c+u|) with the speed of sound c. For the level-set
transport, the maximum signal velocity corresponds to the transport velocity λmax =s.
In two-phase flows, an additional time step restriction is imposed by capillary waves
of the phase interface. According to Denner et al. [10], the dynamic capillary time
step constraint depends on the signal velocity
λmax =s2πσ
ρ∆xFV
+|s|(23)
with the surface tension σ. Instead of the sum of the liquid and the vapor densities, we
only use the density of the current degree of freedom to simplify the implementation.
Since the sum of liquid and vapor density is always larger than either the liquid or
the vapor density, this is a conservative assumption and thus does not compromise
numerical stability. The capillary time step criterion is only relevant at the phase
boundary that is discretized with the FV sub-cell scheme. Therefore, the signal velocity
is formulated for a FV sub-cell element with the grid spacing ∆xFV.
The maximum signal velocity of the level-set transport sis of the same order of
magnitude as the convective velocity of the bulk flow u. Thus, the signal speed of the
Euler equations c+uis usually larger and therefore the dominant time step restriction
in practical applications.
3.3 Indicator for FV sub-cell limiting and p-refinement
p-adaptivity and FV sub-cell limiting are controlled on an element-local level through
a combination of different indicators. First, we infer an estimate for the error and
the smoothness of the solution from the decay rate of the modal polynomial solution
representation [39,44]. To this end, we interpolate the nodal polynomial solution
representation ˆ
unod of the DGSEM scheme to a modal Legendre basis ˆ
umod using a
Vandermonde matrix VLeg
ˆ
umod =VLeg ˆ
unod.(24)
The relative contribution ωmof the mth mode in ξ1direction to the solution can be
expressed as
wm=
ÍN
j,k=0ˆ
u2
mod,ij k i=m
ÍN
i,j,k=0ˆ
u2
mod,ij k 
.(25)

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 11
The contributions ωmare subsequently fitted to the exponential function wm=ae−σm
for every direction and the indicator is obtained as
I
modal =min(|σ1|,|σ2|,|σ3|) (26)
The final indicator value I
modal is then compared against user-defined thresholds to
decided wether p-refinement or FV sub-cell limiting is required. For every degree N,
lower thresholds T
low(N)and uppers threshold T
up(N)are defined for both the DG
p-refinement and FV sub-cell limiting:
T
low,FV(N)=T
min,FV +NT
max,FV − T
min,FV
Nmax −Nmin (27)
T
up,FV(N)=T
min,FV +(N+1)T
max,FV − T
min,FV
Nmax −Nmin (28)
T
low,DG(N)=T
min,DG +NT
max,DG − T
min,DG
Nmax −Nmin (29)
T
up,DG(N)=T
min,DG +(N+1)T
max,DG − T
min,DG
Nmax −Nmin
.(30)
The thresholds depend on four free parameters T
min,FV,T
max,FV,T
min,DG and T
max,DG
that are tuned empirically for the bulk flow and the level-set transport in the test
problems of section 5. For a more in-depth discussion of the modal indicator and the
threshold definitions, the reader is referred to [44].
To stabilize both the bulk flow and the level-set transport in the presence of phase
boundaries, additional criteria for the FV sub-cell limiting are defined. Using the
position of the level-set zero-line, elements containing a phase boundary are detected
and flagged for FV sub-cell limiting in the Euler equations by the indicator I
interface.
In case the level-set function changes its sign multiple times within a DG element, we
assume approaching level-set contours and flag affected elements with the indicator
I
topo for FV sub-cell limiting [33,64].
In conclusion, p-refinement is controlled for the bulk flow and the level-set by the
indicator I
modal. FV sub-cell limiting depends on the combination of I
modal and I
interface
in the bulk, and on I
modal and I
topo for the level-set.
3.4 Discretization of the reinitialization and velocity extrapolation
The scalar Hamilton-Jacobi equations of the reinitialization (6) and the velocity ex-
trapolation (9) are discretized with a fifth-order WENO scheme [32] on the FV sub-cell
grid. They are integrated in pseudo time with an explicit Runge-Kutta scheme [62].
In case of the velocity extrapolation, scalar equations are solved for every space di-
mension. To obtain initial data for the velocity extrapolation, the velocities provided
by the two-phase Riemann solver at the surrogate phase boundary are copied to adja-
cent sub-cells. If a sub-cell is affected by multiple two-phase Riemann problems, an
averaging is performed. Form these pointwise data, the velocity extrapolation routine
constructs a velocity field in the narrow band region. For a detailed discussion of
reinitialization and velocity extrapolation, we refer the reader to [48,56, 18].

12 Pascal Mossier et al.
3.5 Calculation of level-set normals and curvature
The evaluation of the two-phase Riemann problem requires the normal vectors and
the curvature of the level-set field at the phase boundary. The normals (7) and the
curvature (8) are expressed in terms of first and second derivatives of the level-set
field. These derivatives are approximated by applying a central second-order finite
difference scheme to the level-set field in FV representation. The second derivatives
are obtained by reapplying the same operator to the first derivatives.
3.6 Assembly of the level-set ghost fluid framework
Finally, we can assemble the introduced building blocks to obtain an adaptive level-set
ghost fluid framework. Figure 4 illustrates the interaction of the different operators
for one time step [tn,tn+1]. Within one time step, the following steps are performed:
1. Based on the sign of the level-set scalar, the domain Ωis divided into Ωland Ωv.
2. At the phase boundary, DG elements are switched to the FV sub-cell represen-
tation to provide a stable discretization of the sharp interface and to improve the
approximation by the surrogate phase boundary.
3. The level-set field is switched to the FV representation for the subsequent WENO
and FV operators.
4. Before evaluating geometric properties of the level-set, a WENO-based reinitial-
ization of the level-set field is performed to restore the signed distance property.
5. Level-set normals and curvature are calculated with a second-order FV discretiza-
tion.
6. With the geometric properties of the level-set, a two-phase Riemann problem is
solved at the surrogate phase boundary. It provides ghost states to couple the bulk
phases as well as the local velocity of the phase interface.
7. The pointwise information of the interface velocity at the surrogate phase boundary
is copied to adjacent sub-cells and extrapolated with a WENO routine to obtain a
velocity field for the level-set transport.
8. Based on the indicators of section 3.3, the DG/FV distributions for the bulk flow
and level-set transport are determined. Furthermore, the polynomial degree for
DG elements is chosen based on the indicator I
modal.
9. Finally, the adaptive hybrid DG/FV operators are advanced for the bulk flow and
the level-set.
4 Parallelization concept
4.1 MPI communication
In general, the hp-adaptive scheme, employed for the bulk phase (see section 3.1), is
formulated for unstructured, curved grids, which has been demonstrated in [44]. While
the WENO scheme discretizing the Hamilton-Jacobi equations (section 3.4) restricts
the presented multiphase framework to Cartesian grids, it internally still operates on

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 13
Bulk fluid solver Interface tracking
Domain decomposition
in Ωland ΩvEvaluation of level-set root
Set FV sub-cells
at phase interface
Switch to FV
representation
Solve two-phase
Riemann problem
Reinitialization
WENO5
Interface geometry
computation
FD O2
Velocity
extrapolation
WENO5
Set local NDG and
FV sub-cell
hp-Refinement
Set local NDG and
FV sub-cell
Advance bulk flow
Hybrid DG/FV
Advance level-set
tn
tn+1
Fig. 4: Assembly of the level-set ghost fluid framework from the main building blocks
and chronological interaction within one time step. Operators applied in every Runge-
Kutta stage are colored light gray. Steps that are only performed once per time step are
highlighted in dark gray. The novelty is the hp-refinement step prior the application
of the spatial operators for the bulk flow and level-set transport.
an unstructured grid representation. The grid representation and the associated paral-
lelization concept inherit from the work by [29] and are optimized towards massively
parallelized computations, as shown in [37,4].
The parallelization concept exploits that the DG operator can be split into two build-
ing blocks, cf. Eqs. (12) and (13): the volume integral, which depends exclusively
on the element’s inner nodes, and the surface integral, which requires information of
the neighbored elements. In parallel simulations, this information exchange implies
a communication operation if the two elements sharing one side reside on differ-
ent partitions. The induced idle time can conveniently be hidden by computing the
element-local volume integral while exchanging the side data for the surface integral
through non-blocking communication operations. This latency hiding approach in-

14 Pascal Mossier et al.
creases parallel performance as long as the network communication is faster than the
memory buffer operations. The communication routines are provided by the widely
used MPI standard [42], which is commonly available on HPC systems in architecture-
optimized implementations.
Repeat for every discretization
D ∈ {DGNmin , .., DGNmax ,FV}
Count total number of sides
umaster ∈ D to be received
Count total number of sides
uslave ∈ D to be sent
Count number of sides
umaster ∈ D to be received by each proc
Determine number of sides
uslave ∈ D to be sent by each proc
Allocate array urecv to store all
elements of type Dto be received
Allocate array usend to store all
elements of type Dto be sent
Copy elements from uslave
of type Dto array usend
Receive array urecv via MPI
Send array usend via MPI
Copy elements from urecv
to array umaster of type D
loop
MPI communication
start exchange
finish exchange
Fig. 5: Flowchart of a master/slave side communication for the hp-adaptive data
structure.
Moreover, the latency hiding approach is applied to the surface integral itself.
To this end, the data structure separates volume and surface data, with the solution
urepresented at the corresponding interpolation nodes each. Every element face,
except for those at the domain boundaries, forms a pair (side) with the face of the
adjacent element. As the numerical flux evaluated from this data pair is unique, it
needs to be computed only once. We therefore label the element faces as master or
slave and evaluate the numerical flux solely on the master. If two elements share a
common side but reside on different partitions, the surface data of the slave need
to be sent to the master prior to the flux computation, and the resulting flux values
are sent back afterwards. Following the aforementioned latency hiding approach, this

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 15
communication pattern suggests to first calculate the flux on those MPI-sides and use
the idle time to calculate the flux on the processor-local inner sides.
In the present hp-adaptive framework, the recurring send and receive operations are
accomplished efficiently by collecting the surface data per polynomial degree Nin a
dedicated array urecv, as depicted in figure 5. Once the communication has finished,
urecv is unpacked by copying the received data back to the DG side array uslave. These
additional buffer operations account for the fact that the DG side arrays are only
sparsely filled in terms of one specific degree N. The same strategy is applied to the
computed fluxes. That is, transforming uslave to the common polynomial degree as
well as projecting the resulting flux back to N(cf. Sec. 3.1) are both performed locally
on the master side. Thus, only surface data of the actual degree Nare exchanged,
which minimizes the communication amount and further enhances the scalability of
the hp-adaptive scheme.
4.2 Domain decomposition
As indicated, the parallelization strategy of the present framework relies on splitting
the computational domain into separate subdomains (partitions) of one or more grid
cells, which are each assigned to one processor unit. For the given, unstructured mesh
representation, Hindenlang [29] suggested the use of space-filling curves (SFCs)
for the domain decomposition. Compared to graph partitioning approaches, they
were reported to provide a similar surface-to-volume ratio, which correlates with the
communication amount per data operation, while greatly simplifying data structures
(cf. Sec. 4.1), parallel I/O, and dynamic load balancing (Sec. 4.3). Specifically, we
employ the Hilbert SFC [28], which returned highly compact subdomains and thus
proved suited for massively parallel simulations.
4.3 Dynamic load balancing scheme
One of the major challenges to the efficient utilization of current and future HPC
systems is load balance, i.e. the even distribution of the application workload across
the processor units [38, 11]. Temporal changes in the workload at runtime, in particular,
necessitate a dynamic load balancing (DLB) approach.
In the presented hp-adaptive multiphase framework, two main factors contribute to a
workload imbalance: First, the element-local choice between a p-adaptive DGSEM
discretization and the FV sub-cell scheme, and second, the narrow band approach of
the interface tracking algorithm [2], which confines the costly solution of the level-set
equations to a small band of 2-3 elements adjacent to the surrogate phase boundary.
Since both the element-local discretization and the position of the phase interface may
change over time, the processor workload needs to be balanced dynamically.
In this work, we extend and improve the DLB scheme of [2] to fit our adaptive
discretization. It relies on a domain decomposition through SFCs, which conveniently
reduce the partitioning problem to one dimension. The partitioning problem seeks to
distribute Kelements with the weights wi,i=1, ..., Kamong Pavailable processors,

16 Pascal Mossier et al.
such that the maximum processor load, also referred to as bottleneck B, is minimized:
B=max
1≤k≤P{Lk},with Lk=
sk
Õ
i=sk−1+1
wi.(31)
Here skdenotes the index of the last element associated with the processor k, and Lkthe
total processor workload. The separator indices s0=0≤s1≤... ≤sk... ≤sP=K
subdivide the domain of Kelements into Ppartitions of Nk=(sk−sk−1)elements.
The task of the partitioning algorithm is thus to find the sequence of separator indices
(s0, . . . , sP)with the lowest feasible bottleneck Bopt. The value of Bopt is unknown
and bounded below by the ideal bottleneck
B∗=1
P
N
Õ
i=1
wi(32)
which assumes identical loads among all partitions, i.e. Bopt ≥B∗.
The partitioning problem, commonly known as chains-on-chains partitioning problem
[51], has been addressed by a variety of algorithms. Algorithms that always return
an optimal partitioning are called exact, opposed to heuristics, which, however, typ-
ically execute faster and can be parallelized. An extensive overview of both, exact
methods and heuristics is given in [51]. In our work, we adopt the bisection algo-
rithm "ExactBS+PI" recently proposed by [38]. Bisection algorithms in general use
binary search to find a good bottleneck. They repeatedly call a probing function, that
checks a given bottleneck value for feasibility, by adjusting the provided bottleneck
iteratively through bisection. ExactBS+PI specifically employs an improved probing
function and a more accurate initial search interval for the bisection. It thus features
quick execution times while remaining exact. For details, the reader is referred to the
original work, where ExactBS+PI is embedded in a hierarchical method to design a
highly scalable, near-exact partitioning algorithm.
A crucial building block of our load balancing scheme is the determination of the
weights wi. Following [2], we perform element-local wall time measurements. The
code instrumentation exploits that the narrow band approach decomposes the set D
which contains all elements in Ωinto three intersecting subsets: The bulk elements
Dbulk ≡ D, where only the Euler equations (1) are solved, the elements inside the
narrow band DNB ⊂ Dbulk where the interface tracking is required additionally, and
the elements directly at the interface DΓ⊂ DNB, where the two-phase Riemann
problem is evaluated. Evidently, the computational cost of an individual grid element
rises in the listed order of these subsets.
In [2], the measured time was distributed evenly among the elements within each sub-
set, which directly defined the element weight wi. This averaging ansatz was plausible
since all elements used the same discretization and could therefore be expected to
contribute equally to the cost of the considered subset.
To account for the variable element cost in the bulk due to the hp-adaptivity, the cost
share of the individual discretization schemes needs to be determined. To this end,
a calibration run at the beginning of the simulation calls the bulk operator once for
every degree Nand once for a pure FV sub-cell discretization in order to measure the

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 17
average wall times per element τNmin, τNmin+1, . . . , τNmax, τ NFV.
At runtime, the measured time τbulk of subset Dbulk is distributed among the elements
according to the relative cost of the different discretizations. This relative cost is
evaluated by first calculating the expected wall time τcal of each subdomain based on
the calibrated values τi∈ {τNmin, . . . , τ NFV }
τcal =
sk
Õ
i=sk−1+1
τi.(33)
This expected wall time, together with the measured wall time τbulk, defines a dimen-
sionless factor χ,
χ=τbulk
τcal ,(34)
which allows to infer the weight wiof each bulk element from scaling the calibrated
weight
wi=τi·χ. (35)
For now, the imbalance due to p-refinement in the discretization of the level-set
advection (5) is neglected since here both the FV and DG operator are evaluated for
every element. Thus, the variable degree Nhas only a minor effect on the computation
time per element. In the sub-sets DNB and DΓ, the average costs of the additional
interface-related operations are added to wi.
With the element-local weights known, the current workload imbalance
I=B
B∗−1∈ [0,∞) (36)
can be evaluated. It assumes zero for identical workload on all processors, I(B=
B∗)=0, and triggers a repartitionig upon exceeding a user-defined threshold. To
redistribute the elements between the processors, we first store the current solution of
the bulk flow and the level-set field. Then, the new partitioning is computed globally,
allowing each processor to determine which elements it needs to send and receive
by comparing its old partition to the new one. During the communication, each
processor has to recompute the mesh metrics for the new elements, thus hiding most
of the communication time. Once the communication is completed, the computation
resumes. This repartitioning procedure is summarized in figure 6.
4.4 Parallel performance analysis
In this section, we investigate the influence of the hp-adaptive discretization on the
strong scaling behavior. We choose a setup similar to the one used by Appel et. al in
[2], where the parallel performance of the previous, non-adaptive code version was
analyzed. All computations are run on the HPE Apollo System Hawk at the High
Performance Computing Center Stuttgart (HLRS). For the scaling tests, we used up
to 64 compute nodes, each with two 64-core AMD EPYC 7742 CPUs and 256 GB
RAM. The test case consists of a resting droplet in a three-dimensional domain Ω,

18 Pascal Mossier et al.
Bulk fluid solver Interface tracking
Bulk flow operator Level-set operator
Measured relative
DG(N)/FV weights Measure imbalance
Save old mesh partition
Compute rebalanced
mesh partition
Start MPI
communication
Initialize program
Resume computation
Finish MPI
communication
Start DLB
Finish DLB
Fig. 6: Overview of the dynamic load balancing scheme.
discretized by 30 ×30 ×30 elements. To study the effects of the adaptive scheme on
the performance, we enforced a checkerboard-like distribution of the local polynomial
degree. For the bulk, the degree varies between N∈ [2,4]and for the level-set, the
degree changes between N∈ [2,8]. FV elements deploy NFV =9sub-cells per spatial
direction and the standard smoothness indicator presented in section 3.3 is used. Fig.
7 visualizes the discretization for a slice through the z=0plane.
(a) Bulk discretization (b) Level-set discretization
Fig. 7: Setup for the strong scaling test sliced at z=0. The local polynomial degree
Nand the elements discretized by FV sub-cells are indicated with different colors.

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 19
It should be noted that the checkerboard-like discretization is the most expensive
scenario for an adaptive scheme, since all element interfaces are of mixed type.
To isolate the effects of the repartitioning and the novel hp-adaptive data structure
on the parallel performance, we perform a single load balancing step after 5 time
steps, followed by further 44 time steps. During these 44 time steps, we measure the
performance index PID, which expresses the wall time required by a single processor
to advance one DOF for one stage of the Runge-Kutta time integration scheme
PID =#total wall time ·#cores P
#DOFs ·#time steps ·#RK-stages .(37)
The second metric to analyze the parallel performance is the parallel efficiency η
which relates the current PID to the PID obtained with one node (P=128)
η=PIDP=128
PID (38)
We compute the setup with a range of 1,2,4, . . ., 64 compute nodes corresponding to
P=128,256, . . . , 8192 cores and supply the results in figure 8.
124
8163264
#compute nodes
103104
2
4
6
8
·10−4
#DOFs/processor
PID
(a) Performance index
124
8163264
#compute nodes
103104
0.4
0.6
0.8
1
#DOFs/processor
η
(b) Parallel efficiency
1 2 4 8 16 32 64
#compute nodes
102103104
100
101
102
#processors
speedup
(c) Parallel speedup
Fig. 8: Strong scaling behavior for the setup in figure 7.
Up to 32 nodes, a parallel efficiency of about 50% is achieved. At 64 nodes, a sharp
decrease of parallel efficiency can be observed. It can be attributed to the limit of the
load balancing capabilities when the cost of the most expensive elements approaches
the target weight of a single processor unit, B∗. There are two main reasons for
the decreased parallel efficiency: First, for processors containing very few elements,
the latency hiding of the communication becomes less efficient and, secondly, the
partitioning of loads among processors becomes less accurate, i.e. Bopt B∗.
It should be noted, that the overhead due to the load balancing is not included in
the measured PID. For the present setup, one repartitioning and data redistribution
together have a maximum cost equivalent to 17.2time steps at 16 nodes and a minimum
cost of 1.4time steps at 64 nodes. The overhead due to the time measurements for the
weight computation can be neglected.

20 Pascal Mossier et al.
5 Numerical Results
To assess the performance of the presented adaptive hybrid DG/FV sharp interface
framework, we simulate a wide range of benchmark problems. The considered test
cases are compressible, inviscid two-phase flows with the gas modeled as an ideal
gas and the liquid modeled by the stiffened gas equation of state. For all setups,
a hybrid discretization is used with a p-adaptive DG scheme with N∈ [2,4]and
a FV resolution of NFV =9sub-cells per direction in the bulk flow. If not stated
otherwise, the level-set transport is discretized with DG elements of N∈ [2,8]and a
FV resolution of NFV =9. The computations were again run on the Hawk system and
used up to 32 nodes.
5.1 1D Gas-liquid Riemann problem
In a first step, the novel scheme is applied to the well-known one-dimensional gas-
liquid shock tube problem of Cocchi et al. [7]. It considers two initially constant states
inside a computational domain Ω=[−1,1]. The left side of the shock tube, x<0,
contains highly compressed air and the right side liquid at ambient pressure. The
initial conditions are provided in non-dimensionalized form in table 1.
ρu1pγp∞
Compressed air 1.241 0.0 2.753 1.4 0.0
Water 0.991 0.0 3.059·10−45.5 1.505
Table 1: Initial conditions of the gas-liquid shock tube problem in non-dimensionalized
form, following Cocchi et al. [7].
−1−0.5 0 0.5 1
0.9
1
1.1
1.2
1.3
x
ρ
Exact DGSEM FV
−1−0.5 0 0.5 1
0
0.2
0.4
x
u
−1−0.5 0 0.5 1
0
1
2
x
p
Fig. 9: Density, velocity and pressure distribution of the gas-liquid Riemann problem
at the final time t=0.2. The exact solution is compared against a numerical solution
computed with a grid of 40 elements. Smooth regions are discretized by a DG scheme
(blue) with N∈ [2,4], whereas a FV scheme (red) with NFV =9sub-cells per element
is applied at discontinuities.

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 21
The computational domain Ωis discretized by 40 elements. The simulation runs
until the final time t=0.2. In figure 9, the numerical results of our approach are
compared against the reference solution, which relies on the exact Riemann solver
devised by Cocchi et al. [7] for this setup. As expected, the FV sub-cell scheme is only
applied in regions of sharp gradients, that is at the right-moving shock in the liquid
phase, the material interface between the phases and the left-moving rarefaction wave
in the gas. Since the other regions contain nearly constant states, p-refinement is not
necessary in those smooth regions and the DG discretization always uses the lowest
possible degree N=2. Despite the coarse discretization, the numerical resolution
is in good agreement with the exact solution and only a minor undershoot can be
observed at the material interface.
5.2 2D Air-helium shock-bubble interaction
As a first two-dimensional test case, we consider the interaction of a Ma =6shock
wave in air with a helium cylinder [30,27]. The setup involves in severe deformations
of the phase interface during the breakup of the helium bubble and is thus a challenging
benchmark for a sharp interface framework. At time t=0 s, the computational domain
Ω=[0.0,0.356]m×[−0.0445,0.0445]mcontains a shock at x1=0.1 m and a helium
cylinder of radius r=0.025 m is located at x1=0.15 m. The setup is sketched in
figure 10 and the initial conditions for the three constant regions Ωgsbefore the shock,
Ωgafter the shock and Ωgbinside the helium bubble are provided in table 2.
ΩgsΩg
Ωgb
Fig. 10: Initial setup for the two-dimensional air-helium shock-bubble interaction.
ρ[kg m−3]u1[m s−1]p[MPa]γ
Air (pre-shock) 1.20 0.0 0.101 1.4
Air (post-shock) 6.34 1669.0 4.239 1.4
Helium 0.166 0.0 0.101 1.66
Table 2: Initial conditions of the two-dimensional air-helium shock-bubble interaction.

22 Pascal Mossier et al.
For the discretization of the physical domain Ω, a resolution of 512 ×64 elements
is chosen. Since the phase interface is always discretized by FV sub-cells, this results
in an effective resolution of 647 DOFs per bubble diameter. Due to the symmetry of
the setup, we only compute half of the domain Ωand impose symmetry boundary
conditions at x2=0.0 m. For the remaining boundaries, we impose inflow bound-
ary conditions on the left, non-reflecting boundary conditions on the right and wall
boundary conditions on the top side. In the bulk flow, an HLLE approximate Riemann
solver is used. The setup is computed until the final time t=1·10−4s. The current
workload imbalance Iis evaluated every 20 time steps and triggers a repartitioning if
I>1.0. The simulation was computed on 4nodes on Hawk and consumed a total of
710 CPU hours.
(a) t∗=0.6(b) t∗=1.2
(c) t∗=1.8(d) t∗=2.4
Fig. 11: Density plots of the Ma =6air-helium shock-bubble interaction at the time
instances t∗=0.6,t∗=1.2,t∗=1.8and t∗=2.4.
In figure 11, the density fields at the non-dimensional times t∗=(tu1)/(2r)=
{0.6,1.2,1.8,2.4}are visualized. Our results are in good agreement with those by
Han et al. in [27]. An initial deformation followed by a roll-up of the bubble can be
observed. Complex shock patterns develop and vortical structures evolve at entropy

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 23
shear layers. Finally, the separation of helium droplets from the initial bubble and the
enclosure of air filaments can be seen.
The local polynomial degree Nof DG elements and areas, where the FV sub-cell
limiting is applied, are visualized at t∗=2.4in figure 12 for both the bulk flow
and the level-set field. In the bulk flow, shocks and the phase interface are detected
correctly and discretized by the FV scheme on a refined sub-cell grid. An increased
polynomial degree is applied around vortical structures. To improve the accuracy of
the geometry computation, elements containing the level-set root are discretized with
the highest possible degree for the level-set transport equation. FV sub-cells are used
mainly at the level-set cut off and at sharp kinks of the level-set contour or at merging
phase boundaries. All in all, the p-refinement and the FV sub-cell limiting were
adaptively controlled as expected, resulting in an accurate and stable representation
of shocks and the phase interface as well as a high resolution of vortical structures.
The proposed sharp interface framework thereby demonstrates its ability to cope with
complex flow patterns and pronounced interface deformations.
(a) Temperature field and bulk discretization (b) Level-set field and discretization
Fig. 12: Discretization of the bulk flow (left) and the level-set field (right) for the
Ma =6air-helium shock-bubble interaction at t∗=2.4. The element-local degree N
of DG elements is indicated by color, whereas FV sub-cell elements are marked in
gray. Detail plots highlight the refined resolution of the FV sub-cell grid compared to
the underlying DG mesh.

24 Pascal Mossier et al.
5.3 Bubble collapse in water
As second two-dimensional second test case, we study the interaction of a planar
Ma =1.72 shock wave in water with a gas bubble. During the shock-induced col-
lapse of the gas bubble, a shock wave and a high speed liquid jet are emitted. This
mechanism is relevant for hydraulic pumps and turbo machinery, fuel injectors, naval
propulsion systems and medical applications like lithotripsy. Since the setup involves
strong pressure gradients, high speed fluid dynamics and a complex deformation of
the phase interface, it has been frequently used to assess the efficiency of numerical
schemes [25,47,60,27].
We consider the computational domain Ω=[0.0,0.024]m× [−0.012,0.012]mini-
tialized at time t=0.0 s with a planar shock in the liquid at x1=0.0066 m and a gas
bubble of radius of r=0.003 m at x1=0.012 m. The setup is sketched in figure 13,
with the pre and post shock states as well as the state of the bubble provided in table
3.
ΩlsΩl
Ωg
Fig. 13: Initial setup of the two-dimensional shock-induced bubble collapse.
ρ[kg m−3]u1[m s−1]p[MPa]γp∞[MPa]
Water (pre shock) 1000.0 0.0 1.0 4.4 6000.0
Water (post shock) 1323.65 681.058 19000.0 4.4 6000.0
Air 1.0 0.0 1.0 1.4 0.0
Table 3: Initial conditions of the two-dimensional shock-induced bubble collapse.
Exploiting the symmetry of the setup, we consider only half of the domain by
imposing symmetry boundary conditions along the x2=0.0 m axis. Inflow boundary
conditions are applied on left side and the remaining sides are treated as non-reflecting
boundaries. The computational domain Ωis discretized by 360 ×360 elements,
resulting in an effective resolution of 810 FV sub-cells per bubble diameter. For the
bulk flow, we employ an HLLC approximate Riemann solver. The computation is run
until a final time t=4.5µs, using the same load balancing parameters as above. On
2nodes of the Hawk cluster, 102 CPU hours were spent on this test case. Figure 14

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 25
visualizes the non-dimensionalized density and pressure distributions ρ∗=ρ/ρ0and
p∗=p/p0, with reference density ρ0and pressure p0set to the pre-shock state of the
water.
(a) t=1.9µs(b) t=3.0µs
(c) t=3.7µs(d) t=3.8µs
(e) t=4.0µs(f) t=4.2µs
Fig. 14: Non-dimensionalized density and pressure fields of the shock-induced bubble
collapse at various time instances.

26 Pascal Mossier et al.
The simulation reproduces essential flow phenomena like the reflected rarefaction
wave at t=1.9µs, the kidney shaped deformation of the droplet at t=3.7µs, the
high speed water jet piercing the bubble at t=3.8µsand the shock wave resulting
form the bubble collapse at t=4.0µs. Additionally, very delicate flow features like
secondary jets, that further divide the remnants of the initial bubble, can be observed at
t=4.0µsand t=4.2µs. Our numerical results agree very well with the simulations
of Nourgaliev et al. [47]. To provide a qualitative comparison with literature, we
evaluate the pressure field along the x2=0line in figure 15. Until the final simulation
time of t=4.5µs, a maximum pressure peak of 5.1 GPa can be observed which is in
good agreement with the results reported by Tsoutsanis et al. [60].
0.6 0.811.2 1.4 1.6 1.8 2 2.2 2.4
·10−2
0
1
2
3
4
5
x1[cm]
p[GPa]
t= 1.9µ
t= 3.0µ
t= 3.7µ
t= 3.8µ
t= 4.0µ
t= 4.2µ
t= 4.5µ
Fig. 15: Pressure distribution along the x2=0line for various time instances.
The adaptive discretization is analyzed for the time instance t=3.8µsin figure
16. In the bulk flow, shocks and the phase interface are discretized by the FV sub-cell
scheme, while accoustic waves as well as the direct vicinity of shocks are captured with
an increased polynomial degree. Since the gas bubble is displaced by the significantly
heavier water at high speed, the level-set transport velocity is in the order of magnitude
of the incident shock wave. Therefore, the level-set field is only p-refined up to
polynomial degree N=4to avoid a dominant time step constraint through to the
level-set transport equation. The maximum degree is again applied at the level-set
root to improve the accuracy of the geometry computation. The FV sub-cell scheme
of the level-set field does not only capture the cut-off region at the narrow band edge,
but also the tips of the separated bubbles due to their curvature.
5.4 2D Shock-droplet interaction
In this section, we study the interaction of a planar shock wave with a water column,
using the setups investigated by Xiang and Wang [24] and Tsoutsanis et al. [60].

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 27
(a) Temperature field and bulk discretization (b) Level-set field and discretization
Fig. 16: Discretization of the bulk flow (left) and the level-set field (right) for the
shock-induced bubble collapse at t=3.8µs. The element-local degree Nis indicated
by color, whereas FV sub-cell elements are marked in gray. Detail plots highlight the
refined resolution of the FV sub-cell grid compared to the underlying DG mesh.
We consider two configurations: a regular liquid column and a liquid column with an
enclosed cavity. Both configurations are computed with a Weber number of We =1000
to analyze the breakup in the shear-induced entrainment (SIE) regime [57].
5.4.1 Water column
First, we consider the interaction of a Ma =2.4shock wave in gas with a pure liquid
column. The domain Ω=[0.0,0.06]m× [−0.02,0.02]mcontains a planar shock
at x1=0.0074 m and a water column of radius r=0.0048 m at x1=0.015 m
at the initial time t=0.0 s. The setup is illustrated in figure 17, with the initial
conditions provided by table 4. The surface tension is set to σ=11.90 N/mto obtain
We =1000. The computational domain Ωis discretized by 360 ×240 elements,
ρ[kg m−3]u1[m s−1]p[MPa]γp∞[GPa]
Air (pre-shock) 1.20 0.0 1.01 1.4 0.0
Air (post-shock) 3.85 567.3 6.12 1.4 0.0
Water 1000 0.0 1.03479 6.12 0.343
Table 4: Initial conditions of the 2D shock-droplet interaction without cavity.
which corresponds to a FV sub-cell resolution of 518 DOFs per droplet diameter.
We again impose symmetry boundary conditions along the x1=0axis, inflow

28 Pascal Mossier et al.
ΩgsΩg
Ωl
Fig. 17: Initial setup of the 2D shock-droplet interaction without cavity.
boundaries to the left side and non-reflecting boundaries at the right and top. For the
flux computation in the bulk, an approximate HLLC Riemann solver is chosen. The
setup is advanced until a final computation time of t=2.0·10−4s. The workload
distribution is evaluated every 50 time steps, with the imbalance threshold again set
to I>1. The computational cost on 4nodes of the Hawk system amounts to 727
CPU hours for the described setup. Figure 18 supplies the numerical results at the
non-dimensionalized time instances t∗=(tu1)/(2r)={0.8,1.62,5.5,11.8}. Schlieren
plots visualize the vortical structures and shocks, while the pressure field is given in
non-dimensionalized form p∗=p/p0, with p0being the pre shock pressure in the
gas at t=0.0. In particular, the wave pattern at time t∗=0.8has been thoroughly
(a) t∗=0.8(b) t∗=1.62
(c) t∗=5.5(d) t∗=11.8
Fig. 18: Schlieren images and pressure fields of the 2D Ma =2.4shock-droplet
interaction for a Weber number of We =1000 (SIE regime).

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 29
studied in literature, e.g. Meng and Colonius [40] and Xiang and Wang [24], and the
reported numerical results are supported by experimental investigations of Sembian et
al. [55]. Our computation reproduces all relevant flow features: the reflected incident
shock wave, the weak transmitted shock wave, the reflected rarefaction inside the
water column and the Mach stem and slip surfaces.
To support our results quantitatively, we compare the pressure distribution along the
x2=0axis at time t∗=0.8with the results of Xiang and Wang [24] in figure 19. A
near perfect agreement can be observed.
0 0.511.5 2 2.5 3
−1.5
−1
−0.5
0
0.5
1
1.5
2
x1[cm]
p[MPa]
Adaptive DG/FV
Xiang and Wang
Fig. 19: Pressure distribution along the symmetry line x2=0of the shock-droplet
interaction without cavity and We =1000 at t∗=0.8. The results are compared
against the data from Xiang and Wang [24].
During the later stages of the simulation, the water column is flattened and complex
recirculation zones and vortex patterns develop in the wake. With We =1000, the
droplet breakup occurs in the SIE regime and the formation of filaments, that are
stripped from the main liquid body can be observed. The final column shape at
t∗=11.8agrees well with the findings of Xiang and Wang [24] and Tsoutsanis et al.
[60].
Computations with the previous code framework, published by Jöns et al. [33],
applied a constant number of DOFs per element. To demonstrate the efficiency of the
novel adaptive scheme, we compare the adaptive computation with N∈ [2,4]and
NFV =9against non-adaptive computations with N=4and NFV =5. Table 5 lists
the average number of DOFs and the computation time for all setups.
Setup 1 uses the same mesh as the adaptive computation, which results in a lower
effective number of DOFs per droplet diameter due to the coarser FV sub-cell grid. To
achieve the same resolution at the phase interface as the adaptive scheme, setup 2 uses
a finer mesh. In figure 20, the temperature fields and element-local discretizations are
compared for the three configurations.

30 Pascal Mossier et al.
DOFs per droplet diam. Discretization Average DOFs Wall time [CPU h]
Setup 1 288 N=4,NFV =5 1.08 ·106288
Setup 2 518 N=4,NFV =5 3.50 ·1061561
Setup 3 518 N∈ [2,4],NFV =9 0.60 ·106727
Table 5: Efficiency comparison of the non-adaptive discretization (setups 1 and 2)
with the proposed adaptive scheme (setup 3).
(a) Setup 1: non-adaptive with
288 DOFs per droplet diameter
(b) Setup 2: non-adaptive with
518 DOFs per droplet diameter
(c) Setup 3: adaptive with 518 DOFs per
droplet diameter
Fig. 20: Temperature field (top) and bulk discretization (bottom) of the 2D shock-
droplet interaction with We =1000 at t∗=11.8, comparing a static scheme with
N=4and NFV =5(left, middle) to the proposed adaptive scheme with N∈ [2,4]
and NFV =9(right). The element-local degree Nis indicated by color, whereas FV
sub-cell elements are marked in gray.
Shocks and the phase interface are detected well for all setups and discretized by
FV sub-cells. Setups 2 and 3 both achieve the same FV resolution at shocks and the
phase interface. Therefore, shocks appear sharper and more intricate phase interface
geometries can be captured compared to setup 1. In setup 3, p-adaptivity allows to
apply a reduced degree of N=2in most of the domain, while intricate vortical
structures in the wake are discretized with the highest possible degree. Since setup 2
has a finer DG mesh, slightly more detailed vortical structures in the recirculation zone
can be observed. Overall however, setups 2 and 3 produce comparable results. This is
significant, since the adaptive scheme required on average 5.8times less DOFs than
setup 2. Compared to setup 1, the adaptive scheme still requires 1.8times less DOFs,
even though a higher resolution is achieved at shocks and the phase boundary. Due
to the increased number of DOFs in the expensive narrow band, setup 3 requires 727
CPU hours and is thus more expensive than setup 1 with 240 CPU hours. However,
compared with setup 2, requiring 1561 CPU hours, the adaptive scheme achieves a
similar result in less than half the wall time.
To study the evolution of the adaptive discretization over time, the average number
of DOFs per element and the relative amount of FV sub-cell is plotted in figure 21
for setup 3. Starting from a pure FV sub-cell discretization to avoid oscillatory initial
states, the amount of required FV sub-cells increases gradually from a minimum of
1.7% to a maximum of 7.1% at the end of the computation. This increase is mainly

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 31
to be attributed to the shock patterns forming in the wake of the droplet and the
increased surface of the deformed water column. Likewise, a steady increase in the
average number of DOFs per element from 10.3to 16.1can be observed. This can
be explained by the increased number of FV sub-cells and by the p-refinement of
structures in the wake of the column.
0 0.511.5 2
·10−2
0
2
4
6
8
10
12
14
16
t[s]
I
0 0.511.5 2
·10−4
6
8
10
12
14
16
18
20
t[s]
Average NDOF/element
NDOF over time
1
2
3
4
5
6
7
8
FV elements [%]
FV elements over time
Fig. 21: Evolution of the load imbalance (left), the average number of DOFs per
element and the amount of FV sub-cell elements (right) during the simulation time.
In order to assess the scalability of the framework in practical applications, we
finally investigate its parallel performance for the We =1000 setup. To this end, we
study the strong scaling behavior in a range of P=32, . . ., 1024 cores on the cluster
Hawk. Figure 22 evaluates the parallel efficiency ηand the parallel speedup using the
total runtime, opposed to the PID-based results for the generic setup in figure 8.
104105
0.4
0.6
0.8
1
#DOFs/processor
η
100 1,000
1
10
#cores
Speedup
Fig. 22: Parallel efficiency (left) and parallel speedup (right) for the 2D shock-droplet
interaction with We =1000.
Up to 512 cores, a parallel efficiency beyond 50% is achieved. A significant drop in

32 Pascal Mossier et al.
the parallel efficiency occurs for 1024 cores, which is caused by the fact that the target
weight falls below the cost of the most expensive element, B∗< wmax, as discussed
in section 4.4. The imperfect strong scaling behavior can in general be attributed
to two main factors: the overhead of the dynamic load balancing procedure and a
remaining load imbalance due to the fast change in element-local loads. To support
this explanation, the imbalance Iover time was evaluated for the computation with
P=512 cores. Starting from an initial imbalance of more than I=15, the dynamic
load balancing scheme reduced the imbalance to I=0.7on average. Thus, the load
of the slowest processor still exceeded the target load by a factor of 1.7.
Furthermore, a partitioning with P=512 or more cores entails that an increasing
number of partitions in the narrow band region host only a singular element due to
their high computational cost. As the partitioning operates on the element level, no
further decomposition is possible for these partitions, implying the natural limit of
parallelization. Few of these one-element partitions can be seen in the figure 23, which
shows the workload distribution {wi}for the bulk discretization of the state given in
figure 20. The domain decomposition also illustrates the general working principle of
the dynamic load balancing scheme that makes small partitions cluster in regions of
higher computational cost while cheaper elements of N=2form larger partitions. As
computations in practice deploy fewer cores to evade the depicted granularity limit,
an overall acceptable parallel efficiency is achieved with the proposed hp-adaptive
level-set ghost fluid framework.
Fig. 23: Workload distribution and domain decomposition (top) for the bulk dis-
cretization (bottom) of the state given in figure 20 using P=512 processor units.

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 33
5.4.2 Water column with cavity
As seen in the previous setup, a local pressure minimum can be observed inside the
water column due to the reflected rarefaction wave. While phase transitions are not
included in the present model, in reality the low pressure could lead to caviation, i.e.
the formation of a vapor bubble inside the water column. Therefore Xiang and Wang
[24] proposed a setup to study the interaction between a planar incident shock wave and
a water column with a gas-filled cavity. Therefore, we extend the previous scenario by
introducing a circular cavity of radius r=0.0036 m inside the water column. All other
parameters are adopted from above, resulting in the setup sketched in figure 24. The
adapted initial conditions are summarized in table 4. Again, a Weber number of We =
ΩgsΩg
Ωgc
Ωl
Fig. 24: Initial setup of the 2D shock-droplet interaction with cavity.
ρ[kg m−3]u1[m s−1]p[MPa]γp∞[GPa]
Air (pre-shock) 1.20 0.0 1.01 1.4 0.0
Air (post-shock) 3.85 567.3 6.12 1.4 0.0
Air (cavity) 1000 0.0 1.04306 6.12 0.343
Water 1000 0.0 1.03479 6.12 0.343
Table 6: Initial conditions of the 2D shock-droplet interaction with cavity.
1000 is considered to simulate a droplet breakup in the SIE regime. The computational
setup is identical to above (section 5.4, and consumed 816 CPU hours, on 4 compute
nodes on Hawk. Schlieren images and non-dimensional pressure fields are depicted
in figure 25 for the time instances t∗=(tu1)/(2r)={0.8,1.62,5.9,6.5,9.0,11.8}. At
t∗=1.62, the reflected incident shock wave, the Mach step and the slip line develop
similar to the simulation without a cavity. However, the transmitted shock wave is
reflected and retransmitted at the surface of the water ring, creating a more complex
pattern of reflected and transmitted waves inside the water ring and the gas cavity.
Due to the curvature of the bubble, the transmitted waves inside the bubble appear
as circular shapes. Later on, the water column is flattened and a high speed water jet
causes the bubble to collapse, similarly to the phenomena observed in section 5.3. Our

34 Pascal Mossier et al.
(a) t∗=0.8(b) t∗=1.62
(c) t∗=5.9(d) t∗=6.5
(e) t∗=9.0(f) t∗=11.8
Fig. 25: Schlieren images and pressure fields of the 2D Ma =2.4shock-droplet
interaction with a gas-filled cavity for a Weber number of We =1000 (SIE regime).
numerical results agree well with the simulations presented by Xiang and Wang [24]
and Tsoutsanis et al. [60]. During the final phase of the simulation, the gas bubble is
further divided by secondary jets until four separate bubbles are formed. At t∗=9.0,
thin filaments have developed at the surface of the water column, which have detached
from the main liquid body at t∗=11.8. The evolution of the phase boundary in time
is visualized in detail in figure 26.
Finally, we look into the element-local discretization of the bulk flow and the level-
set transport. Figure 27 visualizes the FV sub-cells and the element-local polynomial
degree of the DG elements at time t∗=5.9. Shocks and the phase boundary are
successfully captured by FV sub-cells and the wake is resolved by a higher polynomial
degree to improve the resolution of vortices or weak acoustics. The level-set field is
discretized by the highest possible degree at the level-set zero position and oscillatory
solutions at filaments or merging phase boundaries are avoided by locally applying
the robust and accurate FV sub-cell discretization.

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 35
Fig. 26: Evolution of the phase boundary for the 2D Ma =2.4shock-droplet interaction
with a gas-filled cavity for a Weber number of We =1000. The separation of the
initial gas bubble into four bubbles by liquid jets and the formation of filaments can
be observed.
(a) Temperature field and bulk discretization (b) Level-set field and discretization
Fig. 27: Discretization of the bulk flow (left) and the level-set field (right) for the 2D
shock-droplet interaction with cavity for We =1000 at t∗=5.9. The element-local
degree Nis indicated by color, whereas FV sub-cell elements are marked in gray.
Detail plots highlight the refined resolution of the FV sub-cell grid compared to the
underlying DG mesh.
5.5 3D Shock-droplet interaction
As a final test case, we extend the two-dimensional shock-water-column setups from
the previous sections to a three-dimensional shock-droplet interaction to evaluate the
performance of our scheme in three dimensions. We consider a Ma =2.4shock wave
and a Weber number of We =100. A similar setup with a lower Mach number of
Ma =1.47 was used by Winter et al. [63] to investigate the RTP and SIE breakup
regime in three space dimensions. In analogy to the two-dimensional setup, the
computational domain Ω=[0.0,0.06]m×[−0.02,0.02]m×[−0.02,0.02]mcontains a
shock at x1=0.0074 m and a droplet of radius r=0.0048 m centered at x1=0.015 m.

36 Pascal Mossier et al.
The surface tension is chosen as σ=119.0and the initial conditions are listed in
table 7.
ρ[kg m−3]u1[m s−1]p[MPa]γp∞[GPa]
Air (pre-shock) 1.20 0.0 1.01 1.4 0.0
Air (post-shock) 3.85 567.3 6.12 1.4 0.0
Water 1000 0.0 1.257917 6.12 0.343
Table 7: Initial conditions of the three-dimensional shock-droplet interaction.
To achieve an effective resolution of 138 DOFs per droplet diameter, the domain is
discretized by 96 ×48 ×48 elements. Due to the symmetry of the setup, it is sufficient
to compute a quarter of the domain and impose symmetry boundary conditions at the
x2=0and x3=0planes. The left boundary is defined as an inflow plane, while all
remaining boundaries are non-reflecting. For the flux computation, an approximate
HLLC Riemann solver is applied. The setup is advanced in time until the final time
t=2.0·10−4s. The workload distribution is evaluated every 200 time steps and
(a) t∗=0.8(b) t∗=2.4
(c) t∗=4.7(d) t∗=7.0
(e) t∗=9.4(f) t∗=11.8
Fig. 28: Pressure field of the 3D Ma =2.4shock-droplet interaction with a Weber
number of 100 at different time instances.
triggers a repartitioning if I>1.0. We deployed 32 compute nodes of Hawk to
spend 7220 CPU hours on the simulation. Figure 28 depicts the evolution of the

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 37
droplet geometry and the non-dimensionalized pressure field p∗=p/p0at the time
instances t∗=(tu1)/(2r)={0.8,2.4,4.7,7.0,9.4,11.8}. At t∗=2.4, the development
of interfacial waves can be observed. Kaiser and Winter [34] suggested, that these
waves are triggered by the interaction of pressure fluctuations with the phase interface.
From t∗=0.8until t∗=9.4the interface waves grow in size and merge into larger
liquid sheets. At the final simulation time t∗=11.8, the stripping of the water sheets
from the main liquid body can be observed.
Fig. 29: Three-dimensional Ma =2.4shock-droplet interaction at t∗=7.0. The
velocity magnitude in the x2=0plane is visualized at the bottom and the element-
local bulk diskretization in the x2=0plane is projected to the background. To
visualize the recirculation zone in the wake of the droplet, isocontours of the velocity
magnitude are included.
Figure 29 visualizes the velocity magnitude and the element-local bulk discretiza-
tion at time t∗=7.0. Due to the significantly lower discretization than in the two-
dimensional setups, FV sub-cells are used to discretize the recirculation zone in the
wake of the droplet. This is necessary to avoid oscillations caused by the aliasing error
of a DG discretization in under-resolved regions.
6 Conclusion
We have introduced an hp-adaptive discretization into a level-set ghost-fluid frame-
work for compressible multiphase flows by combining a p-adaptive DGSEM scheme
with a FV sub-cell method. The hybrid discretization is applied to the continuum

38 Pascal Mossier et al.
model of the the bulk fluids, as well as the level-set transport equation. It relies on an
indicator that evaluates the modal decay of the polynomial solution representation and
additionally takes into account the position and topology of the phase interface. The
element-local discretization can thus automatically be adapted at runtime to provide
a high-order accuracy in smooth regions while offering stable and accurate results in
the presence of shocks and severely deformed phase boundaries. The adaptive dis-
cretization, together with applying the interface tracking algorithm only in a narrow
band around the phase interface, cause pronounced variations in the element-local
computational costs throughout the domain. In parallel simulations, these translate
to significant imbalances in the processor workloads, which necessitate a dynamic
load balancing scheme to ensure parallel scalability. The proposed scheme determines
the current workload distribution through element-local wall time measurements and
repartitions the elements along a space-filling curve. Strong scaling tests show ac-
ceptable results, with a parallel efficiency of 50% on up to 4096 cores. We apply our
level-set ghost-fluid framework to a wide range of inviscid, compressible multiphase
flows and obtain results in good agreement with numerical and experimental findings
from literature. When compared to non-adaptive computations, the presented adaptive
scheme produced comparable results with far less DOFs and a significantly reduced
wall time. The results confirm that the proposed hybrid discretization is well suited
for the multi-scale nature of multiphase flows. While the FV sub-cell grid enables
a high resolution and a good localization at the phase interface, the p-adaptive DG
scheme produces accurate results on a relatively coarse mesh. In the future, we plan
to include viscous effects to capture the impact of viscosity on the droplet breakup
regimes. Furthermore, we seek to include phase transfer to model evaporation and
cavitation.
Declarations
Funding We gratefully acknowledge the support by the German Research Foundation
(DFG) for the research reported in this publication through the project GRK 2160/1
"Droplet Interaction Technologies" under the project number 270852890 and through
Germany’s Excellence Strategy EXC 2075 under the project number 390740016.
All simulations were performed on the national supercomputer HPE Apollo Systems
HAWK at the High Performance Computing Center Stuttgart (HLRS) under the grant
number hpcmphas/44084.
Conflict of interest The corresponding author states on behalf of all authors, that
there is no conflict of interest.
Code availability The open source code FLEXI, on which all extensions are based,
is available at www.flexi-project.org under the GNU GPL v3.0 license.
Availability of data and material All data generated or analysed during this study
are included in this published article.

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 39
References
1. Anderson, D.M., McFadden, G.B., Wheeler, A.A.: Diffuse-interface methods in fluid mechanics.
Annual Review of Fluid Mechanics 30(1), 139–165 (1998). DOI 10.1146/annurev.fluid.30.1.139
2. Appel, D., Jöns, S., Keim, J., Müller, C., Zeifang, J., Munz, C.D.: A narrow band-based dynamic load
balancing scheme for the level-set ghost-fluid method. In: High Performance Computing in Science
and Engineering ’21 (in press) (2021)
3. Baer, M., Nunziato, J.: A two-phase mixture theory for the deflagration-to-detonation transition (ddt)
in reactive granular materials. International Journal of Multiphase Flow 12(6), 861–889 (1986).
DOI https://doi.org/10.1016/0301-9322(86)90033-9
4. Blind, M., Kopper, P.,Kempf, D., Kurz, M., Schwarz, A., Beck, A., Munz, C.D.: Performance Improve-
ments for Large Scale Simulations Using the Discontinuous Galerkin Framework FLEXI. Springer,
Cham (2022)
5. Castro, M., Gallardo, J., Parés, C.: High order finite volume schemes based on reconstruction of states
for solving hyperbolic systems with nonconservative products. Applications to shallow-water systems.
Mathematics of Computation 75(255), 1103–1134 (2006)
6. Chéné, A., Min, C., Gibou, F.: Second-order accurate computation of curvatures in a level set frame-
work using novel high-order reinitialization schemes. J. Sci. Comput. 35, 114–131 (2008). DOI
10.1007/s10915-007-9177-1
7. Cocchi, J.P., Saurel, R., Loraud, J.C.: Treatment of interface problems with Godunov-
type schemes. Shock Waves 35(1), 347–357 (1996). DOI 10.1007/BF02434010. URL
https://doi.org/10.1007/BF02434010
8. Cockburn, B., Karniadakis, G., Shu, C.W.: Discontinuous Galerkin Methods: Theory, Computation
and Application, vol. 11. Springer Berlin Heidelberg (2000). DOI 10.1007/978-3-642-59721-3
9. Courant, R., Fr iedrichs, K., Lewy, H.: Über die partiellen Differenzengleichungen der mathematischen
Physik. Mathematische Annalen 100(1), 32–74 (1928). DOI 10.1007/bf01448839
10. Denner, F., van Wachem, B.G.: Numerical time-step restrictions as a result of capillary waves. Journal
of Computational Physics 285, 24–40 (2015). DOI https://doi.org/10.1016/j.jcp.2015.01.021. URL
https://www.sciencedirect.com/science/article/pii/S002199911500025X
11. Dongarra, J., Beckman, P., et al.: The international exascale software project roadmap. The In-
ternational Journal of High Performance Computing Applications 25(1), 3–60 (2011). DOI
10.1177/1094342010391989
12. Dumbser, M., Boscheri, W., Semplice, M., Russo, G.: Central weighted ENO schemes for hyperbolic
conservation laws on fixed and moving unstructured meshes. SIAM Journal on Scientific Computing
39(6), A2564–A2591 (2017). DOI 10.1137/17M1111036
13. Dumbser, M., Käser, M., Titarev, V.A., Toro, E.F.: Quadrature-free non-oscillatory finite volume
schemes on unstructured meshes for nonlinear hyperbolic systems. Journal of Computational Physics
226(1), 204–243 (2007). DOI 10.1016/j.jcp.2007.04.004
14. Dumbser, M., Loubère, R.: A simple robust and accurate a posteriori sub-cell finite volume limiter for
the discontinuous Galerkin method on unstructured meshes. Journal of Computational Physics 319,
163–199 (2016). DOI 10.1016/j.jcp.2016.05.002
15. Dumbser, M., Peshkov, I., Romenski, E., Zanotti, O.: High order ADER schemes for a unified first
order hyperbolic formulation of continuum mechanics: Viscous heat-conducting fluids and elastic
solids. Journal of Computational Physics 314, 824–862 (2016). DOI 10.1016/j.jcp.2016.02.015
16. Dumbser, M., Zanotti, O., Loubère, R., Diot, S.: A posteriori subcell limiting of the discon-
tinuous Galerkin finite element method for hyperbolic conservation laws. Journal of Com-
putational Physics 278, 47–75 (2014). DOI https://doi.org/10.1016/j.jcp.2014.08.009. URL
https://www.sciencedirect.com/science/article/pii/S0021999114005555
17. Einfeldt, B.: On Godunov-type methods for gas dynamics. SIAM Journal on Numerical Analysis
25(2), 294–318 (1988). DOI 10.1137/0725021
18. Fechter, S.: Compressible multi-phase simulation at extreme conditions using a discontinuous Galerkin
scheme. Ph.D. thesis, University of Stuttgart (2015)
19. Fechter, S., Jaegle, F., Schleper, V.: Exact and approximate Riemann solvers at phase boundaries.
Computers & Fluids 75, 112–126 (2013). DOI 10.1016/j.compfluid.2013.01.024
20. Fechter, S., Munz, C.D., Rohde, C., Zeiler, C.: A sharp interface method for compressible liquid–vapor
flow with phase transition and surface tension. Journal of Computational Physics 336, 347–374 (2017).
DOI 10.1016/j.jcp.2017.02.001

40 Pascal Mossier et al.
21. Fechter, S., Munz, C.D., Rohde, C., Zeiler, C.: Approximate Riemann solver for compressible liquid
vapor flow with phase transition and surface tension. Computers & Fluids 169, 169–185 (2018).
DOI 10.1016/j.compfluid.2017.03.026
22. Fedkiw, R.P., Aslam, T., Merriman, B., Osher, S.: A non-oscillatory Eulerian approach to interfaces
in multimaterial flows (the ghost fluid method). Journal of Computational Physics 152(2), 457–492
(1999). DOI 10.1006/jcph.1999.6236
23. Föll, F., Hitz, T., Müller, C., Munz, C.D., Dumbser, M.: On the use of tabulated equations of state
for multi-phase simulations in the homogeneous equilibrium limit. Shock Waves (2019). DOI
10.1007/s00193-019-00896-1
24. Gaoming, X., Wang, B.: Numerical study of a planar shock interacting with a cylindrical water
column embedded with an air cavity. Journal of Fluid Mechanics 825, 825–852 (2017). DOI
10.1017/jfm.2017.403
25. Goncalves, E., Hoarau, Y., Zeidan, D.: Simulation of shock-induced bubble collapse using a four-
equation model. Shock Waves 29, 221–234 (2019). DOI 10.1007/s00193-018-0809-1
26. Grooss, J., Hesthaven, J.: A level set discontinuous Galerkin method for free surface flows. Com-
puter Methods in Applied Mechanics and Engineering 195(25-28), 3406–3429 (2006). DOI
10.1016/j.cma.2005.06.020
27. Han, L., Hu, X., Adams, N.: Adaptive multi-resolution method for compressible multi-phase flows with
sharp interface model and pyramid data structure. Journal of Computational Physics 262, 131–152
(2014). DOI 10.1016/j.jcp.2013.12.061
28. Hilbert, D.: Über die stetige Abbildung einer Linie auf ein Flächenstück", pp. 1–2. Springer Berlin
Heidelberg, Berlin, Heidelberg (1935)
29. Hindenlang, F.: Mesh curving techniques for high order parallel simulations on unstructured meshes.
Ph.D. thesis, Universität Stuttgart (2014)
30. Hu, X., Khoo, B., Adams, N., Huang, F.: A conservative interface method for compressible flows. Jour-
nal of Computational Physics 219(2), 553–578 (2006). DOI https://doi.org/10.1016/j.jcp.2006.04.001
31. Huerta, A., Casoni, E., Peraire, J.: A simple shock-capturing technique for high-order discontinuous
Galerkin methods. International Journal for Numerical Methods in Fluids 69, 1614 – 1632 (2012).
DOI 10.1002/fld.2654
32. Jiang, G.S., Peng, D.: Weighted ENO schemes for Hamilton–Jacobi equations. SIAM Journal on
Scientific Computing 21(6), 2126–2143 (2000). DOI 10.1137/s106482759732455x
33. Jöns, S., Müller, C., Zeifang, J., Munz, C.D.: Recent advances and complex applications of the
compressible ghost-fluid method. In: M.L. Muñoz-Ruiz, C. Parés, G. Russo (eds.) Recent Advances
in Numerical Methods for Hyperbolic PDE Systems, pp. 155–176. Springer International Publishing
(2021)
34. Kaiser, J., Winter, J., Adami, S., Adams, N.: Investigation of interface deformation dynam-
ics during high-Weber number cylindrical droplet breakup. International Journal of Multiphase
Flow 132, 103409 (2020). DOI https://doi.org/10.1016/j.ijmultiphaseflow.2020.103409. URL
https://www.sciencedirect.com/science/article/pii/S0301932220305188
35. Kennedy, C.A., Carpenter, M.H.: Additive Runge-Kutta schemes for convection-diffusion-
reaction equations. Applied Numerical Mathematics 44(1), 139–181 (2003). DOI
https://doi.org/10.1016/S0168-9274(02)00138-1
36. Kopriva, D.A.: Implementing Spectral Methods for Partial Differential Equations: Algorithms for
Scientists and Engineers. Springer Publishing Company, Incorporated (2009)
37. Krais, N., Beck, A., Bolemann, T., Frank, H., Flad, D., Gassner, G., Hindenlang, F., Hoffmann,
M., Kuhn, T., Sonntag, M., Munz, C.D.: Flexi: A high order discontinuous Galerkin framework for
hyperbolic-parabolic conservation laws. Computers & Mathematics with Applications 81, 186–219
(2020). DOI 10.1016/j.camwa.2020.05.004
38. Lieber, M., Nagel, W.E.: Highly scalable SFC-based dynamic load balancing and its application
to atmospheric modeling. Future Generation Computer Systems 82, 575–590 (2018). DOI
https://doi.org/10.1016/j.future.2017.04.042
39. Mavriplis, C.: A posteriori error estimators for adaptive spectral element techniques. In: P. Wesseling
(ed.) Proceedings of the Eighth GAMM-Conference on Numerical Methods in Fluid Mechanics, pp.
333–342. Vieweg+Teubner Verlag, Wiesbaden (1990)
40. Meng, J.C., Colonius, T.: Numerical simulations of the early stages of high-speed droplet breakup.
Shock Waves 25, 399–414 (2015)
41. Merkle, C., Rohde, C.: The sharp-interface approach for fluids with phase change: Riemann problems
and ghost fluid techniques. ESAIM: Mathematical Modelling and Numerical Analysis 41(06), 1089–
1123 (2007). DOI 10.1051/m2an:2007048

An Efficient hp-Adaptive Strategy for a Level-Set Ghost Fluid Method 41
42. Message Passing Interface Forum: MPI: A Message-Passing Interface Standard (2021). URL
https://www.mpi-forum.org/
43. Métayer, O.L., Massoni, J., Saurel, R.: Élaboration des lois d'état d'un liquide et de sa vapeur pour
les modèles d'écoulements diphasiques. International Journal of Thermal Sciences 43(3), 265–276
(2004). DOI 10.1016/j.ijthermalsci.2003.09.002
44. Mossier, P., Munz, C.D., Beck, A.: A p-adaptive discontinuous Galerkin method with hp-shock
capturing. Journal of Scientific Computing 91(1), 1573–7691 (2022). DOI 10.1007/s10915-022-
01770-6
45. Müller, C., Hitz, T., Jöns, S., Zeifang, J., Chiocchetti, S., Munz, C.D.: Improvement of the level-set
ghost-fluid method for the compressible Euler equations. In: G. Lamanna, S. Tonini, G.E. Cos-
sali, B. Weigand (eds.) Droplet Interactions and Spray Processes, pp. 17–29. Springer International
Publishing, Cham (2020)
46. Neumann, J.V., Richtmyer, R.D.: A method for the numerical calculation of hydrodynamic shocks.
Journal of Applied Physics 21, 232–237 (1950)
47. Nourgaliev, R.R., Dinh, T.N., Theofanous, T.G.: Adaptive characteristics-based matching for com-
pressible multifluid dynamics. Journal of Computational Physics 213(2), 500–529 (2006). DOI
https://doi.org/10.1016/j.jcp.2005.08.028
48. Peng, D., Merriman, B., Osher, S., Zhao, H., Kang, M.: A pde-based fast local level set method 1.
Journal of Computational Physics 155.2, 410–438 (1998)
49. Persson, P.O.: Shock capturing forhigh-order discontinuous Galerkin simulation of transient flow prob-
lems. In: 21st AIAA Computational Fluid Dynamics Conference. American Institute of Aeronautics
and Astronautics (2013). DOI 10.2514/6.2013-3061
50. Persson, P.O., Stamm, B.: A discontinuous Galerkin method for shock capturing using a mixed high-
order and sub-grid low-order approximation space. Journal of Computational Physics 449, 110765
(2022). DOI 10.1016/j.jcp.2021.110765
51. Pınar, A., Aykanat, C.: Fast optimal load balancing algorithms for 1d partitioning. Journal of Parallel
and Distributed Computing 64(8), 974–996 (2004)
52. Sethian, J.A.: A fast marching level set method for monotonically advancing fronts. Proceedings of
the National Academy of Sciences 93(4), 1591–1595 (1996). DOI 10.1073/pnas.93.4.1591
53. Sonntag, M., Munz, C.D.: Shock capturing for discontinuous Galerkin methods using finite volume
subcells. Finite Volumes for Complex Applications VII 78, 945–953 (2014). DOI 10.1007/978-3-
319-05591-6_96
54. Sonntag, M., Munz, C.D.: Efficient parallelization of a shock capturing for discontinuous Galerkin
methods using finite volume sub-cells. Journal of Scientific Computing 70, 1262–1289 (2017). DOI
10.1007/s10915-016-0287-5
55. Sundarapandian, S., Liverts, M., Tillmark, N., Apazidis, N.: Plane shock wave interaction with a
cylindrical water column. Physics of Fluids 28, 056102 (2016). DOI 10.1063/1.4948274
56. Sussman, M., Smereka, P., Osher, S.: A level set approach for computing solutions to incom-
pressible two-phase flow. Journal of Computational Physics 114(1), 146–159 (1994). DOI
10.1006/jcph.1994.1155
57. Theofanous, T.G., Li, G.J., Dinh, T.N.: Aerobreakup in Rarefied Supersonic Gas Flows .
Journal of Fluids Engineering 126(4), 516–527 (2004). DOI 10.1115/1.1777234. URL
https://doi.org/10.1115/1.1777234
58. Toro, E.F., Spruce, M., Speares, W.: Restoration of the contact surface in the HLL-Riemann solver.
Shock Waves 4(1), 25–34 (1994). DOI 10.1007/bf01414629
59. Tsai, Y.H.R., Cheng, L.T., Osher, S., Zhao, H.K.: Fast sweeping algorithms for a class of
hamilton–jacobi equations. SIAM Journal on Numerical Analysis 41(2), 673–694 (2003). DOI
10.1137/s0036142901396533
60. Tsoutsanis, P.,Adebayo, E., Merino, A., Arjona, A., Skote, M.: Cweno finite-volumeinterface capturing
schemes for multicomponent flows using unstructured meshes. Journal of Scientific Computing 89
(2021). DOI 10.1007/s10915-021-01673-y
61. Tsoutsanis, P., Titarev, V., Drikakis, D.: Weno schemes on arbitrary mixed-element
unstructured meshes in three space dimensions. Journal of Computational Physics
230(4), 1585–1601 (2011). DOI https://doi.org/10.1016/j.jcp.2010.11.023. URL
https://www.sciencedirect.com/science/article/pii/S0021999110006388
62. Williamson, J.: Low-storage Runge-Kutta schemes. Journal of Computational Physics
35(1), 48–56 (1980). DOI https://doi.org/10.1016/0021-9991(80)90033-9. URL
https://www.sciencedirect.com/science/article/pii/0021999180900339

42 Pascal Mossier et al.
63. Winter, J.M., Kaiser, J.W., Adami, S., Adams, N.A.: Numerical investigation of 3d drop-breakup
mechanisms using a sharp interface level-set method. In: 11th International Symposium on Turbulence
and Shear Flow Phenomena (TSFP 2019) (2019)
64. Zeifang, J.: A discontinuous Galerkin method for droplet dynamics in weakly compressible flows.
Ph.D. thesis, University of Stuttgart (2020)
65. Zeifang, J., Beck, A.: A data-driven high order sub-cell artificial viscosity for the discontinuous
Galerkin spectral element method. Journal of Computational Physics 441, 110475 (2021). DOI
10.1016/j.jcp.2021.110475