Content uploaded by David Nordsletten

Author content

All content in this area was uploaded by David Nordsletten on Jun 28, 2017

Content may be subject to copyright.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

SIAM J. SCI.COMPUT.c

⃝2016 Society for Industrial and Applied Mathematics

Vol . 38 , N o . 3, p p . C1 5 0– C1 7 8

MULTIPHYSICS COMPUTATIONAL MODELING IN CHeart∗

J. LEE†,A.COOKSON

†,I.ROY

†,E.KERFOOT

†,L.ASNER

†,G.VIGUERAS

†,

T. SOCHI†,S.DEPARIS

‡,C.MICHLER

†,N.P.SMITH

§,AND D. A. NORDSLETTEN¶

Abstract. Fr om ba s i c s c ien c e t o tran s l a t ion , mo de r n b i ome d i c al res e a r ch d e m a nds co m p uta-

tional models which integrate several interacting physical systems. This paper describes the infra-

structural framework for generic multiphysics integration implemented in the software CHeart,a

ﬁnite-element code for biomedical research. To generalize the coupling of physics systems, we intro-

duce a framework in which the geometric and operator relationships between the constituent systems

are rigorously deﬁned. We then introduce the notion of topological interfaces and deﬁne speciﬁc

operators encompassing many common model coupling requirements. These interfaces enable the

evaluation of weak form integrals between mesh subregions of arbitrary ﬁnite-element bases’ orders,

type s, and spatial dimensions. Equation maps are intro duced wh ich pr ovide abstra ct representations

of the individual physics systems that can be automatically combined to permit a monolithic matrix

assembly. Flexible solution strategies for the resulting coupled systems are implemented, permitting

ﬁne-tuning of solution updates during ﬁxed point iterations, and subgrouping where several prob-

lems are being solved together. Partitioning of coupled mesh domains for optimal load balancing is

also supported, taking into account the per-processor cost of the entire coupled problem within the

graph problem. The demonstration of the p erformance is illustrated through important real-world

multiphysics problem s relevant to cardiac physi ology.

Key words. cardiac modeling, multiphysics, parallel computing, coupled problems, modeling

software

AMS subject classiﬁcations. 92-04, 92-08, 74F10, 74S05, 76M10, 68W10, 92C30

DOI. 10.1137/15M1014097

1. Introduction. Computational modeling of biophysical phenomena has be-

come a common approach for investigating problems that arise in medical sciences.

From subcellular processes [22, 86] to whole-organ function [65, 81], mathematical

models continue to demonstrate their capacity to predict physiology, provide added

insight, and address questions of both scientiﬁc and clinical interest. The demand

for solving computational models has led to the development of a variety of soft-

ware packages [82, 83, 43, 74, 59, 24, 52, 73, 47, 71, 10, 75]. The success of many

of these tools stems from their ability to oﬀer eﬃcient and scalable computation

for speciﬁc physics problems. These packages widely employ ﬁnite-element methods

∗Submitted to the journal’s Software and High-Performance Computing section March 27, 2015;

accepted for publication (in revised form) March 28, 2016; published electronically May 26, 2016.

This work was supported by the British Heart Foundation (NH/11/5/29058), the Engineering and

Physical Sciences Research Council (EP/G0075727/2), and the Europ ean Commission funded eu-

Heart project (FP7-ICT-2007-224495:euHeart). It was also supported by the Wellcome Trust-EPSRC

Centre of Excellence in Medical Engineering (WT 088641/Z/09/Z) and the NIHR Biomedical Re-

search Centre at Guy’s and St.Thomas’ NHS Foundation Trust and KCL. The views expressed are

those of the authors and not necessarily those of the NHS, the NIHR, or the DoH.

http: //www.siam.org/journal s/s isc/38-3/M101409.html

†Department of Biomedical Engineering, King’s College London, London, UK (jack.lee@kcl.ac.uk,

andrew.cookson@kcl.ac.uk, ishani.roy@kcl.ac.uk, eric.kerfoot@kcl.ac.uk, liya.asner@kcl.ac.uk,

guillermo.vigueras gonzalez@kcl.ac.uk, taha.sochi@kcl.ac.uk, christian.michler@kcl.ac.uk).

‡Mathematics Institute of Computational Science and Engineering, EPFL, Lausanne, Switzerland

(simone.deparis@epﬂ.ch).

§Department of Biomedical Engineering, King’s College London, London, UK, and Faculty of

Engineering, University of Auckland, Auckland, New Zealand (np.smith@auckland.ac.uk).

¶Corresponding author. Department of Biomedical Engineering, King’s College London, London,

UK (david.nordsletten@gmail.com).

C150

Downloaded 06/13/16 to 138.38.74.106. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C151

Fig. 1.1.An example of ﬁnite-eleme nt meshes used for mult iphysics simulations. Illustrated

is the coupling between overlapping domains (hexahedral (1) and tetrahedral meshes (5, 6) for solid

mechanics, monodomain, and poroelasticity problems) and over partially overlapping domains (en-

docardial (2) and cavity mesh (3) for ﬂuid-structure interaction (FSI) problems and vascular mesh

(4) for the coronary ﬂow model).

(FEMs) for numerical approximation of diﬀerential equations and have introduced a

range of techniques that span the computational spectrum. From support for low

order h-reﬁned techniques [82, 83], to high order p-FEMs [43], to varied mixed meth-

ods [24, 10, 33] and h-adaptive techniques [73, 52], diverse approaches for problem

solving have been enabled. A number of software packages also utilize mesh parti-

tioning (such as ParMETIS [44], Scotch [72], or Zoltan [19]) and parallel linear solver

libraries (such as Trilinos [34] and PETSc [5, 4, 6]) for eﬃcient multicore paralleliza-

tion. These strategies for scability are necessary to address the computational chal-

lenges presented by many mathematical models (e.g., hemodynamics [74, 43, 52, 24]

and electrophysiology [59, 82, 83, 73]).

Despite these advances, the seemingly simple task of multiphysics modeling by in-

tegrating existing computational models remains challenging. For a general coupling

between arbitrary problems, many aspects of the individual problem—including nu-

merical discretization and solution schemes, disparate solution domains, and data ac-

cessibility issues—have to be formalized, presenting new implementational challenges.

For instance, in the context of cardiac modeling a variety of numerical techniques are

employed (see Figure 1.1), spanning from the one dimensional (1D) network mani-

folds used in vascular ﬂow modeling [79], to curvilinear high order elements used in

mechanics [61], to extremely reﬁned tetrahedral meshes used in electrophysiology [62].

The resulting equations also require the ability to solve large scale linear systems as

well as highly nonlinear systems. The varied nature of these physical models leads to

many diﬀerent forms of coupling, where integrated models may require equating kine-

matics/force (as found in ﬂuid-solid interaction), mass transport/force (as found in

poromechanics or multilevel transport), or strain/tension (as found in electromechan-

Downloaded 06/13/16 to 138.38.74.106. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

C152 LEE ET AL.

ics) between models on heterogeneous domains. The permutation of these factors

has made the generalized coupling of multiphysics systems a signiﬁcant challenge to

the scientiﬁc community. While the integration of speciﬁc physical systems has been

explored in the literature [24, 52, 33], there is still much that can be gained from

a focused examination of the fundamental design that consolidates a multiphysics

solver.

In this paper we describe an infrastructural framework for generalizing the integra-

tion of physical phenomena within CHeart, a multiphysics software engine developed

at King’s College London. To date, it has been used to solve a variety of problems in

physiological modeling, including the Navier–Stokes equations for ﬂuid ﬂow [17], non-

linear solid and porous mechanics [13, 32, 31, 3, 2], scalar transport equations [60, 41],

elastic wave mechanics [49], pressure Poisson for approximating pressure from Navier–

Stokes using known velocities [48, 20], and electrophysiology [84]. These essential

single-physics building blocks provide the components for assembly and linking to

form multiphysics models. This is achieved through a software infrastructure using

what we call object-sets that act as generic sets to store diﬀerent core FEM objects

that may be constructed, used, or shared by any multiphysics problem. This strat-

egy is used to manage bases, meshes and variables, topological interfaces, equation

abstraction through equation mappings, and a series of general physics couplings to

ﬂexibly interlink heterogeneous physical models. These processes enable partitioned

coupling, monolithic coupling, or a mixed approach for diﬀerent components of the

physical systems allowing for a multitude of possible solution strategies. The infra-

structural organization outlined has enabled the interlinking of a variety of models,

including ﬂuid-solid coupling [67, 68, 70, 56, 18, 57, 58], vascular-porous ﬂow [50, 51],

and electromechanics [84].

The remainder of the paper will describe a general strategy for multiphysics inte-

gration and demonstrate its application. In section 2 we outline the physics systems

which provide core simulation components for multiphysics integration. The general

object-set approach, construction of function and variable spaces as well as the intro-

duction of topological interfaces, is provided in section 3. A series of single-physics

coupling strategies is described in section 4. The generic construction and handling

of these diﬀerent systems is introduced in section 5 with particular emphasis on their

abstraction through the use of equation mappings. In section 6, we propose a graph

parallelization strategy which enables eﬃcient multicore processing. These techniques

are then validated and demonstrated on a series of multiphysics systems relevant to

cardiac physiology (see theonlinesupplement (M101409 01.pdf [local/web 4.23MB])),

with a speciﬁc example for ﬂuid-structure interaction (FSI) provided in section 8.2.

2. Core single-physics module. In the ﬁnite-element framework the govern-

ing equations for physical systems are provided in standard weak form. In this section

we introduce a generic notation used for deﬁning operators, or functionals, for dif-

ferent physical systems. These functionals provide the building blocks for integrated

multiphysics systems.

Diﬀerent physical systems and their weak form functionals vary in the terms,

function spaces, number of variables, and their inclusion of diﬀerent ﬁelds. Here we

consider each problem as a specialization of a general weak functional, f:Vh(Ωh)×

Yh(Ωh)→R,

(2.1)

f(vh,yh;Ω

h)=!Ωh

a(vh)·yh+c(vh):∇yhdΩh,vh∈Vh(Ωh),yh∈Yh(Ωh),

Downloaded 06/13/16 to 138.38.74.106. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C153

Tab l e 2 . 1

Summary of available options for diﬀerent physics systems and physics-based operations.

Physics-based operators & operations

ODE/DAE systems Adaptive time stepping, implicit/explicit schemes (Euler, RK4, etc.)

Lumped parameter cardiovascular models (incl. [77, 85])

Var i ou s c e l l mo d e l s (i n c l. t e n Tu ss che r 2 0 06 [8 0 ] , Lu o – Ru d y 1 9 91 [5 3 ] )

Dynamic loading of precompiled cell models

Solid mechanics Incompressible, penalty, weak penalty, nearly incompressible formulations

Arbitrary mixed formulations: displacement, velocity, pressure

Arbitrary constitutive laws (isotropic, orthotropic, anisotropic laws)

Summable constitutive laws, i.e., σ(x)=!kwk(x)σk(x)

Scalar transport Static/transient, inclusion/exclusion of advection and/or reaction

Optional SUPG stabilization

Solve as linear or nonlinear system

Fluid mechanics Stokes and Navier–Stokes, static and transient

Eulerian and arbitrary Lagrangian–Eulerian (ALE) formulations

Conservative and noncon servative ALE formulations

General mixed formulations

SUPG formulation for all forms

1D ﬂuid ﬂow Static/transient linear/nonlinear formulations

Area-ﬂow or area-velocity formulations

Riemann invariant or compatibility-based b oundary condition formulations

Simpliﬁcation to Poiseuille network ﬂow (compliant and rigid vessel wall)

Accommodates arbitrary wall constitutive mo dels

Primitive variable or Windkessel-type boundary conditions

Pressure Poisson Stokes and Navier–Stokes, static and transient

Elastic wave equation Linear and nonlinear formulations

Norms & measures Compute L2,H1norms/seminorms of variable expressions

Compute general inner-product expressions

Projections Compute L2,H1,andHdiv projection operators

Parameter e stim atio n Reduced-order unscented Kalman ﬁlter

Support for arbitrary spatially varying parameters

where vhis an n-dimensional variable (which may consist of one or more state vari-

able(s)), yhis the nth-dimensional test function, Vh(Ωh)andYh(Ωh)aretheset

of trial and test spaces, respectively, and Ωh⊂Rdis the FEM domain description.

Here,

a:Vh→"L2(Ωh)#n,c:Vh→"L2(Ωh)#n×d

are speciﬁc functionals of the variable, vh, which depend on the underlying physical

system. In a single-physics example, the functional fwould be used to ﬁnd the

solution vhin the weak form statement: ﬁnd vh∈Vh(Ωh)suchthat

(2.2) f(vh,yh;Ω

h)=0 ∀yh∈Yh(Ωh).

As the variability between physics arises predominantly in the aand cfunctionals, this

abstract deﬁnition of the weak form problem enables generic handling of the core FEM

procedures. In the remainder of this section we introduce the speciﬁc core physics

currently implemented in CHeart. A summary of these modules is also provided in

Table 2.1.

2.1. Scalar transport. The conduction of electrical current, movement of me-

tabolites, and conduction of simple ﬂow potential problems are a few examples of

a wide group of transport processes. Their simulation can be achieved through the

solution of the transient scalar advection-diﬀusion-reaction equation or some simpli-

ﬁcation of it (see Table 2.1). As a result, scalar transport equations encompass a

broad range of problems ranging from the most simpliﬁed form, Laplace, to the full

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

C154 LEE ET AL.

transport system. In the case of a transient advection-diﬀusion-reaction equation

discretized using the backward Euler scheme, the operators in (2.1) have the form

a(vh) := 1

δt

(vk

h−vk−1

h)+u·∇vk

h−λvk

h+s, c(vh) := D∇vk

h,

where vk

h∈Vh=Vhis the unknown scalar value at the kth time step, δtis the time

step interval, uis an advective velocity ﬁeld, λis a linear reaction coeﬃcient ﬁeld, D

is a tensor diﬀusion coeﬃcient ﬁeld, and sis a source/sink ﬁeld.

2.2. Solid mechanics. Solid mechanics in cardiac applications is solved us-

ing large strain ﬁnite deformation theory [55, 9, 7] within quasi-static or transient

frameworks. The speciﬁc formulations can vary depending on whether the material is

modeled as compressible, incompressible, or nearly incompressible [32] as well as the

speciﬁc selection of constitutive laws (see Table 2.1). As an example, a quasi-static

incompressible material can be modeled by (2.1) using the aand coperators

a(uh,p

h) := (ρb,det |∇uh+I|−1),c(uh,p

h) := (σ(uh)−phI,0),

where (uh,p

h)∈Uh×Wh=Vhare displacement and pressure, ρand bdenote mate-

rial density and body force, respectively, σ(uh) is the deviatoric Cauchy stress tensor

which may be deﬁned as an arbitrary sum of core constitutive laws (see Table 2.1)

scaled by a weighting ﬁeld, and Iis the identity tensor. Currently, a collection of

isotropic (i.e., neo-Hookean, Mooney–Rivlin [9]), general anisotropic constitutive laws

(i.e., Costa [14], Guccione [30], and Holzapfel–Ogden [36]) and viscoelastic models are

implemented and can be selectively applied tothetotalorthedeviatoriccomponent

of the strain tensor. Extensions to the existing constitutive laws are achieved by the

generalized implementation of the stress tensor, enabling straightforward inclusion of

new materials.

2.3. Fluid mechanics. Fluid mechanics can be solved for a variety of problems

in the heart (see Table 2.1), ranging from steady Stokes ﬂow in small coronary vessels

to the more complex arbitrary Lagrangian–Eulerian (ALE) form of Navier–Stokes

equations [35, 40, 37, 64, 25, 66, 70] in the cardiac heart chambers. As an example, in

the nonconservative form of the Navier–Stokes equations discretized using backward

Euler, the operators aand cof (2.1) are written as

a(vh,p

h):=!ρ

δt

(vk

h−vk−1

h)+ρ(vk

h−wk

h)·∇vk

h,∇·vk

h",c(vh,p

h):=(µσ(vk

h)−pk

hI,0),

where (vk

h,p

k

h)∈Vh×Wh=Vhare the ﬂuid velocity/pressure at the kth time step

with step-size δt;ρ, µ are the density and viscosity; and σ(vh) is the stress operator

(see Table 2.1). By using various bases, Vh×Whmay be deﬁned for a wide range of

inf-sup stable spaces (see section 3.4), such as Pκ−Pκ−1Taylor–Hood elements [11],

Qκ−Pκ−1Nicolaides–Boland [8] elements, and Crouzeix–Raviart [16] spaces as well

as others. Time step handling, in general, is consistent with that shown in [70] for

ALE formulations. Further, as stabilization is often required for practical problems,

the test and trial spaces may bediﬀerent,enablinggeneralintroductionofstreamline

upwinding Petrov–Galerkin (SUPG) methods [38].

2.4. Darcy ﬂow. Darcy’s law in combination with conservation of mass is used

to model the ﬂow of a ﬂuid through a permeable porous medium [15, 60, 13]. In the

steady form, the operators aand cof (2.1) are written as

a(vh,p

h) := (µK−1vh,∇·vh−s),c(vh,p

h) := (−ph,0),

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C155

where (vh,p

h)∈Vh×Wh=Vhare the Darcy velocity and pore pressure, respec-

tively, µis the dynamic viscosity, Kis the permeability tensor, and sis a volumetric

ﬂow rate source or sink. Currently, this implementation relies on inf-sup stable spaces

Vhand Wh.

2.5. 1D ﬂuid ﬂow. The 1D ﬂuid ﬂow equations in an elastic vessel can be

obtained from the incompressible Navier–Stokes equations written in cylindrical co-

ordinates by assuming axisymmetric ﬂow and by absorbing the dependence on the

radial coordinate into a lumped parameter [39, 26, 76]. This results in an equation

set deﬁned by

a(Qh,A

h) := $Qk

h−Qk−1

h

δt

+κQk

h

Ak

h

,Ak

h−Ak−1

h

δt%,

c(Qh,A

h) := −$α(Qk

h)2

Ak

h

+!c2

wdA, Qk

h%,

where (Qk

h,A

k

h)∈Vh×Wh=Vhis the ﬂuid volumetric ﬂow rate/vessel cross-

sectional area at the kth time step with step-size δt,κis a friction coeﬃcient, cw

is the local pulse wave velocity, and αis the momentum correction factor which

depends on the assumed ﬂuid velocity proﬁle. At junctions where branching occurs,

the above equations are further augmented by a set of coupling conditions, i.e., mass

conservation and the Bernoulli equation [76].

2.6. ODE and DAE systems. Various systems of ordinary diﬀerential equa-

tions (ODEs) and diﬀerential algebraic equations (DAEs) ranging from 0DWindkessel

models [85, 78, 1] to tissue-level cell models [23, 53, 80] can be described using the

equation v′=g(v), where gis known as the rate function. The solution of such prob-

lems can be obtained using explicit, semi-implicit, or implicit time integration with

constant time steps or adaptive time stepping. For example, the standard explicit

Euler scheme can be written in the form of (2.1) using the aand coperators,

a(vh) := 1

δk

t

(vk+1

h−vk

h)−g(vk

h),c(vh) := 0,

where vk

h∈Vh=Vhis the state variable, and δk

tis the step-size at the kth time

step. The domain of the problem is normally either a point or a series of points which

eﬀectively decouples the solution to individual nodes or modes (i.e., for a system with

nvariables, vh∈[L∞(Ωh)]n).

In practice, directly incorporating a wide spectrum of ODE and DAE models into

the code base requires frequent editing and recompilation. Flexibility is achieved in

CHeart through dynamic loading of user-deﬁned rate functions g.Thisallows,for

example, a direct C-code export from online model repositories such as CellML1to

be integrated without global recompilation.

3. FEM weak form construction. The core physics-based operators intro-

duced in the previous section rely on a series of routines common to all FEM codes.

The additional challenge of multiphysics comes with the eﬃcient reuse, integration,

and cross-communication of ﬁelds, domains, and weak form functionals. Critical to

this is the formation of object-sets—representing a collection of core objects—that may

be reused freely. An illustrated example of how diﬀerent object-sets within CHeart

interact together is shown in Figure 3.1.

1www.cellml.org

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

C156 LEE ET AL.

VARIABLES PHYSICS SOLVER SOLUTION DOMAIN

Solver

matrix

(FSI)

Solver

matrix

(ALE)

Sequential

fixed-point

(line search)

Math

Expression

Section 3.5

Basis

Function

Section 3.1

Mesh

Topology

Section 3.2

Solver

Matrix

Section 5.3

Solve

Group

Section 5.5

Interface

Section 3.3

Equation

Section 2

Variable

Section 3.4

Identity

Linear fluid

domain

Identity

Cubic solid

domain

Quadratic

solid domain

Injective

Fluid pressure

Fluid velocity

Fluid Space

Domain velocity

Boundary Space

Lagrange multiplier

Solid displacement

Solid pressure

Navier-

Stokes

Solid

mechanics

ALE

problem

Coupling

problem

(

LM

)

Linear

tetrahedral

Quadratic

tetrahedral

Quadratic

triangular

Cubic

hexahedral

Quadratic

hexahedral

Quadratic

fluid domain

Boundary

surface

Fig. 3.1.Illustration relating various object-sets used by CHeart to solve physics problems.

In this case, we consider the coupled ﬂuid-solid interaction (see Results). The ﬂuid is solved on

atetrahedralgridusingP2−P1Ta ylo r–H ood e le m ent s a nd t he AL E N avi e r–S t ok e s for m ul a tio n

introduced in section 2.3.ThesolidissolvedonahexahedralcurvilineargridusingQ3−Q2Tayl or–

Hood elements and the solid mechanics formulation from section 2.2.Object-setBis constructed

using ﬁve basis object types with varying order shape and dimension. These are used with ﬁve

topology objects comprising T,andtheinterfaceIobject-set is composed of two identity and two

injective interfaces. Vis then built from eight variables used in the four core problems feeding into

the monolithic solver.

In this section we review the core routines implemented to facilitate multiphysics

integration. As bas is interpolation schemes may be widely diﬀerent, we formulate a

basis object-set for which we developed a single strategy for computation and interpo-

lation of diﬀerent basis functions. These basis tools form the foundation for the gen-

eration of topologies—meshes with speciﬁed interpolation—which are integrated into

a topology object-set forming ﬂexible structures to house meshesofdiﬀerent order,

size, dimension, etc. To enable cross-communication between ﬁelds and topologies, we

introduce a general construct of interfaces that allows the formulation of functionals

which relate variables deﬁned on diﬀerent topologies. With these tools, we construct

variable object-sets, which may be used in the general functionals introduced above

to assemble weak form statements.

3.1. Basis generation. The core physics modules presented in section 2 re-

quire a range of diﬀerent interpolation functions that may rely on diﬀerent refer-

ence elements, interpolation orders, and expansion types. This is handled by con-

struction of a basis object-set Bwith each individual basis object, B, being deﬁned

as B={τ,{ϕ1,...,ϕ

K},Q}.Hereτdenotes the master (or reference) element,

{ϕ1,...,ϕ

K}is a set of K-basis functions used to construct ﬁnite-dimensional func-

tion spaces, and Q={(wp,ξp),p=1,...,N

p}deﬁnes the quadrature rule via the list

of weights wp∈R+and points ξp∈τ. The full list of available options is detailed in

Table 3.1. All bases deﬁned for a sp eciﬁc problem constitute the basis ob j e c t - s e t B.

A wide range of basis types, including modal [43], nodal [28, 88], Crouzeix–Raviart

[16], and bubble functions [28], can be written using the cardinal expansion. If Mis

the total number of unique polynomial terms of B,d=dim(τ), and Kis the number

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C157

Tab l e 3 . 1

Summary of available element/basis speciﬁcations in CHeart.Optionsareavailableinany

combinati on (ex cl ud in g poi nt e le me nt ty pes which cannot s up port i nt er pol at io n).

Element/basis speciﬁcations

Element types point, line, quadrilateral, triangle, tetrahedral, hexahedral, prism

Basis types nodal, continuous/discontinuous, order (0–5)

modal, continuous/discontinuous, order (1–15)

user speciﬁed‡

Quadrature schemes Gauss–Legendre, integrable order (0–60)

Keast–Lyness, integrable order (0–10)

‡:Userdeﬁnedbasisenablesusageofalternativebasisdeﬁnitions,includingbubbleandCrouzeix–

Raviart element types, which are supplied using a simple text ﬁle containing basis coeﬃcients and

powers (i.e., αand β).

of functions in the basis B,thenϕi(i=1,...,K) and its derivatives can be computed

as

ϕi(ξ)=

M

&

j=1

α(i, j)

d

'

k=1

ξβ(k,j)

k,(3.1)

∂ϕi(ξ)

∂ξn1

1···∂ξnd

d

=

M

&

j=1

α(i, j)

d

'

k=1

D(βkj ,n

k)ξβ(k,j)−nk

k,

where ξ=(ξ1,...,ξ

d)Tis a local coordinate in τ,α∈RK×Mis the sparse coeﬃ-

cient matrix, β∈Rd×Mis the corresponding powers in the cardinal expansion, and

D(p, n)=(n−1

k=0 (p−k),D(p, 0) = 1. As shown in (3.1), the derivatives can be auto-

matically evaluated and refactored as a new sparse cardinal expansion with adjusted

coeﬃcients. Here we note that for nodal Lagrange and modal basis functions the

number of terms in the cardinal expansion is equivalent to the number of basis func-

tions (i.e., K=M). However, in the case of bubble or Crouzeix–Raviart elements,

this is not the case.

The coeﬃcient matrix αmay either be user-supplied or constructed algorithmi-

cally. For example, for nodal Lagrange polynomials, α=A−Tis the inverse transpose

of the Vandermonde matrix (i.e., A(i, j)=(d

k=1(ξi

k)β(k,j)with ξ1,...,ξK∈τbeing

nodal coordinates).

Furthermore, tensor-based expansions can be written in a similar form,

ϕi(ξ)=

d

'

k=1

K1/d

&

j=1

α(γ(i, k),j)ξβ(j)

k,

∂ϕi(ξ)

∂ξn1

1···∂ξnd

d

=

d

'

k=1

K1/d

&

j=1

α(γ(i, k),j)D(β(j),n

k)ξβ(j)−nk

k,

where α∈RK×K1/d and β∈RK1/d are the 1D sparse expansions, and γ∈RK×d

is an index array deﬁning the corresponding 1D basis. For eﬃcient computation the

expansion for each basis function is written as a sum of the nonzero components of

α.

Due to the arbitrary degree of the basis, a range of quadrature schemes is required.

Schemes speciﬁcally for triangular and tetrahedral elements, such as those by Lyness

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

C158 LEE ET AL.

and Jespersen [54] and Keast [46], were integrated based on the source code provided

by [63]. Higher order Gauss–Legendre schemes are also available based on ﬁnding the

roots of the Jacobi polynomials [43], and using tensor product expansion/collapsed

expansions to integrate up to 60th order. In addition to these built-in bases, CHeart

supports arbitrary user-deﬁned bases and/or quadrature schemes to be provided via

a simple text ﬁle.

3.2. Topologies. Similar to the outlined basis object-set B,anobject-setof

topologies Tis constructed. A topology object T∈Tin CHeart is constructed

using a speciﬁc basis B∈Band provides the mapping between mesh element indices

and global basis function (node or mode) indices used to evaluate ﬁelds over elements.

Speciﬁcally, the topology T={E,B,N

E,N

N}is comprised of NEand NN, denoting

the total number of elements and global basis functions, respectively, Bbeing the

basis used to perform ﬁeld evaluations, and E∈NNE×Kthe element mapping such

that N=E(e, k)forsome e∈{1,...,N

E},k∈{1,...,K},N∈{1,...,N

N}.

3.3. Interfaces. By default, each topology T∈Tis deﬁned as a singular entity

with an unknown relation to others in the object-set. However, in many cases, such

as mixed formulations or multiphysics interactions, it is desirable for a well-deﬁned

relationship to exist between topologies, enabling the transfer of data from one to the

other. This may be accomplished by deﬁning an interface object Ii→j∈I(with I

being the interface object-set) which eﬀectively maps element indices and local element

coordinates from Tito Tj,Ti,T

j∈T. Each interface object Ii→j={Ti,T

j,ei,j ,Ii,j }

contains components to construct a mapping. Letting NE,i and NE,j be the numbers

of elements, and τiand τjthe master elements in Tiand Tj, respectively, then ei,j ∈

NNE,i deﬁnes the relationship between element indices in the two topologies, and

Ii,j ∈RNE,i×Kthe relationship between local coordinates. The mapping can be fully

described by the mapping ζi→j:{1,...,N

E,i}×τi→{1,...,N

E,j}×τj,

(3.2) ζi→j(e, ξ)=)ei,j (e),

K

&

m=1

Ii,j (e, m)ϕi,m(ξ)*=(e′,ξ′),

for some (e′,ξ′)∈{1,...,N

E,j}×τjand (e, ξ)∈{1,...,N

E,i}×τi.Themapping

ζi→jenables interpolation and evaluation of ﬁelds and ﬁeld derivatives as will be

discussed in section 3.4.

In CHeart all interfaces are assumed to be null mappings by default, i.e., no

mapping exists between topologies. Various kinds of interfaces may, subsequently, be

deﬁned. These are outlined below and illustrated in Figure 3.2.

Identity interface. The most commonly used interface in FEMs is the identity

interface, given in Deﬁnition 3.1 and shown in Figure 3.2a, which simply returns the

same element and local element coordinates. Here, the associated basis of each topol-

ogy must be compatible in that their respective master elements must be geometrically

equivalent and their number of elements identical.

Definition 3.1 (identity interface). Let Ti,T

j∈Twith NE,i =NE,j =NE

and τi=τj=τ.AninterfaceIi→j∈Iis called an identity interface if for any

(e, ξ)∈{1,...,N

E}×τthe mapping function ζi→j(e, ξ)=(e, ξ),whereζi→jis

deﬁned by (3.2).

This is often employed for classic mixed-element formulations, such as those used

in ﬂuid mechanics or solid mechanics formulations, where velocity/displacement share

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C159

Fig. 3.2.Types o f i nt e r fa ce s s up po rted in CHeart.(a) Identity interfaces between linear,

quadratic, and cubic quadrilateral meshes. (b) Injective interface. (c) Injective interface via an

intermediary. (d) An injectiv e interface between nonconforming meshe s through a conforming in-

termediary. (e) Adegenerateinterfacebetweena1Dvesselterminalandtheboundaryfaceofthe

cylinder.

the same mesh as pressure variables but use diﬀerent interpolations (i.e., diﬀerent

topologies).

Topology groups. As topologies linked by an identity interface share many

commonalities, it is often useful to deﬁne topology groups (see Deﬁnition 3.2), which

link these objects together. This is particularly convenient, as more complex interfaces

may then be deﬁned for one member of the topology group and used for all other group

topologies.

Definition 3.2 (topology group). Let Tg⊆T;thenTgis a topology group if

there exists an identity interface Ii→j∈Ifor every Ti,Tj∈T

g.Further,Tg(i)is

referred to as the topology group of Tiif Ti∈T

g(i).

Injective interface. In multiphysics simulations the interconnectivity of topolo-

gies is often more complex. For this injective interface, mappings may be introduced

which provide a continuous map between element/local coordinate data and topolo-

gies. This may be on a portion of a domain, as in FSI where exchange between the

ﬂuid and solid occurs on an interface. Deﬁnition 3.3 speciﬁes the details of injective

interfaces which are also illustrated graphically in Figure 3.2b–d.

Definition 3.3 (injective interface). Let Ti,T

j∈T.AninterfaceIi→j∈Iis

cal led injective if for every (e, ξ)∈{1,...,N

E,i}×τithere exists a unique (e′,ξ′)∈

{1,...,N

E,j}×τjsuch that ζi→j(e, ξ)=(e′,ξ′),whereζi→jis deﬁned by (3.2).

This type of interface provides a mapping for all element/local coordinate pairings

from Tiinto some or all pairings from Tj. As a result, the injective interface allows for

the embedding of reﬁned grids or lesser dimensional manifolds into another topology.

This formulation enables a wide range of associations between topologies. Injective

maps can be used to couple subsets of two topologies where only a portion of each is

interfaced by introducing an intermediary topology which is injective to both over the

desired subsets (see Figure 3.2c). This need arises in ﬂuid-solid coupling, for example,

where the interaction between ﬂuid and solid domains occurs on some portion of the

domain boundaries (see section 8.2). Further, injective interfaces can be easily ex-

tended to fully nonconforming grids by introducing an additional polygonal topology

nested within the intersections of the interfacing topologies (see Figure 3.2d).

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

C160 LEE ET AL.

An important property of injective interfaces is their transferability to the associ-

ated topology groups, which facilitates data transfer between single-physics problems

employing mixed interpolation spaces. As shown in Lemma 3.4, it is clear that an

injective interface Ii→j={Ti,T

j,ei,j ,Ii,j }can be extended to any Tk∈T

g(Ti) using

the mapping ζi→j.

Lemma 3.4. Suppose Ti,T

k∈T

g(i)and Ii→j∈Iis injective. Then, there is an

implicit mapping Ik→j∈Ithat is injective.

Proof. Since Ti,T

k∈T

g(i), then ζk→i(e, ξ)=(e, ξ)andIk→iis an identity in-

terface. The mapping for Ik→jcan therefore be constructed as ζk→j(e, ξ)=ζi→j◦

ζk→i(e, ξ)=ζi→j(e, ξ) for all (e, ξ)∈{1,...,N

Ek}×τk. Since the interface Ii→jis

an injective interface, Ik→jis also an injective interface.

An important subset of identity and injective interfaces includes those which

are complete (see Deﬁnition 3.5). By deﬁnition, all identity interfaces are complete.

However, in general, an injective interface enables linking one topology Tito a subset

of another Tj.Acompleteinterfaceensuresthatthissubsetextendstotheentirety

of Tj. This is particularly important for FEM weak form statements which must be

held over a deﬁned domain which may require interfaces to other variables.

Definition 3.5 (complete interface). An injective interface Ii→j∈Iis called

complet e if ζi→jis bijective.

If an injective interface Ii→j∈Iis deﬁned between two topologies Ti,T

j∈T,then

an inverse mapping can be implicitly evaluated and constructed (see Deﬁnition 3.6).

However, explicitly constructing Ij→iin the form shown in (3.2) is not, in general,

straightforward. Instead, the inverse of an injective interface is computed procedurally

by CHeart based on the provided injective interface.

Definition 3.6 (inverse interface). For an inj ect i v e interfa ce Ii→j∈Iwe may

implicitly deﬁne an inverse interface Ij→ithat for some (e′,ξ′)∈{1,...,N

E,j}×τj

there exists a unique (e, ξ)∈{1,...,N

E,i}×τisuch that ζi→j(e, ξ)−(e′,ξ′)=(0,0).

Degenerate interface. In certain cases, the requirements of injective interfaces

or their inverses can be too restrictive, requiring the construction of degenerate inter-

faces (see Deﬁnition 3.7), where a topology may be mapped nonuniquely. This case

arises when reduced-dimensional formulations are coupled to full-dimensional prob-

lems, such as 1D blood vessel ﬂow coupled to three-dimensional (3D) ﬂuid ﬂow (see

Figure 3.2e). Clearly, the reverse of this mapping is ill-posed and can only be obtained

in simpliﬁed instances.

Definition 3.7 (degenerate interface). An interface Ii→j={Ti,T

j,ei,j ,Ii,j }

is called degenerate if for some (e1,ξ1),(e2,ξ2)∈{1,...,N

E,i}×τithere exists a

single pair (e′,ξ′)∈{1,...,N

E,j}×τjsuch that ζi→j(e1,ξ2)=(e′,ξ′),ζi→j(e2,ξ2)=

(e′,ξ′).

An object-set of well-deﬁned interfaces plays a crucial role in reusability of

a single-physics code within coupled problems, as it enables the abstraction of

interpolation-speciﬁc implementations as well as cross-talk between diﬀerent com-

putational domains.

3.4. Variables and function spaces. Using the basis, topology, and interface

structures outlined, it is possible to formally construct interpolated variables as well

as discrete function spaces. As with previous structures, an object-set of variables V

may be deﬁned, enabling the construction or calculation of numerous quantities that

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C161

are either required for solving physical systems or desired as outputs. Each variable

object V∈Vis deﬁned as V={TV,CV},whereTV={EV,B

V,N

E,N

N}∈Tis the

topology, and CV∈Rn×NNthe coeﬃcient matrix, so that a ﬁnite-element variable

can be computed as

(3.3) ˆvh(e, ξ)=

n

&

l=1

K

&

k=1

CV(l, EV(e, k))ϕk(ξ)el,(e, ξ)∈{1,...,N

E}×τV,

where {ϕ1,...ϕ

K}∈BV,e1,...,en∈Rnare the unit base vectors for Rn.Herewe

add a hat (ˆ), distinguishing the variable from those introduced in section 2, as it is

principally a function of the element index and master element coordinate. All ﬁelds

in CHeart, including the mesh coordinates, are speciﬁed in this way, enabling the use

of any topology deﬁned for these ﬁelds. In the case of spatial domains, this ﬂexibility

allows for the construction of standard simplex and hexahedral domains as well as

the use of curvilinear grids. We assume each domain variable D={CD,T

D}∈V

is well constructed (i.e., continuous, nonoverlapping, locally invertible), enabling the

construction of a computational domain,

(3.4) Ωh:= +x∈Rd,,x=ˆx(e, ξ)∀(e, ξ)∈{1,...,N

E}×τD-,

where the deﬁnition of ˆx follows from (3.3) using the appropriate domain topology

TD(and its components) as well as the scaling array CD.

However, unlike standard variables, domains have additional properties enabling

them to represent diﬀerent spatial reference frames, for example, Eulerian/Lagrangian

[55] or ALE [21]. This is accomplished by allowing the domain Ωhto be considered

as a function of time Ωh(t) where the spatial positions of x∈Ωhare summed with

an additional variable V={TV,CV}∈Vto give the current x(t)∈Ωh(t), i.e.,

x(t)=ˆx(e, ξ,t)=

d

#

l=1 !KD

#

k=1

CD(l, ED(e, k)"ϕD,k(ξ)+

KV

#

k=1

CV(l, EV(e, k))$

$tϕV,k(ξ))el,

where {ϕm,1,...,ϕ

m,Km}∈Bmfor m={D, V }.WenotethatID→Vis assumed to

be an identity interface to enable summing.

Vari a b l e- s p a c e pai r i n g. While (3.3) outlines the nth-dimensional vector func-

tion ˆvhas a function of local element index and local coordinates, it is desirable

in FEMs to instead deﬁne vh:Ω

h→Rnas a function of the spatial domain Ωh.

Whether this deﬁnition is possible depends on if the variable-space pairing (ˆvh,Ωh)

is well deﬁned.

Definition 3.8. Avariable-spacepairing(ˆvh,Ωh)(deﬁned in (3.3) and (3.4))

is well deﬁned if the spatial domain Ωhis well constructed (i.e., continuous, non-

overlapping, locally invertible), and there exists a topology Tr∈Tand interfaces

Ir→D,I

r→V∈I,whereIr→Dis complete and Ir→Vis an identity or injective inter-

face. In this case, Tris called a root topology.

The stipulations of Deﬁnition 3.8 ensure that for each x∈Ωhthere exists a unique

corresponding pair (e, ξ)∈{1,...,N

Er}×τr. The existence of a root topology Tras

outlined implies that the composition

(3.5) vh(x)=ˆvh◦ζr→V(e, ξ),x=ˆx ◦ζr→D(e, ξ)

is well deﬁned for all x∈Ωh. As a result, Trprovides a topology for which vhand

Ωhcan be related. The completeness requirement for Ir→Ωensures that each xhas

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

C162 LEE ET AL.

acorrespondingvh;however,Ir→Vneed not be complete, allowing for conditions or

weak equations on subsets of a variable ﬁeld (i.e., use of constraints along boundaries

or interior regions).

Discrete function spaces. Using the deﬁnition of discrete variables as shown

in (3.3), and the concept of variable-space pairings introduced in Deﬁnition 3.8 and

(3.5), it is natural to deﬁne the discrete function spaces introduced in (2.1) as

Vh(Ωh)=.vh∈[L2(Ωh)]n,,vh(x)=ˆvh◦ζr→V(e, ξ),x=ˆx ◦ζr→D(e, ξ)

∀(e, ξ)∈{1,...,N

Er}×τr,∀CV∈Rn×NN/.

The properties of the discrete space Vh(Ωh), such as the smoothness and order, depend

on the underlying connectivity of TVgiven by EV, the choice of basis, as well as the

spatial domain over which the space is deﬁned.

3.5. Expressions. Often during a modeling process, it is desirable to conduct

algebraic computations with the variables deﬁned over the ﬁnite-element meshes at

runtime. This functionality is implemented in CHeart through an object-set of plain-

text mathematical expressions provided at problem speciﬁcation time. The expres-

sions can take variable pointers, time, and other expressions as inputs and produce

outputs that can be assigned to set boundary conditions, material and source ﬁelds,

and other variables as part of partitioned solver strategies and postprocessing. The

parsing of expressions is implemented using an internal bytecode-compiler.2Based

on an algebraic expression provided by the user, the internal compiler decomposes

the string into its component operators which are then stored as a sequence of simple

bytecode instructions. This conversion occurs only once, when the problem is set up,

and so in this way the expressions can be evaluated at key points in the computation

phase of execution with a minimal amount of runtime overhead. The user can specify

both scalar and vector expressions that contain standard algebraic and trigonomet-

ric functions as well as logical and conditional operators, modulo max/min and data

interpolation.

3.6. General weak form systems. Multiphysics systems are comprised of mul-

tiple weak form operators which act on diﬀerent variables. These operators, outlined

for speciﬁc physics in section 2, are constructed as needed and built into an object-set

F,whereeachoperatorf∈Fis deﬁned as in (2.1).

Remark 3.9. In the case of Lagrangian or arbitrary Lagrangian–Eulerian (ALE)

domains, functionals can be deﬁned as

f(vh,yh;Ω

h(t)) = !Ωh(t)

a(vh)·yh+c(vh):∇yhdΩh,

enabling deﬁnition of the weak form on a speciﬁc reference frame. In the case of

conservative ALE formulations [25, 70],

f(vh,yh;Ω

h)=!Ωh(t+δt)

a0(vh)·yhdΩh−!Ωh(t)

a1(vh)·yhdΩh

+!t+δt

t!Ωh(s)

a2(vh)·yh+c(vh):∇yhdΩhds

2http://fparser.sourcefor ge.net/

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C163

is used with functionals a0,a1,anda2.

Remark 3.10. Boundary conditions may be introduced to operators f∈Fas

discussed in section 4.2.

Each operator f∈Fmay be combined with others to form multiphysics systems.

This is done by deﬁning the summed operator f⊆F,

(3.6) f(vh,yh) := &

k

fk(vh,k,yh,k ;Ω

h,k),f

k∈F,

where fkand Ωh,k are the kth operator/domain and (vh,k ,yh,k) its respective variables

(where vh,k ∈vhand yh,k ∈yh). Importantly, the operator fand its variables,

vh,yh, do not reside on a single domain. This enables the merger of operators which

span diﬀerent Ωh,k,asoftenoccursinmultiphysics systems. Each operator fdenotes

aspeciﬁcsystemforwhichwelooktoﬁndvh∈Vhso that

(3.7) f(vh,yh)=0 ∀yh∈Yh,

where Vh,Yhdenote the complete trial/test spaces.

4. Physics coupling. In sections 2 and 3 we introduced the formulation of

single-physics operators as well as their construction through a series of object-sets.

To further enable ﬂexible multiphysics integration, this section discusses a series of

coupling strategies developed to formally link operators (as shown in (3.7)) which

represent multiphysics weak form systems. Here we deﬁne a coupling as a functional

relationship between single-physics systems within a problem, the examples of which

can be found in Table 4.1. While this may occur in numerous instances, here we

identify three principal mechanisms:

(1) functional dependence of existing operators on evolving variables from an-

other problem,

(2) functional dependence of boundary conditions on evolving variables from an-

other problem, and

(3) new coupling operators which eﬀectively link problems together.

These approaches are all made possible using diﬀerent couplings methods.

4.1. Coupling via existing operators. As full access to all variables in V

is available to any problem, the inclusion of variables through existing operators is

straightforward. For example, the diﬀusion coeﬃcient in a scalar transport problem

may be linked to any variable in V, including one which is solved for within a diﬀerent

system (e.g., as is the case in strongly coupled electromechanics). This enables the

automatic coupling of systems. Further, as diﬀerent solve strategies may be applied as

discussed in section 5.5, CHeart enables this coupling to be either explicit or handled

through a ﬁxed point iteration.

4.2. Coupling via boundary conditions. Each operator (see section 3.6) may

be linked to any number of boundary conditions. Steady or time-dependent Dirichlet

or Neumann boundary conditions may be applied to diﬀerent portions of the domain

boundary (typically a series of user-deﬁned nodes or element faces).

CHeart boundary conditions may be speciﬁed using variables as well as numeric

expressions, so that the solution of one problem may be provided as the boundary

data for another problem. This is often required in partitioned ALE Navier–Stokes

where the domain motion can be computed independently from the ﬂow while the two

problems are coupled along portions of the domain boundary. This type of coupling is

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

C164 LEE ET AL.

Tab l e 4 . 1

Summary of available coupling options for diﬀerent styles of multiphysics.

Physics coupling operators/options

Simple coupling Coupling via share d variables

Coupling via algebraic expressions

Coupling via b oundary conditions

Lagrange coupling Coupling one or m ore variables to a constraint

Coupling variable or variable rate, i.e., Du=uor Du=∂u/∂t

Coupling all components or norm al components, i.e., L(u)=uor L(u)=u·n

User deﬁned constraint expressions available

Support for constraints on 1D, 2D, and 3D manifolds or volumes

Volume coupling Linear coupling through reaction terms

Support for 2D, 3D

also common in FSI, where the motion of the interface between ﬂuid and solid domains

is set as a Dirichlet condition and the traction on ﬂuid is applied as a Neumann-type

condition to the solid.

4.3. Coupling via expressions. While the integration of multiple physics us-

ing variables provides a powerful tool for coupling systems, this is not always suﬃcient

as the coupling may require algebraic or diﬀerential manipulation of a variable. This

feature is enabled in CHeart via expression strings (see section 3.4) which may be

provided by users to deﬁne the speciﬁc mathematical form of coupling.

In many cases, such as with boundary conditions, expressions and variables may

be used interchangeably. In addition, CHeart allows the casting of expressions to

variables, which enables the coupling of certain operations which would otherwise

need to be hardcoded.

4.4. Monolithic integration via coupling operators or Lagrange multi-

pliers. Multiphysics integration often necessitates the inclusion of additional terms

or constraints in the system. In some cases, as in partitioned approaches, this merely

involves adding a source or augmenting boundary conditions. In other approaches,

the coupling may be imposed via cou pl ing o perat ors ,orbytheintroductionofLa-

grange multipliers. In these cases, the additional elements are considered as separate

physical problems which augment traditional single-physics examples. The modular

implementation of these couplin g problems in CHeart promotes code reuse in various

types of multiphysics problems.

Coupling operators. In CHeart coupling operators provide a link between

variables from distinct physics problems through the addition of new terms or opera-

tors into the weak form. Considering a set of variables (u1,...,um)∈Uh

1×···×Uh

m

which have been introduced for a set of physics problems, the general form of ain

equation (2.1) for coupling operators is given by

a(u1,...,um) := $m

&

k=1

γ1kL1kuk, ...,

m

&

k=1

γmkLmk uk%,

where γik is a scalar coeﬃcient detailing the coupling between uiand ukand Lik

the corresponding linear operator (see Table 4.1). As a result, the weak form deﬁned

for uiis augmented with 0m

k=1 γikLik uk,whichmaybetailoredtoaddressaspeciﬁc

type of coupling. One classical example of the use of coupling operators is in Boussi-

nesq convection, where the Navier–Stokes equations are coupled to advection-diﬀusion

through the advective velocity ﬁeld and a temperature-based forcing term.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C165

Lagrange multipliers. Lagrange multipliers are typically used for the addition

of new constraint equations which may augment a single-physics system or link mul-

tiple systems. Speciﬁcally, an additional variable λ∈Mhis added to the set of

variables (u1,...,um) in order to weakly impose over some domain or manifold, Ωh,

that the variables satisfy

&

k

wkLk(Dkuk)−s=0,

where Lkand Dkare the kth linear algebraic and diﬀerential operators, respectively

(see Table 4.1), wkis the kth scalar weight, and sis some provided source. The

additional constraint terms can be included using the operator aof (2.1) deﬁned as

a(u1,...,um,λ) := $w1L1(λ),...,w

mLm(λ),&

k

wkLk(Dkuk)−s%,

where (u1,...,um,λ)∈Uh

1×···×Uh

m×Mh.Thisenablesabroadrangeofcoupling

using multipliers and may also be used to construct projection operators for iterative

ﬁxed point schemes. Examples of this approach are found in the imposition of complex

physical constraints as well as ﬂuid-solid coupling.

5. Assembly, matrix operations, and building. Eﬃcient and ﬂexible ma-

trix assembly operations are a principal enabler of multiphysics integration in ﬁnite-

element modeling and a key engine driving its computational performance. The core

CHeart library provides automated functionalities and standardized interfaces to en-

able rapid and extensible developments. The process by which the topology group,

equation map, and variable are used to build and populate sparse matrices is described

below.

5.1. FEM element assembly. FEM element assembly of residuals, matri-

ces, and other objects (see section 3) is usually handled by an element loop over

amesh. Inthecode,CHeart contains a single element loop routine, using a series of

procedure pointers—Setup,LocalElementCalculate,LocalElementAssemble,and

CleanUp procedures—to control the tasks executed. These procedure pointers may

be linked (or nulliﬁed) before execution of the element loop and altered accord-

ing to the desired assembly task (i.e., residual evaluation, Jacobian construction,

norm calculations, etc.). Setup and CleanUp procedures are commonly used to ex-

tract tasks which need not be repeated during the element loop—such as dynami-

cally allocating/deallocating memory or constructing/destroying access shortcuts to

data required for FEM assembly—to optimize computational performance. The

Calculation and Set routines are established to do tasks which may vary by ele-

ment, with the Calculation step involving the computation of a physics-dependent

element-level matrix or residual, for example. The Set step is a low-level, abstracted

task used to incorporate element-level data into global matrix structures.

Beyond the standard steps followed in FEM assembly, the presence of merged

functionals and interface mappings necessitates important extensions. As the weak

form functional for a physical system may be a sum of multiple physical systems

(see (3.6)), the general assembly occurs over each individual functional fk∈f,for

which we deﬁne a symbolic variable dependence using equation maps, E(fk). Cru-

cially, this equation map contains a root topology Tr, over which element-level proce-

dures may be computed.

Since integral evaluations of weak form equations (see (2.1)) are computed over

Tr, all other topologies used as part of the problem governed by fkmust be evaluated

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

C166 LEE ET AL.

on Tr. This list may be automatically compiled by recording the usage of various

topologies as problems are deﬁned by the user. The completeness of Trensures the

looping process covers the indicated spatial domain.

In addition, the corresponding basis functions and their gradients of all involved

terms must be evaluated on Tr.IfIr→kfor Tk∈Tis an identity interface (see

Deﬁnition 3.1), no mapping is required, and pre-evaluations of the basis functions

(and their derivatives) can be used directly. However, when Ir→kis a more complex

mapping, such as an injective interface, then the basis on Tkmust be mapped onto

Tr(see (3.6)). Users can elect to either execute these computations on the ﬂy or have

the program precompute these quantities into fast access cached lists.

Algorithm (FEM LOOP)

for all fk∈f

call SetupProcedure

for all e∈Tr⇐E(fk)

for all Ti∈{Tk}

Compute needed ϕ,∇rϕ, . . . at Qrfor all ϕ∈Bi

end for

call LocalElementCalculateProcedure

call LocalElementAssembleProcedure

end for

call CleanUpProcedure

end for

5.2. Equation mappings. The translation of weak form operators finto dis-

crete ﬁnite-element equivalents may be achieved via the use of equation maps (Deﬁni-

tion 5.1). An equation map is an abstraction of the dependence of weak form operators

on their spaces, which completely encapsulates the row-column variable mappings

necessary to construct the local element tensors. Considering (2.2), the equation

mapping deﬁnes whether f(v′

h,y′

h;Ω

h)̸=0forsomev′

h=(0,...,vh,i ,...,0)and

y′

h=(0,...,yh,j ,...,0)withvh,i ∈Vh

iand yh,j ∈Yh

j, that is, whether there are

operators in fwhich pair the ith and jth trial and test spaces.

Definition 5.1 (equation map). Let {Vh

1,...,Vh

m}and {Yh

1,...,Yh

n}be well-

deﬁned function spaces on Ωh.Then,iffis a functional form as shown in (2.2) on

the discrete ﬁnite-element spaces, the equation map E(f)of fis deﬁned for each row

variable i∈{1,...,n}by Ei(f)where

Ei(f) := {j∈{1,...,m}|f(vh,yh;Ω

h)̸=0 for some (vh,yh)∈Vh

j×Yh

i},

denoting the symbolic nonzero dependence of terms in fon {Vh

1,...,Vh

m}×{Yh

1,...,Yh

n}.

Remark 5.2. In Deﬁnition 5.1, the equation map E(f) implies that the operator,

f,operatesonwell-deﬁnedspacesonΩ

h(with topology TD∈T). Hence, there is

a root topology, Tr, which links all topologies of variables in E(f). That is, Ir→Dis

complete and Ir→Kis nonnull for each Tk⇐E(f).

Encoding the operators of f,theequationmapE(f) enables the generalization

of many lower-level routines and operations which require knowledge of the weak

form structure, such as linearized system assembly, merger, etc. In addition to the

symbolic dependence outlined, the equation map E(f)offalso labels the structure

of each term as identity,trace,orfull. This relates to the pairing of vector variables,

which, due to operators, may share components in diﬀerent ways (see Deﬁnition 5.3).

For example, the standard inner-product of two vector variables may be classiﬁed as

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C167

v

p

u

q

v

u

λ

v

p

u

q

λ

Fluid mechanics

Solid mechanics

Lagrange multiplier coupling Monolithic cou

p

led s

y

stem

Fig. 5.1.Illustration of three separate equation maps—incompressible Navier–Stokes with ve-

locity and pressure variables (v,p),incompressiblesolidmechanicswithdisplacementandpressure

variables (u,q),andLagrangemultipliercoupling,λ,between(v,u)—combined into a single equa-

tion map to form the monolithically coupled system used in FSI; see section 8.2.

having trace structure, while the standard inner-product of two scalar variables is a

full structure operator.

Definition 5.3 (equation map structure). Let E(f)be the equation map of the

operator fgiven in Deﬁnition 5.1.ConsideringthevariablepairingVj×Yi,forsome

j∈Ei(f),supposeeachvh∈Vh

jand yh∈Yh

itakes vh:Ω

h→Rsand yh:Ω

h→Rr.

Letting vn,k

h∈Vh

jdenote the nth basis function in the kth component and, similarly,

letting ym,p

h∈Yh

idenote the mth basis function in the pth component, the structure

of the variable pairing is deﬁned as

(Identity) if f(vn,k

h,ym,p

h)̸=0for some n=m, k =p,

(Trace) if f(vn,k

h,ym,p

h)̸=0for some n, m, k =p,

(Full) if f(vn,k

h,ym,p

h)̸=0for some n, k , m, p,

where the least generic structure applicable is selected for the speciﬁc functional f.

While the equation mapping of an operator enables the construction of a generic

template for FEM calculations, its dependence on a single domain, Ωh, is limiting,

particularly in multiphysics contexts where multiple domains are linked together. As

aresult,themultiphysicsoperatorfis enabled through the formation of equation

map sets where the ﬁnal structure is the union of each equation map automatically

evaluated from the individual maps provided, i.e., E(f) := 1kE(fk). An example of

this process is illustrated in Figure 5.1. In these cases, operations are done on each

individual equation map, E(fk), and their results are combined to emulate the ﬁnal

mapping, E(f).

5.3. Matrix assembly and monolithic integration. The ﬁnal weak form

system shown in (3.7) may be solved in a variety of ways. This is achieved, in part,

by allowing single-physics or other operators to be integrated into a single monolithic

problem which may be solved implicitly. Importantly, not all operators of the total

problem fneed be monolithic, thereby allowing partitioned solving of some/all physics

systems.

For problems with monolithic integration, their implicit solution requires the for-

mulation of a Jacobian matrix (or some approximation to it) which may be inverted

or iteratively solved to give the solution or a solution update (as in Newton–Raphson

methods [57, 32]). Similar to other structures, we construct an object-set Mof ma-

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

C168 LEE ET AL.

trices. Each matrix, M∈M, is assembled on an underlying equation mapping E(f)

which may be composed of a simple physical system or a more complex multiphysics

system spanning multiple domains and variables. From the equation map deﬁnition,

we seek to assemble Min compressed sparse row (CSR) format [27] by determining

the connectivity between a series of rows and columns. In this sense, the matrix,

(5.1) M={Vr,Vc,E},

is a graph with row/column vertices (Vrand Vc)andedges(E)whichprovidetheir

connectivity. In the case of structurally simple equation maps—such as that which

arises from scalar Laplace—the matrix construction process is straightforward. In

this case, if the solution variable is U={U,T}∈Vwith NNnodes/modes, then by

computing the inverse mapping of the topology array E∈T(the node-to-element

map), E−1(n)={e∈{1,...,N

E}| n=E(e, k),for some k∈{1,...,K}},n ∈

{1,...,N

N}one can eﬀectively determine the interdependence of nodes/modes on

elements. Subsequently, the inverse map can be used to directly compute the matrix

graph M={Vr,Vc,E}.HereVr=Vcis deﬁned so that, for n∈{1,...,N

N},

Vr(U, n)=nand

(5.2)

E(n)={m∈{1,...,N

N}| m=E(e, k)forsomek∈{1,...,K}and e∈E−1(n)}.

Remark 5.4. In the case of discontinuous Galerkin assembly, ˜

E−1is deﬁned and

used in place of E−1in (5.2) with ˜

E−1(n)={e∈{1,...,N

E}|n=E(el,k)forsome

k∈{1,...,K},and el∈N(e)},n ∈{1,...,N

N},and N(e)denotingthesetof

neighboring element indices of eand eitself.

In a multiphysics environment, this approach must be adapted to deal with ad-

ditional (potentially multidimensional) variables, multiple equation maps (i.e., f=

{fk}), and complex interfaces. However, the construction of more complex linear

systems can be eﬃciently managed using both the equation mapping E(f) as well as

the underlying interface mappings, I.E(f)providesdetailsofhowthesetofi-row

variables, Y={Y1,...,Y

i}⊆V, as well as j-column variables, V={V1,...,V

j}⊆V,

is interrelated. Using this data, we may construct a variable-matrix mapping for both

rows and colums. That is, for any Y={C,T}∈Yand n∈{1,...N

N},wedeﬁnethe

row mapping Vr(Y,n)=R, R ∈{1,...,N

Vr},whereNVris the total number of rows

(and similarly for the column mapping). We note that, in this case, Ris uniquely

paired to Yand n. This deﬁnition eﬀectively orders how nodes/modes associated

with speciﬁc variable spaces are mapped to the linear system.

Remark 5.5. Construction of Vrand Vcmappings follows element clustering,

where all variables are ordered according to elements, or variable clustering, where

each variable is ordered sequentially, depending on the parallelization strategy.

Subsequently, assembly is managed by constructing the matrix connectivity, E,as

seen in Deﬁnition 5.6. Here, the connection between row and column variables is ex-

tracted by ﬁrst operating on each equation mapping E(fk) resulting from functionals

fk∈f.ThisprovidesalinktotheroottopologyTrkover which all row and column

variables of E(fk) are related. Each element of the root topology Trkthen acts as

the common point, enabling mapping through the deﬁned interfaces to determine the

corresponding elements in the topology used by all variables in the mapping. By col-

lating these nodes/modes, it is then possible to construct the collective connectivity,

E, as given in Deﬁnition 5.6 and illustrated in Figure 5.2.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

MULTIPHYSICS COMPUTING IN CHeart C169

Fig. 5.2.An il lustration of mon olit hic matrix assemb ly procedure. Top left: T wo topologies

(quadrilateral and triangle types) with node numbers indicated. The equation maps for the individual

systems are E(f1(U)) and E(f2(V)).Whenthetwosystemsarecoupled,thecouplingoperator

provides the additional functionals, f3,andequationmaps,E(f3),linkingthevariables. Following

the formation of the monolithic E(f),thematrixconnectivityEcan be dete rm in ed thro