ArticlePDF Available

Manifold embedding data-driven mechanics

May 2022
Journal of the Mechanics and Physics of Solids 166(1)

DOI:10.1016/j.jmps.2022.104927

Authors:

Bahador Bahmani

Hopkins Extreme Materials Institute, Johns Hopkins University

Waiching Sun

Columbia University

This article introduces a manifold embedding data-driven paradigm to solve small-and finite-strain elasticity problems without a conventional constitutive law. This formulation follows the classical 6 data-driven paradigm by seeking the solution that obeys the balance of linear momentum and compatibility conditions, while remaining consistent with the material data through minimizing a distance measure. Our key point of departure is the introduction of a global manifold embedding as a means to learn the geometrical trend of the constitutive data mathematically represented by a smooth manifold. By training an invertible neural network to embed the data of an underlying constitutive manifold onto a Euclidean space, we reformulate the local distance-minimization problem that requires a computationally intensive combinatorial search to identify the optimal data points closest to the conservation law with a cost-efficient projection step. Meanwhile, numerical experiments performed on path-independent elastic materials of different material symmetries suggest that the geometrical inductive bias learned by the neural network is helpful to ensure more consistent predictions when dealing with data sets of limited sizes or those with missing data.

Content uploaded by Waiching Sun

Content may be subject to copyright.

Journal of Mechanics and Physics of Solids manuscript No.

(will be inserted by the editor)

Manifold embedding data-driven mechanics1

Bahador Bahmani ·WaiChing Sun2

Received: May 4, 2022/ Accepted: date4

Abstract This article introduces a manifold embedding data-driven paradigm to solve small- and ﬁnite-5

strain elasticity problems without a conventional constitutive law. This formulation follows the classical6

data-driven paradigm by seeking the solution that obeys the balance of linear momentum and compatibil-7

ity conditions while remaining consistent with the material data through minimizing a distance measure.8

Our key point of departure is the introduction of a global manifold embedding as a means to learn the9

geometrical trend of the constitutive data mathematically represented by a smooth manifold. By training10

an invertible neural network to embed the data of an underlying constitutive manifold onto a Euclidean11

space, we reformulate the local distance-minimization problem that requires a computationally intensive12

combinatorial search to identify the optimal data points closest to the conservation law with a cost-efﬁcient13

projection step. Meanwhile, numerical experiments performed on path-independent elastic materials of14

different material symmetries suggest that the geometrical inductive bias learned by the neural network15

is helpful to ensure more consistent predictions when dealing with data sets of limited sizes or those with16

missing data.17

Keywords data-driven mechanics, manifold learning, geodesic, constitutive manifold18

1 Introduction19

Conventional computer simulations of mechanics problems often employ discretizations to convert bound-20

ary value problems into systems of equations and obtain the discretized solution from a solver. The bound-21

ary value problems are often split into two components, i.e., the constraints for the solution space-time22

domain that often derives from balance principles, and constitutive models that relate physical quantities23

derived from the solutions based on a combination of phenomenological observations and constraints de-24

rived from thermodynamics. Examples of these constitutive models are abundant in the literature of the25

last few centuries [Timoshenko,1983] even before the development of numerical methods for boundary26

value problems. Recently, the interest in deep learning has sparked a renewed interest in training neural27

networks to generate constitutive responses, (an idea that can track back to the last AI winter [Ghaboussi28

et al.,1991]) or replace classical solvers to generate numerical solutions of boundary value problems [Raissi29

et al.,2019].30

Kirchdoerfer and Ortiz [2016] introduce a new paradigm in which the modeling of constitutive re-31

sponses can be bypassed. Instead, a solver can be established such that one may ﬁnd the data points in a32

material database that is closest to the manifold that obeys the conservation principle. The upshot of this33

data-driven approach is that it does not require ﬁnding the functional forms of the constitutive laws which34

Corresponding author: WaiChing Sun, PhD

Associate Professor, Department of Civil Engineering and Engineering Mechanics, Columbia University , 614 SW Mudd, Mail

Code: 4709, New York, NY 10027 Tel.: 212-854-3143, Fax: 212-854-6267, E-mail: wsun@columbia.edu

ii Bahador Bahmani, WaiChing Sun

could be an ambiguous, ad-hoc, and time-consuming process [Cranmer et al.,2020]. Furthermore, bypass-35

ing the use of constitutive laws also eliminates the calibration process. For materials that exhibit complex36

physics and/or do not possess symmetry, it becomes necessary to introduce more material parameters in37

order to predict the material responses with sufﬁcient precision [Ehlers and Scholz,2007,Wang et al.,2016,38

Liu et al.,2016]. On-the-ﬂy multiscale calculations, such as hierarchical upscaling methods (e.g., FEM2)39

[Wang and Sun,2016,Matouˇ

s et al.,2017] or concurrent multiscale domain coupling methods [Hughes40

et al.,1998,Sun and Mota,2014,Sun et al.,2017,Badia et al.,2008] may eliminate the need to compose con-41

stitutive laws that captures the responses of the effective medium through computational homogenization.42

However, the computational cost of these multiscale techniques remains a major technical barrier despite43

years of research progress [Wang and Sun,2018,Karapiperis et al.,2021].44

The removal of the constitutive laws, therefore, brings in a new paradigm where the universal princi-45

ples are enforced and the solutions are sought by ﬁnding the data points that are closest to the conservation46

manifold. However, one important question central to the success of this paradigm is the following– how47

to properly measure the ”distance” between the material database and conservation manifold?48

Interestingly, there have been very few research on the choice of the distance measure for the data-49

driven/model-free approach. In data-driven mechanics research, such as Kirchdoerfer and Ortiz [2016]50

and Kirchdoerfer and Ortiz [2018], the independent components of the input-output pairs (e.g., the strain-51

stress pair) of the constitutive laws are often used to constitute a product space, and the measure of distance52

relies on the equipped weighted energy norms. This idea is utilized not only for elasticity problems but53

also for multiphysics (e.g., Bahmani and Sun [2021a] and higher-order continua (e.g., Karapiperis et al.54

[2021]). Noticeably, Leygue et al. [2018] compare simulations results obtained from different choices of55

distance measure and demonstrate a dependence between the energy-minimized solutions and the choice56

of the measure (see Fig. 6 of Leygue et al. [2018]). The underlying issue is not necessarily the choice of57

the distance measure but on the discrete nature of data point clouds. Without any prior assumption on the58

intrinsic properties of the geometry, deﬁning a unique length or distance between two data points becomes59

impossible. For instance, the distance between two points on a ﬂat surface and that on a curved surface60

are different. Assuming that the data are on a Euclidean space such as R12 for stress-strain data, could61

be feasible when the data distributed in the phase space is sufﬁciently dense or the data set itself is close62

to linear (and hence the difference between the length measured from the local tangential space and the63

constitutive manifold is minor). However, these conditions are not necessarily realistic.64

There have already been efforts that introduce resolutions to circumvent this spurious dependence. A65

common approach is to introduce locally linear embedding.InIba˜

nez et al. [2017], for instance, the data points66

used to measure the distance are factored by a weighting function such that each local linear patch around67

a data point can be mapped onto a lower-dimensional embedding space. In He and Chen [2020], locally68

convex material manifolds are also introduced to achieve locally linear exactness with dimensional reduc-69

tions. Meanwhile, Kanno [2021] introduce a kernel-based method to extract the manifold of the constitutive70

responses globally. By extracting the global constitutive manifold where all the data points belong, a metric71

can be established via embedding. However, like the original data-driven approach where the computa-72

tional cost may become signiﬁcantly higher with an increasing number of data points, the complexity of73

the kernel method scales with the number of data points and hence can be expensive when handling a74

sufﬁciently large database even the dimensions of the data is relatively low (e.g., 1D constitutive laws).75

In this paper, we introduce a neural network based global manifold embedding technique such that76

a proper distance measure corresponding to the manifold of the constitutive responses can be introduced77

for the data-driven mechanics simulations for path-independent materials. Our focus is, therefore, on the78

constitutive laws that exhibit no path dependence (e.g., elasticity and nonlinear heat transfer). Applying79

the global embedding technique for path-dependent materials is potentially feasible but is out of the scope80

of this paper.81

To achieve this goal, we consider the cases where we have collected data points from experiments or82

direct numerical simulations. We then train an invertible neural network such that it may map all data83

points on the constitutive manifold of path-independent materials onto a Euclidean space equipped with84

a Euclidean norm.85

As illustrated in Figure 1, The distance-minimization algorithm iterates between two steps: ﬁrst, a86

global-to-local projection PG7→Lfrom the equilibrium state (see Def. 8)z∈ C (Step 1 ) in (a) to the cor-87

Manifold embedding data-driven mechanics iii

<latexit sha1_base64="rG82jm8Ksz+eBsF/qzzwL12+2dM=">AAACAXicdVBJS8NAGJ3UrdYt6kXwMlgETyGJoctFCoJ4rGBboQllMp20QycLMxOhhHjxr3jxoIhX/4U3/42TtoKKPhh4vPdt8/yEUSFN80MrLS2vrK6V1ysbm1vbO/ruXlfEKcekg2MW8xsfCcJoRDqSSkZuEk5Q6DPS8yfnhd+7JVzQOLqW04R4IRpFNKAYSSUN9AN3NiPzWUpy6IZIjjFi2UU+0Kum0WzUbKcGTcM065ZtFcSuO6cOtJRSoAoWaA/0d3cY4zQkkcQMCdG3zER6GeKSYkbyipsKkiA8QSPSVzRCIRFeNtuew2OlDGEQc/UiCWfq944MhUJMQ19VFieK314h/uX1Uxk0vIxGSSpJhOeLgpRBGcMiDjiknGDJpoogzKm6FeIx4ghLFVpFhfD1U/g/6dqG5RjNK6faOlvEUQaH4AicAAvUQQtcgjboAAzuwAN4As/avfaovWiv89KStujZBz+gvX0CfKeXlA==</latexit>

G=F1

<latexit sha1_base64="Yj6Z2939xyUWXbzy5bQzYwU8rK8=">AAACInicdVBLSwMxGMzWV62vVY9egkXw4rJblz4OSkFQjxXsA9q1ZNO0Dc0+SLJCWfa3ePGvePGgqCfBH2O2rVJFBwLDzPclk3FDRoU0zXcts7C4tLySXc2trW9sbunbOw0RRByTOg5YwFsuEoRRn9QllYy0Qk6Q5zLSdEdnqd+8JVzQwL+W45A4Hhr4tE8xkkrq6pXO5I7YZQiPEtjxkBxixOKLBJ7Aby8ic9Z5chMfWUlXz5tGpVws2EVoGqZZsgpWSgol+9iGllJS5MEMta7+2ukFOPKILzFDQrQtM5ROjLikmJEk14kECVUKNCBtRX3kEeHEkwgJPFBKD/YDro4v4USd34iRJ8TYc9VkmlP89lLxL68dyX7ZiakfRpL4ePpQP2JQBjDtC/YoJ1iysSIIc6qyQjxEHGGpWs2pEr5+Cv8njYJh2Ublys5XT2d1ZMEe2AeHwAIlUAWXoAbqAIM78ACewLN2rz1qL9rbdDSjzXZ2wQ9oH5/LSqUU</latexit>

P(z#)

<latexit sha1_base64="V5PovuzIKz3k8MDWTVkpTGoeuK4=">AAACDHicdVDLSgMxFM34rPVVdekmWIS6GTKlo3UjBTcuK9gHdGpJ0rQNzTxIMkId5gPc+CtuXCji1g9w59+YaSuo6IHA4dxzHzkkElxphD6shcWl5ZXV3Fp+fWNza7uws9tUYSwpa9BQhLJNsGKCB6yhuRasHUmGfSJYi4zPs3rrhknFw+BKTyLW9fEw4ANOsTZSr1D0fKxHhCT11JtOS4iIWVrySCj6yW167RWPjAvZ6NitOFWIbBc5Vcc1pOw6CJWhY6MpimCOeq/w7vVDGvss0FRgpToOinQ3wVJzKlia92LFIkzHeMg6hgbYZ6qbTNen8NAofTgIpXmBhlP1e0eCfaUmPjHO7HT1u5aJf9U6sR5UuwkPolizgM4WDWIBdQizZGCfS0a1mBiCqeTmVkhHWGKqTX55E8LXT+H/pFm2nYp9elkp1s7mceTAPjgAJeCAE1ADF6AOGoCCO/AAnsCzdW89Wi/W68y6YM179sAPWG+fwHGcEQ==</latexit>

Equilibrium manifold

Material database

Data points

G(ˆ

<latexit sha1_base64="PATmK+dIqF1VOQDG4unqVLHcYIQ=">AAACBHicbVDLSsNAFJ3UV62vqMtuBotQNyWRgrqRggtdVrAPaEKZTCbt0MkkzEyEGrJw46+4caGIWz/CnX/jpM1CWw9cOJxzL/fe48WMSmVZ30ZpZXVtfaO8Wdna3tndM/cPujJKBCYdHLFI9D0kCaOcdBRVjPRjQVDoMdLzJle537snQtKI36lpTNwQjTgNKEZKS0Oz6oRIjTFi6XVWd8ZIpY4XMT99yLKToVmzGtYMcJnYBamBAu2h+eX4EU5CwhVmSMqBbcXKTZFQFDOSVZxEkhjhCRqRgaYchUS66eyJDB5rxYdBJHRxBWfq74kUhVJOQ0935ifLRS8X//MGiQrO3ZTyOFGE4/miIGFQRTBPBPpUEKzYVBOEBdW3QjxGAmGlc6voEOzFl5dJ97RhNxsXt81a67KIowyq4AjUgQ3OQAvcgDboAAwewTN4BW/Gk/FivBsf89aSUcwcgj8wPn8ATiiYhw==</latexit>

PG!L=GPF

<latexit sha1_base64="rNTV1i5mCpJRGE+PrQyJJzxKncU=">AAACPXicbVA9SwNBFNyL3/ErammzGASrcCeCphAEwVhYREg0kIvh3WaTLNn7YPedEo77Yzb+Bzs7GwtFbG3dS4KocWBhmJnHvjdeJIVG236ycjOzc/MLi0v55ZXVtfXCxuaVDmPFeJ2FMlQNDzSXIuB1FCh5I1IcfE/ya29wmvnXt1xpEQY1HEa85UMvEF3BAI3ULtTcPmDi+oB9BjKppulNUqGuEr0+glLhHb1I6TH9DlRS6jKh2FjxPDPxU8giZ2m7ULRL9gh0mjgTUiQTVNuFR7cTstjnATIJWjcdO8JWAgoFkzzNu7HmEbAB9HjT0AB8rlvJ6PqU7hqlQ7uhMi9AOlJ/TiTgaz30PZPMVtR/vUz8z2vG2D1qJSKIYuQBG3/UjSXFkGZV0o5QnKEcGgJMCbMrZX1QwNAUnjclOH9PniZX+yXnoFS+PCielCd1LJJtskP2iEMOyQk5J1VSJ4zck2fySt6sB+vFerc+xtGcNZnZIr9gfX4BMQivxQ==</latexit>

<latexit sha1_base64="jFn3DZoDVcWNjfO8Y/4cMO+AO+A=">AAAB9XicdVDLSgMxFM3UV62vqks3wSK4GpLS0bqRghuXFewD2rFkMpk2NPMgySh1mP9w40IRt/6LO//G9CGo6IELh3Pu5d57vERwpRH6sApLyyura8X10sbm1vZOeXevreJUUtaisYhl1yOKCR6xluZasG4iGQk9wTre+GLqd26ZVDyOrvUkYW5IhhEPOCXaSDf9EdFZ34uFn93n+aBcQTY6cWq4DpHtIFzHjiFVByNUhdhGM1TAAs1B+b3vxzQNWaSpIEr1MEq0mxGpORUsL/VTxRJCx2TIeoZGJGTKzWZX5/DIKD4MYmkq0nCmfp/ISKjUJPRMZ0j0SP32puJfXi/VQd3NeJSkmkV0vihIBdQxnEYAfS4Z1WJiCKGSm1shHRFJqDZBlUwIX5/C/0m7auOafXZVqzTOF3EUwQE4BMcAg1PQAJegCVqAAgkewBN4tu6sR+vFep23FqzFzD74AevtE5++k0Q=</latexit>

Mapped material database

PL!G

<latexit sha1_base64="xYkoRSxDzc3sB71JnArfFhpj12o=">AAACBnicdVDLSsNAFJ34rPUVdSnCYBFchaSGPnYFF7pwUcE+oIllMp20QycPZiZKCVm58VfcuFDErd/gzr9x0lZQ0QMXDufcy733eDGjQprmh7awuLS8slpYK65vbG5t6zu7bRElHJMWjljEux4ShNGQtCSVjHRjTlDgMdLxxqe537khXNAovJKTmLgBGobUpxhJJfX1AydAcoQRS5vZdXoBHU6HI4k4j27hWdbXS6ZRr1XKdgWahmlWrbKVk3LVPrGhpZQcJTBHs6+/O4MIJwEJJWZIiJ5lxtJNEZcUM5IVnUSQGOExGpKeoiEKiHDT6RsZPFLKAPoRVxVKOFW/T6QoEGISeKozP1r89nLxL6+XSL/mpjSME0lCPFvkJwzKCOaZwAHlBEs2UQRhTtWtEI8QR1iq5IoqhK9P4f+kXTYs26hf2qVGfR5HAeyDQ3AMLFAFDXAOmqAFMLgDD+AJPGv32qP2or3OWhe0+cwe+AHt7RPXbple</latexit>

PG!L

<latexit sha1_base64="YzIchNEr4XCPDMbnpYmF18bAWH8=">AAACBnicbVDLSsNAFJ3UV62vqEsRBovgqiRS0O4KLnThooJ9QBPLZDpph05mwsxEKSErN/6KGxeKuPUb3Pk3TtoutPXAhcM593LvPUHMqNKO820VlpZXVteK66WNza3tHXt3r6VEIjFpYsGE7ARIEUY5aWqqGenEkqAoYKQdjC5yv31PpKKC3+pxTPwIDTgNKUbaSD370IuQHmLE0kZ2l15CT9LBUCMpxQO8znp22ak4E8BF4s5IGczQ6NlfXl/gJCJcY4aU6rpOrP0USU0xI1nJSxSJER6hAekaylFElJ9O3sjgsVH6MBTSFNdwov6eSFGk1DgKTGd+tJr3cvE/r5vo8NxPKY8TTTieLgoTBrWAeSawTyXBmo0NQVhScyvEQyQR1ia5kgnBnX95kbROK261Urupluu1WRxFcACOwAlwwRmogyvQAE2AwSN4Bq/gzXqyXqx362PaWrBmM/vgD6zPH3cZmRw=</latexit>

PL!G

(a) original distance minimization method (b) this paper

<latexit sha1_base64="1HP5gDjhOBC9ux6TjyATSGlu2E0=">AAAB73icdVDLSsNAFJ34rPVVdelmsAiuQlJDHxspuHFZwT6gDWUymbRDJzNxZiLU0J9w40IRt/6OO//GSVtBRQ9cOJxzL/feEySMKu04H9bK6tr6xmZhq7i9s7u3Xzo47CiRSkzaWDAhewFShFFO2ppqRnqJJCgOGOkGk8vc794RqajgN3qaED9GI04jipE2Um8QCBZm97NhqezYjXq14lWhYztOza24OanUvHMPukbJUQZLtIal90EocBoTrjFDSvVdJ9F+hqSmmJFZcZAqkiA8QSPSN5SjmCg/m987g6dGCWEkpCmu4Vz9PpGhWKlpHJjOGOmx+u3l4l9eP9VR3c8oT1JNOF4silIGtYD58zCkkmDNpoYgLKm5FeIxkghrE1HRhPD1KfyfdCq269mNa6/cvFjGUQDH4AScARfUQBNcgRZoAwwYeABP4Nm6tR6tF+t10bpiLWeOwA9Yb5/VV5CD</latexit>

z#=F(z)

<latexit sha1_base64="VJhGUrg1PtxWBhAFkM3eJ2VJRMQ=">AAACL3icdVDLSgMxFM34tr6qLt0Ei6CbMpFi24UiCMVlBWuFTi2ZzG0bmnmQZIQ6zB+58Ve6EVHErX9hpq1QRQ8EDufcR+5xI8GVtu0Xa25+YXFpeWU1t7a+sbmV3965UWEsGTRYKEJ561IFggfQ0FwLuI0kUN8V0HQHF5nfvAepeBhc62EEbZ/2At7ljGojdfI1xw2Flzykd04Bn2JnPDFxRQyp41PdZ1QktfRwqkvw0u+G2dKjTr5gF23bJoTgjJDyiW1ItVo5JhVMMsuggKaod/IjxwtZ7EOgmaBKtYgd6XZCpeZMQJpzYgURZQPag5ahAfVBtZPxyhQfGMXD3VCaF2g8Vmc7EuorNfRdU5ndoH57mfiX14p1t9JOeBDFGgI2WdSNBdYhzsLDHpfAtBgaQpnk5q+Y9amkTJuIcyaE70vx/+TmuEhKxepVqXB+No1jBe2hfXSICCqjc3SJ6qiBGHpEI/SK3qwn69l6tz4mpXPWtGcX/YD1+QXxvKuG</latexit>

<latexit sha1_base64="gXMrMnujbNzXxXNternY0SfnW1I=">AAAB8nicbVDLSgMxFL1TX7W+qi7dBIvgqsxIQbsr6MJlBfuA6VAyaaYNzSRDkhHK0M9w40IRt36NO//GTDsLbT0QOJxzLzn3hAln2rjut1Pa2Nza3invVvb2Dw6PqscnXS1TRWiHSC5VP8SaciZoxzDDaT9RFMchp71wepv7vSeqNJPi0cwSGsR4LFjECDZW8gcxNhOCeXY3H1Zrbt1dAK0TryA1KNAeVr8GI0nSmApDONba99zEBBlWhhFO55VBqmmCyRSPqW+pwDHVQbaIPEcXVhmhSCr7hEEL9fdGhmOtZ3FoJ/OIetXLxf88PzXRTZAxkaSGCrL8KEo5MhLl96MRU5QYPrMEE8VsVkQmWGFibEsVW4K3evI66V7VvUa9+dCotZpFHWU4g3O4BA+uoQX30IYOEJDwDK/w5hjnxXl3PpajJafYOYU/cD5/AHYIkV0=</latexit>

<latexit sha1_base64="MPtgEgJ/WlSqEuBQdd2Fr3FaGEk=">AAAB8nicdVDLSgMxFM34rPVVdekmWARXw8w49LErdOOygn3AdCiZNNOGZpIhyQhl6Ge4caGIW7/GnX9jpq2gogcCh3PuJeeeKGVUacf5sDY2t7Z3dkt75f2Dw6PjyslpT4lMYtLFggk5iJAijHLS1VQzMkglQUnESD+atQu/f0+kooLf6XlKwgRNOI0pRtpIwTBBeooRy9uLUaXq2M1GzfNr0LEdp+56bkG8un/tQ9coBapgjc6o8j4cC5wlhGvMkFKB66Q6zJHUFDOyKA8zRVKEZ2hCAkM5SogK82XkBbw0yhjGQprHNVyq3zdylCg1TyIzWURUv71C/MsLMh03wpzyNNOE49VHccagFrC4H46pJFizuSEIS2qyQjxFEmFtWiqbEr4uhf+Tnme7vt289aut5rqOEjgHF+AKuKAOWuAGdEAXYCDAA3gCz5a2Hq0X63U1umGtd87AD1hvn9SSkZ4=</latexit>

Equilibrium manifold

Material database

<latexit sha1_base64="ymfMxg82jqnhF5x+3m2in7R1gGE=">AAAB+nicdVDLSgMxFM3UV62vqS7dBIvgqkxKsZ1dQRcuK9gHtEPJpJk2NJMZkoxSxvkUNy4UceuXuPNvzLQVVPRA4HDOvdyT48ecKe04H1ZhbX1jc6u4XdrZ3ds/sMuHXRUlktAOiXgk+z5WlDNBO5ppTvuxpDj0Oe35s4vc791SqVgkbvQ8pl6IJ4IFjGBtpJFdHk6xToch1lOCeXqZZSO74lQdx0EIwZygxrljiOs2a6gJUW4ZVMAK7ZH9PhxHJAmp0IRjpQbIibWXYqkZ4TQrDRNFY0xmeEIHhgocUuWli+gZPDXKGAaRNE9ouFC/b6Q4VGoe+mYyz6h+e7n4lzdIdND0UibiRFNBloeChEMdwbwHOGaSEs3nhmAimckKyRRLTLRpq2RK+Pop/J90a1VUr7rX9UrLXdVRBMfgBJwBBBqgBa5AG3QAAXfgATyBZ+veerRerNflaMFa7RyBH7DePgEePZSR</latexit>

z⇤

<latexit sha1_base64="HTtUSy8+qsq9n/pmr8DCX1pRxoE=">AAAB9XicbVBNS8NAEJ3Ur1q/qh69BIsgHkoiBfUiBS8eK9gPaNOy2WzapZvdsLtRasj/8OJBEa/+F2/+G7dtDtr6YODx3gwz8/yYUaUd59sqrKyurW8UN0tb2zu7e+X9g5YSicSkiQUTsuMjRRjlpKmpZqQTS4Iin5G2P76Z+u0HIhUV/F5PYuJFaMhpSDHSRuqnPV+wIH3Ksn56lg3KFafqzGAvEzcnFcjRGJS/eoHASUS4xgwp1XWdWHspkppiRrJSL1EkRniMhqRrKEcRUV46uzqzT4wS2KGQpri2Z+rviRRFSk0i33RGSI/UojcV//O6iQ4vvZTyONGE4/miMGG2FvY0AjugkmDNJoYgLKm51cYjJBHWJqiSCcFdfHmZtM6rbq16dVer1K/zOIpwBMdwCi5cQB1uoQFNwCDhGV7hzXq0Xqx362PeWrDymUP4A+vzByzVkvU=</latexit>

z⇤

Fig. 1: A comparison between (a) the original distance minimization method Kirchdoerfer and Ortiz [2016]

and (b) the introduced method in this paper. See Appendix Bfor terminology.

responding material state (see Def. 6)z∗(Step 2 ). Then, a local-to-global projection PL7→Gfrom material88

states to an updated equilibrium state is performed (Step 3 ). In the proposed embedded method, the local89

step ˆ

PG7→Lin Kirchdoerfer and Ortiz [2016] is modiﬁed. We project the equilibrium state not directly onto90

the material database but onto the closest point on a hyperplane called mapped material database ˆz∈ˆ

D91

(the blue plane in Fig. 1). We then use the established mapping operator obtained from the invertible neu-92

ral network G=F−1to convert the mapped state ˆz∈ˆ

Din the mapped material database back to the93

original material database (the gray manifold in Fig. 1), i.e. z∗∈ D such that PG7→L=G ◦ P◦ F.Pis a94

linear operator that projects the intermediate state z#onto the closest point (with respect to the Euclidean95

norm) on the hyperplane, i.e., ˆz.96

This treatment is introduced to serve two purposes. First, we would like to improve the robustness of97

the data-driven predictions when data are not sufﬁciently dense in the phase space. Second, we want to98

remove the computational bottleneck due to the combinatorial optimization necessary for the closest point99

search in the data-driven paradigm. Our numerical experiments indicate that the global embedding tech-100

nique may improve the robustness of the data-driven approach such that (1) it still functions reasonably101

well when data is sparse or not distributed with even density and (2) remains sufﬁciently efﬁcient when102

data is abundant. Such an improvement is particularly important for high-consequence simulations where103

the robustness of the predictions becomes critical.104

The rest of the paper is organized as follows. We will ﬁrst brieﬂy review the data-driven paradigm and105

outline the current state-of-the-art distance minimization techniques (Section2). The data-driven paradigm106

employs global and local minimization problems sequentially to locate the data points (on the constitutive107

manifold) closest to the conservation manifold. Our work employs a similar design, but the local mini-108

mization problem is reformulated in the mapped Euclidean space. To make the presentation self-contained,109

Section 3is provided to outline the global optimization step for the data-driven paradigm in both the small110

and ﬁnite strain regimes. We then revisit the local minimization problem in Section 4and introduce the in-111

vertible neural network approach in Section 5as the means to improve the efﬁciency of the local search (by112

replacing the combinatorial optimization with a projection performed in the mapped Euclidean space). In113

Section 3.1, we provide the ﬁnite-strain formulation of the data-driven paradigm and show the versatility114

of the proposed framework for large deformation problems. Numerical examples are conducted to illus-115

trate the ideas, verify the implementation and demonstrate and compare the performance of the manifold116

iv Bahador Bahmani, WaiChing Sun

learning approach with the classical Euclidean counterpart in Section 6. A few key observations are sum-117

marized in Section 7. Unless otherwise speciﬁed, we assume that all constitutive manifolds studied in this118

paper are Riemannian.119

In Appendix A, the manifold embedding and the manifold learning techniques commonly used for120

regression, classiﬁcation, and computer vision problems are reviewed. Other related works on predictions121

made on point cloud data in manifold and metric space are outlined in Appendix A.1. Finally, we have also122

included the terminology commonly used in the data-driven paradigm literature for readers who might123

not be familiar with the data-driven approach or concepts essential to the key ideas (e.g., hyperplane) but124

could have been otherwise less well-known among mechanicians are provided in Appendix B.125

2 Literature review on data-driven/model-free solid mechanics126

The model-free distance-minimization method (cf. Kirchdoerfer and Ortiz [2016]) is a new paradigm that127

directly incorporates material databases as the replacements of explicit surrogate constitutive laws into128

the numerical simulations in a variationally consistent manner. However, due to the minimal constraints129

imposed to generate numerical solutions, there must be sufﬁcient data to ensure the distance between130

the data points and the conversation manifold are sufﬁciently close. Hence, the original formulation may131

exhibit sensitivity to noises and outliers [Kirchdoerfer and Ortiz,2017,Iba˜

nez et al.,2017,He and Chen,132

2020,Bahmani and Sun,2021a,Kanno,2021]. In Iba˜

nez et al. [2017], the manifolds of the constitutive133

responses are constructed in an unsupervised learning setting, and a locally linear embedding method134

proposed in Roweis and Saul [2000] is used to recover the low dimensional characteristics of a material135

database locally.136

Meanwhile, the robustness against outliers can be improved by a clustering scheme based on the137

maximum-entropy estimation [Kirchdoerfer and Ortiz,2017]. [Kanno,2018a,b] formulate the distance min-138

imization method based on a local kernel regression method (in an ofﬂine manner) to enhance the robust-139

ness in the noisy-data and limited-data scenarios. [He and Chen,2020] construct a locally linear subspace140

for points in a small neighborhood surrounding each material point (in an online manner). They perform141

convex interpolation on the constructed subspace to increase the robustness against noisy data.142

However, since the locally linear embedding technique requires collecting the neighborhood data143

points to reconstruct the local linear space, the quality of the predictions is sensitive to how the data points144

are distributed. An unevenly distributed data set may lead to the constructed local space with uneven res-145

olution, while data set with limited data point may not be able to capture the topology of the manifold146

precisely.147

Kanno [2021] introduces a new formulation that constructs a global embedding parameterized by a148

kernel method in an ofﬂine manner. One drawback of kernel methods is the scalability issue with respect149

to the data size [Bishop,2006,Lan et al.,2019]. Since kernel methods often operate on dense kernel matrices150

that scale quadratically with the data size, a database with many instances may therefore slow down the151

kernel method and make it less effective for big data applications. Eggersmann et al. [2021b] use a tensor-152

voting interpolation method locally inside a ball around a material point (in an ofﬂine manner) to increase153

the performance in the limited data regime and enhance the robustness against noise. [He et al.,2021]154

introduces a nonlinear dimensionality reduction method based on the autoencoder architecture to perform155

noise-ﬁltering.156

The distance-minimization paradigm has also been extended for different applications such as elasto-157

dynamics [Kirchdoerfer and Ortiz,2018], ﬁnite-strain elasticity [Nguyen and Keip,2018], plasticity [Eg-158

gersmann et al.,2019], fracture mechanics [Carrara et al.,2020], geometrically exact beam theory [Gebhardt159

et al.,2020], poroelasticity [Bahmani and Sun,2021a], and micro-polar continuum [Karapiperis et al.,2021].160

Its scalability issue with respect to the data size is addressed in [Bahmani and Sun,2021a] where an isomet-161

ric projection is introduced to efﬁciently organize the database via the kd-tree data structure that provides162

a logarithmic time complexity (in average) for nearest neighbor search. Other efﬁcient data structures are163

also explored in [Eggersmann et al.,2021a].164

Manifold embedding data-driven mechanics v

3 Data-driven framework165

For completeness, we brieﬂy summarize the distance-minimization data-driven paradigm [Kirchdoerfer166

and Ortiz,2016]. In this approach, the classical solver for a mechanics problem (or generally speaking,167

a boundary value problem) is reformulated as a minimization problem that ﬁnds an admissible solution168

satisfying the conservation laws with the minimum distance to a given database populated by discrete169

point clouds in the phase space. Due to its minimal model assumption about the constitutive behavior, it170

is known as a model-free method and categorized as a non-parametric model. As shown in Fig. 1(a), the171

classical model-free paradigms often consist of two iterative algorithmic steps operated in the conservation172

(governing equation) and constitutive manifold respectively where the equilibrium states are ﬁrst deter-173

mined and projected onto the material data. In the current work, to simplify the formulation and avoid174

the usage of a nonlinear solver, the search of the equilibrium states is based on the energy norm originated175

from Kirchdoerfer and Ortiz [2016] whereas material data identiﬁcation is conducted in the embedded176

Euclidean space of the constitutive manifold.177

To obtain the equilibrium state, we consider pairs of strain-stresses {z∗

a= (e∗

a,σ∗

a)}Nquad

a=1at Nquad

178

quadrature points for a spatial domain discretized by ﬁnite elements. To ﬁnd the equilibrium state that179

has the minimum distance to the given strain-stress 2-tuple, the following constrained optimization prob-180

lem is solved,181

arg min

zZΩd2

M(z,z∗)dΩ,

subject to: ∇x·σ+b=0in Ω,

σ·n=¯

ton Γσ,

u=¯uon Γu,

e=1

2{∇xu+ (∇xu)T}in Ω,

where z= (e,σ),e⊂Rmis the strain tensor, σ∈Rmis the stress tensor, dM(·,·)is the distance function,182

b∈RNdim is the body force vector, u∈RNdim is the displacement vector, nis the normal vector to the183

external boundary, ¯

tis the prescribed traction on the boundary, ¯uis the prescribed displacement on the184

boundary, Ω⊂RNdim is the volumetric domain, Γσ⊂∂Ωis the Dirichlet boundary surface, Γσ⊂∂Ωis the185

Neumann boundary surface, Ndim ={1, 2, 3}is the space dimensionality, and m=Ndim(Ndim +1)/2 is186

the number of free components in the second-order symmetric tensor. The material state at each quadrature187

point z∗

a⊂ D where Dis the product space spanned by Ndata data points D=

Ndata

b=1

(e∗

b,σ∗

b). The search188

of the equilibrium state in the mechanical phase space Z ⊂ Rm×Rmis conducted by minimizing the189

following norm |z|M:190

|z|2

M=1

2e:C:e+1

2σ:S:σ, (1)

where Cand Sare two fourth-order symmetric positive deﬁnite tensors. As suggested in [Kirchdoerfer and191

Ortiz,2016,He and Chen,2020], S=C−1is used herein. According to the introduced norm, the distance192

function reads as follows:193

dM(z,z∗) = |z−z∗|M=|(e−e∗,σ−σ∗)|M(2)

The physical constraints listed in the problem statement are enforced via the Lagrange multipliers.

Then, following the conventional calculus of variations procedure, we obtain the following Euler-Lagrange

equations [Nguyen et al.,2020,He and Chen,2020,Bahmani and Sun,2021a]:

Ru=ZΩδu·∂e(u)

∂u:C:(e−e∗)dΩ=0, (3)

Rβ=−ZΩ∇xδβ:σdΩ+ZΩδβ·bdΓ+ZΓσ

δβ·¯

tdΓ=0, (4)

Rσ=ZΩδσ:[S:(σ−σ∗)−∇xβ]dΩ=0, (5)

vi Bahador Bahmani, WaiChing Sun

where β∈RNdim is the Lagrange multiplier to enforce the balance of linear momentum, and δindicates194

an arbitrary admissible variation, i.e., δu=δβ=0on the Dirichlet boundary surface. Here, we reduce the195

number of independent ﬁelds by a local (point-wise) satisfaction of the last equation via:196

σ=σ∗+S−1:∇xβin Ω. (6)

Given the material states {z∗

a}Nquad

a=1where z∗

a∈ D, the admissible equilibrium states {za}Nquad

a=1at each197

quadrature point can be obtained after solving the above linear, decoupled system of equations. This step198

can be considered, geometrically, as a global projection of material states to the conservation law manifold,199

which is called global minimization or projection; see PL7→Gin Fig. 1(a). However, we do not know the200

optimal distribution of the material states a priori. The distance-minimization method iterates between the201

space spanned by the data points and the conservation laws manifold until the optimal solution is found.202

After updating the equilibrium states at each quadrature point, the new material states should be updated203

accordingly such that they have a minimum total distance to the equilibrium states. This step is performed204

locally at each quadrature point to project the equilibrium state onto the closest data point and is called205

local minimization or projection; see PG7→Lin Fig. 1(a). The entire data-driven algorithm is provided in206

Algorithm 1.207

Algorithm 1 Data-driven small-strain solver

1: Input: Database {(e∗

b,σ∗

b)}Ndata

b=1, and numerical parameters C,S

2: Randomly initialize the material state at quadrature points {(e∗

a,σ∗

a)}Nquad

a=1from the database

3: while not converged do

4: Global projection by solving Eqs. (3) and (4). .PL7→G

5: for a=1 : Nquad do .local operations PG7→L

6: Update equilibrium state (ea,σa)by Eq. (6)

7: Update new material state (e∗

b,σ∗

b)by the local projection of (ea,σa)via Eq. (7)

Here, we generalize the local minimization step of the original distance minimization method as fol-208

lows:209

arg min

z∗∈D

D(za,z∗), (7)

where dDis an appropriate metric to measure the distance in the data space D. In the original distance min-210

imization setup (cf. Kirchdoerfer and Ortiz [2016]), we have dD(·,·)≡dM(·,·), and the local minimization211

step is the nearest neighbor search in the database to ﬁnd the closest data point to a given equilibrium state212

za. Here we hypothesize that an efﬁcient metric to measure the distance in each manifold could be different213

because the conservation and constitutive manifolds may possess different geometrical structures.214

3.1 Extension to the ﬁnite-strain regime215

This section extends the small-strain data-driven formulation to simulate elastic continua undergoing ﬁnite216

deformation. The procedure for the local projection step in the ﬁnite-strain is conceptually similar. The217

only required change is the the changes of the strain and stress measures that constitute the data set and218

the constraints used in the global project step. For completeness, we provide the formulation of the global219

projection for ﬁnite-strain elasticity problems in this section.220

A straightforward extension of the data-driven framework in ﬁnite-strain theory can be established for221

the following metric in the phase space of Green-Lagrange strain Eand second Piola-Kirchhoff Stensors222

as follows Nguyen and Keip [2018], He et al. [2021]:223

|w|2

M=1

2E:CE:E+1

2S:SS:S, (8)

Manifold embedding data-driven mechanics vii

(a) material data points (b) mapped material data points

Fig. 2: (a) synthesized database by σ=√ewith 20 data points that are generated by the regular sampling

along the strain axis. (b) mapped database to a vector space by the invertible neural network. Colors show

the data point number.

where w= (E,S)∈ W ⊂ Rm×Rm,CEand SSare two fourth-order symmetric positive deﬁnite tensors.224

Accordingly, we assume the material database is obtained in E−Spairs as D={(E∗

b,S∗

b)}Ndata

b=1.225

To ﬁnd the equilibrium state that has the minimum distance to a given distribution of material strain-226

stress states {w∗

a= (E∗

a,S∗

a)}Nquad

a=1⊂ D at Nquad quadrature points for a spatial domain discretized by ﬁnite227

elements, we pose the following constrained optimization problem in the reference coordinate system X,228

i.e., total Lagrangian description:229

arg min

zZΩd2

M(w,w∗)dΩ,

subject to: ∇X·P+B=0in Ω,

P·N=¯

Ton ΓP,

u=¯

Uon Γu,

E=1

2{FT·F−I}in Ω,

where u(X)is the displacement vector X,P=FS is ﬁrst Piola-Kirchhoff stress, F=I+∇Xuis the230

deformation gradient, ΓPis the traction boundary, N(X)is the unit outward normal vector to the boundary,231

T(X)is the prescribed traction, ¯

U(X)is the prescribed displacement, and Iis the second-order identity232

tensor. The distance function reads as follows, according to the introduced norm:233

dM(w,w∗) = |w−w∗|M=|(E−E∗,S−S∗)|M. (9)

Similar to the small-strain formulation, the above constrained optimization is ﬁrst unconstrained via

the method of the Lagrange multipliers. Then, using the calculus of variations, we obtain the following

Euler-Lagrange equations:

Ru=ZΩδu·∂E(u)

∂u:CE:(E−E∗)dΩ−ZΩδu·∂F(u)

∂u:(∇Xβ·S)dΩ=0, (10)

Rβ=ZΩ∇Xδβ:PdΩ−ZΩδβ·BdΓ+ZΓP

δβ·¯

TdΓ=0, (11)

RS=ZΩδS:[SS:(S−S∗)−FT·∇Xβ]dΩ=0, (12)

viii Bahador Bahmani, WaiChing Sun

where β∈RNdim is the Lagrange multiplier that enforces the balance of linear momentum, and δindicates234

an arbitrary admissible variation, i.e., δu=δβ=0on the Dirichlet boundary surface. Here, we reduce the235

number of independent ﬁelds by a local satisfaction of the last equation via:236

S=S∗+ (SS)−1:(FT· ∇Xβ)in Ω. (13)

Due to the geometrical non-linearity, the system of equations for the global projection is nonlinear and237

strongly coupled. They are solved in a monolithic solver where a standard Newton-Raphson method is238

used. We summarize the ﬁnite-strain model-free framework in Algorithm 2.239

Algorithm 2 Data-driven ﬁnite-strain solver

1: Input: Database {(E∗

b,S∗

b)}Ndata

b=1, and numerical parameters CE,SS

2: initialize all the material states to zero

3: while not converged data-driven do

4: while not converged Newton-Raphson do .Global projection PL7→G

5: solve linearized Eqs. (10) and (11).

6: for a=1 : Nquad do .Local operations PG7→L

7: Update equilibrium state (Ea,Sa)by Eq. (13)

8: Update new material state (E∗

b,S∗

b)by the local projection of (Ea,Sa)via Eq. (7)

3.2 Illustrative example 1: dependence of metrics for local projections240

To show the importance of choosing a suitable metric in the local minimization step, let’s consider a simple241

one-dimensional problem where the material database is given in Fig. 2(a). Figure 3(a-c) studies the effect of242

parameter C(see Eq. (1)) on the found closest data point z∗shown by red star marker to a query equilibrium243

state z= (0.61, 0.67)shown by a black square marker. Notice that, in the one-dimensional setup, the244

forth-order tensor Cbecomes a scalar C. As we observe, based on the value of C, there exist two different245

responses.246

To further investigate this issue, we consider 9 random query points shown by black square markers in247

Fig. 4(a-c). In this experiment, we observe that the ﬁnal responses can vary among only ﬁve different choices248

based on the different assigned values of the parameter C. This observation conﬁrms that the choice of the249

norm may have a signiﬁcant impact on the ﬁnal results, which is also reported in Leygue et al. [2018].250

Notice that increasing the amount of data might reduce the chance of encountering this spurious issue.251

Here, we aim to enhance the consistency of this local projection step by introducing an algorithm that252

implicitly explores the underlying geometry of the data space Dto ﬁnd the closest interpolated data point253

based on a data-driven metric formulation.254

Remark 1 As pointed out by Kirchdoerfer and Ortiz [2016], the data-driven framework would require255

solving the Euler-Lagrange equations that updates the nodal displacement and the Lagrange multiplier256

associated with the out-of-balance forces. This increased number of unknowns leads to a considerably257

larger system of equations than that of a classical displacement-based ﬁnite element solver. Moreover, the258

data-driven paradigm solves a double-minimization problem through an iterative scheme. As such, the259

distance-minimization paradigms, including the one presented in this paper, are not expected to simulate260

nonlinear elasticity problems faster than the conventional displacement-based ﬁnite element solver that261

employs an explicitly written constitutive law to update stress directly at the Gauss point.262

4 Revisited local minimization in constitutive manifold263

As demonstrated in, for instance, Iba˜

nez et al. [2017] and Ibanez et al. [2018], nonlinear constitutive data264

often appear to be belonging to a real Riemannian manifold. While the distance between two very close265

Manifold embedding data-driven mechanics ix

(a) Nearest neighbor projection (b) Nearest neighbor projection

Fig. 3: Comparing local projections for one arbitrary query point z= (0.61, 0.67)shown by a black square

marker between (a-c) the original distance minimization method and (d) manifold embedding method

introduced in this paper. A query point in the local minimization step is a point that belongs to the conser-

vation manifold, found in the global minimization step, but not necessarily has the minimum distance to

the material database. In (a-c), the Cparameter in Eq. (1) is vary (0.1, 0.5, 1.0) to examine how the chosen

norm affect the selection of material data points. Points with colorful circular markers are material data

points. The star marker shows the projected material state for the query point. The solid black curve in (d)

indicates the underlying constitutive manifold used to synthesize the database.

points in the same tangential space of the manifold can be characterized by the Euclidean distance, the266

geodesics distance of data points in a constitutive manifold is generally different from the Euclidean coun-267

terpart. Our major point of departure here is the introduction of an indirect measurement of distance for268

the local minimization problem.269

In the original work by Kirchdoerfer and Ortiz [2016], the constitutive data are thought to be in a phase270

space equipped with an energy norm that is used to measure distance or discrepancy between the consti-271

tutive and equilibrium manifolds. The drawback is that the choice of the optimal solution may be affected272

by the bias introduced by the different choices of the energy norms and the Euclidean distance in the phase273

space which has no connection to the geodesic of the constitutive manifold. This is not a signiﬁcant issue274

for accuracy when the constitutive law is linear or when there is sufﬁciently high data density to guarantee275

that the distances measured by the norm equipped to a tangential space of the manifold and the geodesic276

are sufﬁciently close.277

In the latter case, He and Chen [2020] propose a local embedding method where a locally linear patch278

formed among the nearest neighbors of a data point is constructed each time, and a distance is measured.279

Kanno [2021], on the other hand, directly uses a kernel method to establish the constitutive manifold based280

on the available data. Furthermore, one may also identify a ﬁnite number of afﬁne subspaces between the281

x Bahador Bahmani, WaiChing Sun

(a) Nearest neighbor projection (b) Nearest neighbor projection

Fig. 4: Comparing local projections between (a-c) the original distance minimization method and (d) man-

ifold embedding method introduced in this paper. Query points are shown by black square markers. In

(a-c), different Cparameters in Eq. (1) are used to exhibit the dependence of projected results with respect

to the equipped norm. The solid lines show the projection results. The dashed line indicates the underlying

constitutive manifold used to synthesize the database (colorful dot points). The results in (a-c) suggest that

the norm may considerably affect the ﬁnal results, especially for those points too far away from the train

data points.

data points and use them to measure the distance between points from different atlases of the manifold.282

In particular, the local reconstruction of the Euclidean space in He and Chen [2020] is a special case where283

a local space is constructed in an online manner. As an analogy, the readers may consider measuring the284

distance between two points by drawing a straight line through different patched surface of a soccer ball285

where the surface is locally ﬂat and has a single normal vector within each patch but globally is of the286

shape of a sphere.287

In this work, we reﬁne the view that the data resides in the phase space as in Kirchdoerfer and Ortiz288

[2016] and restrict the data points to be elements of a sufﬁciently smooth manifold as in the case of Kanno289

[2021]. However, we also want to avoid handling multiple afﬁne subspaces to measure the distance or290

generate local Euclidean subspace during the local search. As such, we introduce a new idea where we291

simply deform the manifold where the data reside and make it a single hyperplane (see Deﬁnition 2). This292

treatment then enables us to introduce a simpler perception where a single norm is sufﬁcient to measure293

the distance without introducing the coordinate chart. This is done by measuring the distance of the data294

Manifold embedding data-driven mechanics xi

points of an imaginary hyperplane where all the data points lie on the nonlinear constitutive manifold are295

mapped onto.296

We consider a constitutive response as a nonlinear mapping that maps a m-dimensional input onto a297

m-dimensional output. The data of the material databases are instances of this constitutive response where298

each instance is stored as a 2m-dimensional vector. For instance, the symmetric strain and stress tensors299

are stored as a 12-dimensional vector since both the strain and stress tensor in three-dimensional space300

consists of six independent components (degrees of freedom) due to their symmetries).301

As such, we globally map the constitutive manifold data onto a hyperplane by a nonlinear mapping302

function F:(D ⊂ R2m)7→ R2m. Hence, if such a mapping function does exist and can be determined, the303

Euclidean norm is indeed a valid norm for the constructed space that embedded the constitutive manifold.304

An important property we want to achieve is to ensure that the image of Fis a hyperplane. In particu-305

lar, want to ensure that, for any data point ˆz, the coordinates ˆziand ˆzjcan be linearly related by a constant306

matrix ˆ

K, i.e.,307

ˆzj=ˆ

Kji ˆzifor 1 ≤i≤m,m<j≤2m, (14)

where ˆz=F(z). To justify this claim, the hyperplane parameterized by a constant non-zero normal vector308

ˆc∈R2mcan be found straightforwardly such that ˆz·ˆc=0 and hence the image of the function Fis a309

vector space embedded in R2m. In the rest of this subsection, we would assume such a mapping exists, and310

later we introduce an algorithm to ﬁnd such a mapping through the expressivity power of neural networks311

[Hornik et al.,1989,Balestriero et al.,2021].312

The image of Fis a vector space constructed via the points sampled from the data manifold z∗⊂ D,313

therefore a constitutive response outside the material data manifoldz/∈ D will not be mapped onto the con-314

structed hyperplane, as depicted in Fig. 1(b) where the mapped point z#in Step 2 does not belong to the315

hyperplane colored by blue. Since the normal vector of the constructed hyperplane is identical everywhere316

on the hyperplane, we can replace the discrete minimization statement in Eq. (7) with a continuous coun-317

terpart where a projection of the point on the hyperplane is sufﬁcient to determine the shortest distance318

deﬁned in the mapped space. This continuous relaxation changes the initial NP-hard problem [Bahmani319

and Sun,2021a] to a tractable one. We provide the details of this new local projection scheme as follows.320

For the ease of interpretation and explanation, we call the ﬁrst mcomponents of the mapped variable ˆz

pseudo-strain ˆeand the rest pseudo-stress ˆ

σ, i.e., ˆzT= [ ˆeT,ˆ

σT]. To ﬁnd a closed-form solution for the local

minimization projection, we restrict the ˆ

Kto be symmetric positive-deﬁnite in Eq. (14), i.e., ˆ

K∈SPD(m).

We introduce the local minimization in the mapped space as follow:

arg min

ˆz∈R2m

d2(z#, ˆz),

subject to: ˆ

σ=ˆ

Kˆe,

where ˆ

d(·,·)is the energy distance in the mapped domain as follows:321

d2(z#, ˆz) = 1

2(e#−ˆe)Tˆ

K(e#−ˆe) + 1

2(σ#−ˆ

σ)Tˆ

K−1(σ#−ˆ

σ). (15)

We obtain the following unique solution for the above quadratic objective by setting the objective’s deriva-

tive to zero and using basic linear algebra manipulations:

ˆe=1

2(e#+ˆ

K−1σ#), (16)

σ=1

2(σ#+ˆ

Ke#), (17)

ˆzT=PT(z#) = [ ˆeT,ˆ

σT], (18)

where Pis the projection operator to ﬁnd the closest point on the hyperplane, see step 2 →3 in Fig. 1(b).322

Next, we should ﬁnd the corresponding point in the actual data space since the governing equations323

are established with respect to the actual data space. Therefore, in our introduced framework, a proper324

map function Fshould be globally bijective (invertible). In the next section, we introduce a method to ﬁnd325

an appropriate bijective map function parameterized by neural networks.326

xii Bahador Bahmani, WaiChing Sun

Remark 2 The energy metric in Eq. (15) is equivalent to the Euclidean metric and they can be mapped327

bijectively upon a linear transformation [Bahmani and Sun,2021a]. This metric can be reduced into the328

Euclidean metric by setting ˆ

K=Imwhere Imis the m×midentity matrix.329

Remark 3 In Eq. (14), ˆ

Kcontrols the constructed hyperplane orientation (its normal vector). Provided that330

the embedding is successful such that the mapping and inverse mapping between the physical space and331

hyperplane both exist, the choice of ˆ

Kshould not have any effect on the predictions.332

5 Neural network embedding of constitutive manifold333

G=F1

<latexit sha1_base64="7clbuyZDFadA4+DdU5zjTv4Vdf4=">AAACAnicdVDLSsNAFJ3UV62vqCtxM1gEVyGJoY+NFFzosoJthSaUyXTSDp08mJkIJQQ3/oobF4q49Svc+TdO2goqemDgcO7j3Dl+wqiQpvmhlZaWV1bXyuuVjc2t7R19d68r4pRj0sExi/mNjwRhNCIdSSUjNwknKPQZ6fmT86LeuyVc0Di6ltOEeCEaRTSgGEklDfQDd7Yj8xnCkxy6IZJjjFh2kQ/0qmk0GzXbqUHTMM26ZVsFsevOqQMtpRSoggXaA/3dHcY4DUkkMUNC9C0zkV6GuKSYkbzipoIkygWNSF/RCIVEeNnMPofHShnCIObqRRLO1O8TGQqFmIa+6ixOFL9rhfhXrZ/KoOFlNEpSSSI8NwpSBmUMizzgkHKCJZsqgjCn6laIx4gjLFVqFRXC10/h/6RrG5ZjNK+cautsEUcZHIIjcAIsUActcAnaoAMwuAMP4Ak8a/fao/aivc5bS9piZh/8gPb2CS0Fl/Q=</latexit>

z⇤

<latexit sha1_base64="irGWihaq5mQPr2rXUBltfsrIRNY=">AAAB8XicbVBNSwMxEJ31s9avqkcvwSKIh7IrBfUiBS8eK9gPbNeSzWbb0GyyJFmhLv0XXjwo4tV/481/Y9ruQVsfDDzem2FmXpBwpo3rfjtLyyura+uFjeLm1vbObmlvv6llqghtEMmlagdYU84EbRhmOG0niuI44LQVDK8nfuuRKs2kuDOjhPox7gsWMYKNle67geRh9jR+OO2Vym7FnQItEi8nZchR75W+uqEkaUyFIRxr3fHcxPgZVoYRTsfFbqppgskQ92nHUoFjqv1sevEYHVslRJFUtoRBU/X3RIZjrUdxYDtjbAZ63puI/3md1EQXfsZEkhoqyGxRlHJkJJq8j0KmKDF8ZAkmitlbERlghYmxIRVtCN78y4ukeVbxqpXL22q5dpXHUYBDOIIT8OAcanADdWgAAQHP8ApvjnZenHfnY9a65OQzB/AHzucPlJmQ3Q==</latexit>

<latexit sha1_base64="UOtLrVuenv9kLi/801X08P3L/5A=">AAAB9XicdVDLSsNAFJ3UV62vqks3g0VwVTKl2HYjBTcuK9gHNLFMJpN26GQSZiZKDfkPNy4Uceu/uPNvnLQVVPTAhcM593LvPV7MmdK2/WEVVlbX1jeKm6Wt7Z3dvfL+QU9FiSS0SyIeyYGHFeVM0K5mmtNBLCkOPU773vQi9/u3VCoWiWs9i6kb4rFgASNYG+nGmWCdOl7E/fQ+y0blil21bRshBHOCGme2Ia1Ws4aaEOWWQQUs0RmV3x0/IklIhSYcKzVEdqzdFEvNCKdZyUkUjTGZ4jEdGipwSJWbzq/O4IlRfBhE0pTQcK5+n0hxqNQs9ExniPVE/fZy8S9vmOig6aZMxImmgiwWBQmHOoJ5BNBnkhLNZ4ZgIpm5FZIJlphoE1TJhPD1Kfyf9GpVVK+2ruqV9vkyjiI4AsfgFCDQAG1wCTqgCwiQ4AE8gWfrznq0XqzXRWvBWs4cgh+w3j4BnhiTRA==</latexit>

Material database

Mapped material database

z⇤

(a)

(b)

Fig. 5: (a) global mapping of the material data space into a vector space by a bijective function and vice-

versa. (b) a vanilla autoencoder architecture to parametrize a bijective function by encoder Fand decoder

Gdeep neural networks.

We are interested in ﬁnding an appropriate function Fwith the desired properties mentioned in the334

previous section to map the data points of the actual database onto a hyperplane (a Euclidean space), see335

Fig. 5(a). We hypothesize that such a mapping function can be found in the class of multilayer perceptrons336

(MLPs) due to their expressiveness power Hornik et al. [1989]. In a general setup, MLPs are not guaranteed337

to be invertible. However, we may use the similar idea introduced in the autoencoder architecture (cf.338

Hinton and Salakhutdinov [2006], Bengio et al. [2007], Vincent et al. [2008], Baldi [2012]) to deﬁne different339

MLPs for the forward Fand backward Gfunctions, see Fig. 5(b). The connection between forward and340

backward maps, i.e., G=F−1, is incorporated as a soft constraint in the optimization statement. Therefore,341

our proposed local projection can be expressed as the following optimization problem:342

arg min

θ∈RNF,β∈RNG,ˆ

K∈SPD(m)

Ndata

∑

i=1||ˆ

σi−ˆ

Kˆei||2

2+||z∗

i−G(F(z∗

i,θ),β)||2

2, (19)

where functions Fand Gare MLPs parameterized by unknown vectors θand βwith sizes NFand NG,343

respectively. In this notation, these vectors concatenate weights and biases. The ﬁrst term in the above344

objective is designated to enforce the linearity of the target vector space. The second term enforces the345

bijectivity constraint also known as the reconstruction error for the autoencoder architecture where F346

Manifold embedding data-driven mechanics xiii

and Gare encoder and decoder functions. The above architecture is a naive formulation of the problem347

statement explained in the previous section, and we call it the vanilla autoencoder formulation.348

By leveraging the expressive power of the deep neural networks [Hornik et al.,1989,Raghu et al.,349

2017], we argue that, as long as (1) the training is successful and (2) the unsampled data not used in the350

training process are indeed on the same manifold of the training data, the numerical value of ˆ

Kopt in351

19 is not consequential. Since it is equivalent to any ﬁxed ˆ

Kﬁx ∈SPD(m)up to a linear transformation352

Kopt =Tˆ

Kﬁx;T∈SPD(m), every two members of SPD group commutes by the matrix multiplication.353

This linear transformation can be indirectly found as the outcome of a successfully trained neural network354

in which we are interested in the properties of the embedding space but not its explicit value. Therefore, in355

this work, we ﬁx the matrix ˆ

Kﬁx ∈SPD(m)to simplify the optimization problem.356

The optimization over MLPs is known to be non-convex [Goodfellow et al.,2016]; hence, ﬁnding the357

global minimizer of the above objective is not trivial in a general setting. Even for the near-optimal local358

minimizers, the bijectivity constraint cannot be achieved precisely. Moreover, this optimization problem359

is multi-objective, which makes it necessary to consider the possibility of conﬂict situations where Pareto360

efﬁciency could be a primary concern [Yu et al.,2020,Bahmani and Sun,2021b]. To address these draw-361

backs, we incorporate a new architecture in the next subsection that leads to neural network predictions362

preserving the bijectivity condition by construction and hence bypassing the need to handle conﬂicting363

objectives.364

5.1 Invertible neural network365

<latexit sha1_base64="Ik+r57fH5xGyx3t8VcsiA2H93aY=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoMgCGFXAupFAl48JmAekCxhdtKbjJmdXWZmhRDyBV48KOLVT/Lm3zhJ9qCJBQ1FVTfdXUEiuDau++3k1tY3Nrfy24Wd3b39g+LhUVPHqWLYYLGIVTugGgWX2DDcCGwnCmkUCGwFo7uZ33pCpXksH8w4QT+iA8lDzqixUv2iVyy5ZXcOskq8jJQgQ61X/Or2Y5ZGKA0TVOuO5ybGn1BlOBM4LXRTjQllIzrAjqWSRqj9yfzQKTmzSp+EsbIlDZmrvycmNNJ6HAW2M6JmqJe9mfif10lNeO1PuExSg5ItFoWpICYms69JnytkRowtoUxxeythQ6ooMzabgg3BW355lTQvy16lfFOvlKq3WRx5OIFTOAcPrqAK91CDBjBAeIZXeHMenRfn3flYtOacbOYY/sD5/AF0MYy3</latexit>

⇥

<latexit sha1_base64="fZnesU6PvlvvfTtvH7Lk4k5WxrY=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoF6k4MVjBfsBbSib7aZdu9mE3YlQQv+DFw+KePX/ePPfuG1z0NYHA4/3ZpiZFyRSGHTdb6ewtr6xuVXcLu3s7u0flA+PWiZONeNNFstYdwJquBSKN1Gg5J1EcxoFkreD8e3Mbz9xbUSsHnCScD+iQyVCwShaqdVDEXHTL1fcqjsHWSVeTiqQo9Evf/UGMUsjrpBJakzXcxP0M6pRMMmnpV5qeELZmA5511JF7RI/m187JWdWGZAw1rYUkrn6eyKjkTGTKLCdEcWRWfZm4n9eN8Xwys+ESlLkii0WhakkGJPZ62QgNGcoJ5ZQpoW9lbAR1ZShDahkQ/CWX14lrYuqV6te39cq9Zs8jiKcwCmcgweXUIc7aEATGDzCM7zCmxM7L86787FoLTj5zDH8gfP5A7jXjzw=</latexit>

⇥

zi+1

<latexit sha1_base64="4PefMRIOUAhCEyaKLg2nRs989EI=">AAAB9XicdVDLSgMxFM3UV62vqks3wSIIQklKse1GCm5cVrAPaKclk8m0oZkHSUapw/yHGxeKuPVf3Pk3ZtoKKnrgwuGce7n3HicSXGmEPqzcyura+kZ+s7C1vbO7V9w/6KgwlpS1aShC2XOIYoIHrK25FqwXSUZ8R7CuM73M/O4tk4qHwY2eRcz2yTjgHqdEG2k4cELhJvfpMOFnOB0VS6iMEMIYw4zg2jkypNGoV3Ad4swyKIElWqPi+8ANaeyzQFNBlOpjFGk7IVJzKlhaGMSKRYROyZj1DQ2Iz5SdzK9O4YlRXOiF0lSg4Vz9PpEQX6mZ75hOn+iJ+u1l4l9eP9Ze3U54EMWaBXSxyIsF1CHMIoAul4xqMTOEUMnNrZBOiCRUm6AKJoSvT+H/pFMp42q5cV0tNS+WceTBETgGpwCDGmiCK9ACbUCBBA/gCTxbd9aj9WK9Llpz1nLmEPyA9fYJ606Szg==</latexit>

zi+1

<latexit sha1_base64="kPpfR8Viuofmk8cwhDZAceE/++M=">AAAB+XicdVDLSsNAFJ3UV62vqEs3g0UQhJIpxTa7ghuXFewD2hgmk0k7dPJgZlKoIX/ixoUibv0Td/6Nk7aCih64cDjnXu69x0s4k8qyPozS2vrG5lZ5u7Kzu7d/YB4e9WScCkK7JOaxGHhYUs4i2lVMcTpIBMWhx2nfm14Vfn9GhWRxdKvmCXVCPI5YwAhWWnJNc+TF3M/u87uMXaDcRa5ZtWqWZSGEYEFQ89LSxLZbddSCqLA0qmCFjmu+j/yYpCGNFOFYyiGyEuVkWChGOM0ro1TSBJMpHtOhphEOqXSyxeU5PNOKD4NY6IoUXKjfJzIcSjkPPd0ZYjWRv71C/MsbpipoORmLklTRiCwXBSmHKoZFDNBnghLF55pgIpi+FZIJFpgoHVZFh/D1Kfyf9Oo11KjZN41q217FUQYn4BScAwSaoA2uQQd0AQEz8ACewLORGY/Gi/G6bC0Zq5lj8APG2yeRcJOe</latexit>

zi+1

<latexit sha1_base64="LADe5zm3Bnnv3TAY8OOVIjp8REo=">AAAB+XicdVDLSsNAFJ3UV62vqEs3g0UQhJIpxTa7ghuXFewD2hgmk0k7dPJgZlKoIX/ixoUibv0Td/6Nk7aCih64cDjnXu69x0s4k8qyPozS2vrG5lZ5u7Kzu7d/YB4e9WScCkK7JOaxGHhYUs4i2lVMcTpIBMWhx2nfm14Vfn9GhWRxdKvmCXVCPI5YwAhWWnJNc+TF3M/u87uMXaDcrbtm1apZloUQggVBzUtLE9tu1VELosLSqIIVOq75PvJjkoY0UoRjKYfISpSTYaEY4TSvjFJJE0ymeEyHmkY4pNLJFpfn8EwrPgxioStScKF+n8hwKOU89HRniNVE/vYK8S9vmKqg5WQsSlJFI7JcFKQcqhgWMUCfCUoUn2uCiWD6VkgmWGCidFgVHcLXp/B/0qvXUKNm3zSqbXsVRxmcgFNwDhBogja4Bh3QBQTMwAN4As9GZjwaL8brsrVkrGaOwQ8Yb5+S9JOf</latexit>

<latexit sha1_base64="4uheGEZIYO4LizZlIBzCRJ1FsvQ=">AAAB9XicbVBNS8NAEJ3Ur1q/qh69BIvgqSSloL0VvHisYD+gTctms2mXbnbD7kapIf/DiwdFvPpfvPlv3LY5aOuDgcd7M8zM82NGlXacb6uwsbm1vVPcLe3tHxwelY9POkokEpM2FkzIno8UYZSTtqaakV4sCYp8Rrr+9Gbudx+IVFTwez2LiRehMachxUgbaTjwBQvSp2yY0mxUG5UrTtVZwF4nbk4qkKM1Kn8NAoGTiHCNGVKq7zqx9lIkNcWMZKVBokiM8BSNSd9QjiKivHRxdWZfGCWwQyFNcW0v1N8TKYqUmkW+6YyQnqhVby7+5/UTHV57KeVxognHy0Vhwmwt7HkEdkAlwZrNDEFYUnOrjSdIIqxNUCUTgrv68jrp1Kpuvdq4q1eajTyOIpzBOVyCC1fQhFtoQRswSHiGV3izHq0X6936WLYWrHzmFP7A+vwB7f2SyA==</latexit>

<latexit sha1_base64="LVViIsFTb4JS9PHnVH7VFYYL7js=">AAAB9XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoLkFvHiMYB6QF7Ozs8mQ2ZllZlaJy/6HFw+KePVfvPk3TpI9aGJBQ1HVTXeXH3Omjet+O4W19Y3NreJ2aWd3b/+gfHjU0jJRhDaJ5FJ1fKwpZ4I2DTOcdmJFceRz2vYnNzO//UCVZlLcm2lM+xEeCRYygo2VBj1f8iB9ygYpy4besFxxq+4caJV4OalAjsaw/NULJEkiKgzhWOuu58amn2JlGOE0K/USTWNMJnhEu5YKHFHdT+dXZ+jMKgEKpbIlDJqrvydSHGk9jXzbGWEz1sveTPzP6yYmvO6nTMSJoYIsFoUJR0aiWQQoYIoSw6eWYKKYvRWRMVaYGBtUyYbgLb+8SloXVe+yWru7rNRreRxFOIFTOAcPrqAOt9CAJhBQ8Ayv8OY8Oi/Ou/OxaC04+cwx/IHz+QPseZLH</latexit>

<latexit sha1_base64="TY9h3e5yaRkB0XhFEBGmGoe647c=">AAAB83icbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoL0VvHisYFuhiWWz2bRLN7thdyPU0L/hxYMiXv0z3vw3btsctPXBwOO9GWbmhSln2rjut1NaW9/Y3CpvV3Z29/YPqodHXS0zRWiHSC7VfYg15UzQjmGG0/tUUZyEnPbC8fXM7z1SpZkUd2aS0iDBQ8FiRrCxku+Hkkf50/QhZ9NBtebW3TnQKvEKUoMC7UH1y48kyRIqDOFY677npibIsTKMcDqt+JmmKSZjPKR9SwVOqA7y+c1TdGaVCMVS2RIGzdXfEzlOtJ4koe1MsBnpZW8m/uf1MxNfBTkTaWaoIItFccaRkWgWAIqYosTwiSWYKGZvRWSEFSbGxlSxIXjLL6+S7kXda9Sbt41aq1nEUYYTOIVz8OASWnADbegAgRSe4RXenMx5cd6dj0VrySlmjuEPnM8fvXmSIw==</latexit>

<latexit sha1_base64="s+JqhudL5lCoqFPXDnsnFPafG10=">AAAB9HicbVDLSgMxFL1TX7W+qi7dBIvgqsxIQd1VBHFZwT6gHUsmzbShmWRMMoUyFPwLNy4UcevHuPNvzLRdaOuBC4dz7iUnJ4g508Z1v53cyura+kZ+s7C1vbO7V9w/aGiZKELrRHKpWgHWlDNB64YZTluxojgKOG0Gw+vMb46o0kyKezOOqR/hvmAhI9hYye9E2AwI5unN5IF1iyW37E6Blok3JyWYo9YtfnV6kiQRFYZwrHXbc2Pjp1gZRjidFDqJpjEmQ9ynbUsFjqj202noCTqxSg+FUtkRBk3V3xcpjrQeR4HdzELqRS8T//PaiQkv/JSJODFUkNlDYcKRkShrAPWYosTwsSWYKGazIjLAChNjeyrYErzFLy+TxlnZq5Qv7yql6tXTrI48HMExnIIH51CFW6hBHQg8wjO8wpszcl6cd+djtppz5hUewh84nz8ibJLC</latexit>

<latexit sha1_base64="W9nbDDE3X5sRm3Pc7fubVDRRZsY=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoL0VvHisYNpCG8tmu22XbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhYkUBl332ylsbG5t7xR3S3v7B4dH5eOTlolTzbjPYhnrTkgNl0JxHwVK3kk0p1EoeTuc3M799hPXRsTqAacJDyI6UmIoGEUr+eNH0ff65YpbdRcg68TLSQVyNPvlr94gZmnEFTJJjel6boJBRjUKJvms1EsNTyib0BHvWqpoxE2QLY6dkQurDMgw1rYUkoX6eyKjkTHTKLSdEcWxWfXm4n9eN8XhTZAJlaTIFVsuGqaSYEzmn5OB0JyhnFpCmRb2VsLGVFOGNp+SDcFbfXmdtK6qXq1av69VGvU8jiKcwTlcggfX0IA7aIIPDAQ8wyu8Ocp5cd6dj2VrwclnTuEPnM8fbrSObg==</latexit>

<latexit sha1_base64="DQNJ6Su0sqrMrMm5VWm3kEXSFbY=">AAAB7HicbVBNS8NAEJ34WetX1aOXxSJ4KkkpaG8FLx4rmLbQxrLZbtulm03YnQgl9Dd48aCIV3+QN/+N2zYHbX0w8Hhvhpl5YSKFQdf9djY2t7Z3dgt7xf2Dw6Pj0slpy8SpZtxnsYx1J6SGS6G4jwIl7ySa0yiUvB1Obud++4lrI2L1gNOEBxEdKTEUjKKV/PGj6Ff7pbJbcRcg68TLSRlyNPulr94gZmnEFTJJjel6boJBRjUKJvms2EsNTyib0BHvWqpoxE2QLY6dkUurDMgw1rYUkoX6eyKjkTHTKLSdEcWxWfXm4n9eN8XhTZAJlaTIFVsuGqaSYEzmn5OB0JyhnFpCmRb2VsLGVFOGNp+iDcFbfXmdtKoVr1ap39fKjXoeRwHO4QKuwINraMAdNMEHBgKe4RXeHOW8OO/Ox7J1w8lnzuAPnM8fcDiObw==</latexit>

<latexit sha1_base64="wNezb+TO1nKTISggdd90lPFIFvI=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoL0VvHisYNpCG8tmu2mXbjZhdyKU0t/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332ylsbG5t7xR3S3v7B4dH5eOTlkkyzbjPEpnoTkgNl0JxHwVK3kk1p3EoeTsc38799hPXRiTqAScpD2I6VCISjKKV/OhR9L1+ueJW3QXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoJpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWldVr1at39cqjXoeRxHO4BwuwYNraMAdNMEHBgKe4RXeHOW8OO/Ox7K14OQzp/AHzucPa6SObA==</latexit>

<latexit sha1_base64="F80yjUuDTgq0+NhybKnoedQZ38E=">AAAB7HicbVBNS8NAEJ34WetX1aOXxSJ4KkkpaG8FLx4rmLbQxrLZbtqlm03YnQil9Dd48aCIV3+QN/+N2zYHbX0w8Hhvhpl5YSqFQdf9djY2t7Z3dgt7xf2Dw6Pj0slpyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++4lrIxL1gJOUBzEdKhEJRtFKfvQo+tV+qexW3AXIOvFyUoYczX7pqzdIWBZzhUxSY7qem2IwpRoFk3xW7GWGp5SN6ZB3LVU05iaYLo6dkUurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoJpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/RhuCtvrxOWtWKV6vU72vlRj2PowDncAFX4ME1NOAOmuADAwHP8ApvjnJenHfnY9m64eQzZ/AHzucPbSiObQ==</latexit>

Fig. 6: Computational graph of the ith coupling layer in the invertible architecture. hi

1,hi

2,fi

1, and fi

2are any

arbitrary functions such as MLP. ziand zi+1are the input and output feature vectors of the coupling layer.

Due to the drawbacks of the vanilla autoencoder framework mentioned in the previous section, we

formulate a new machine learning strategy built upon a type of neural network that is bijective by its

construction (as an inductive bias). The invertible neural network is built upon several coupling layers. A

coupling layer maps the ith layer’s feature vector zi∈Rnito the (i+1)th layer ’s feature vector zi+1∈Rni+1

via the function Fi:zi7→ zi+1customized as follows [Dinh et al.,2014,2016,Ardizzone et al.,2018,Beitler

et al.,2021]:

zi+1

1=zi

1hi

1(zi

2) + fi

1(zi

2), (20)

zi+1

2=zi

2hi

2(zi+1

1) + fi

2(zi+1

1), (21)

where is the Hadamard product (element-wise multiplication), and hi

1,hi

2,fi

1, and fi

2are four arbitrary366

functions called internal functions. In each coupling layer, before the forward pass, the input feature ziis367

divided into two arbitrary, disjoint halves zi

1∈Rmi

1and zi

2∈Rmi

2where mi

1+mi

2=ni. Notice that the368

xiv Bahador Bahmani, WaiChing Sun

internal functions can be deep neural networks to increase the complexity and expressivity of each layer.369

In this work, we parametrize these functions via MLPs.370

Because of the simple yet elegant structure in each coupling layer, the backward map (Fi)−1:zi+17→ zi

is readily available as follows:

2= (zi+1

2−fi

2(zi+1

1)) 1

2(zi+1

1)(22)

1= (zi+1

1−fi

1(zi

2)) 1

1(zi

2). (23)

A deep bijective, invertible neural network F:(zin ∈Rnin )7→ (zout ∈Rnout )can be constructed by the371

composition of Lcoupling layers:372

F(z;θ) = FL◦FL−1◦· ··◦ F1(z). (24)

where θconcatenates all the parameters deﬁned for deep neural networks {hi

1,hi

2,fi

1,fi

2}L

i=1.373

We tailor the introduced architecture for our purpose as follows. The input feature vector z1←z∗∈374

R2mcan be naturally halved into z1

1←e∗and z1

2←σ∗. In our experience, we ﬁnd that a single coupling375

layer is sufﬁcient for the problems solved herein. In this setup, the output feature vector becomes (z2)T=376

[(z2

1)T,(z2

2)T]and ¯e←z2

1and ¯

σ←z2

2. Also, to reduce the training computational cost, we set h1

1≡h1

2≡Jm

377

and f1

1=0mwhere Jmand 0mare vectors of size mﬁlled by ones and zeros, respectively. This architecture378

is equivalent to the network introduced in [Dinh et al.,2014].379

Utilizing this architecture, the previous multi-objective loss function in Eq. (19) becomes a single objec-380

tive optimization problem as follows:381

arg min

θ∈RNF

Ndata

∑

i=1||ˆ

σi−ˆ

Kﬁx ˆei||2

2, (25)

where ˆz=F(z∗;θ). In the following subsection, we showcase the application of the introduced local382

projection via a simple example.383

Algorithm 3 Query inference with the introduced local projection

1: Input: a query point (e,σ)

2: Map forward (e,σ)via Eqs. (20) and (21) to (e#,σ#).

3: Find the closest point (ˆe,ˆ

σ)on the constructed vector space to (e#,σ#)by Eqs. (16) and (17).

4: Map backward (ˆe,ˆ

σ)via the inverse operations in Eqs. (22) and (23) to (e∗,σ∗).

Remark 4 In all of the examples solved in this paper, we set ˆ

Kﬁx equal to the identity tensor which is384

corresponding to the regular l2-norm projection as discussed in Remark 2.385

Remark 5 Our proposed mapping operation Fis an embedding (not immersion) operation since it is glob-386

ally invertible and homeomorphism to its image.387

Remark 6 The introduced method in this paper relaxes the discrete optimization step of the original dis-388

tance minimization to a continuous local projection onto a constructed smooth hyperplane; hence it breaks389

the NP-hardness.390

Remark 7 The global embedding approach only requires one ofﬂine neural network training per data set to391

construct the hyperplane but does not require generations of new local space during the simulations. The392

trade-off is that embedding the manifold globally is, in general, more complex than creating a local linear393

space that preserves the convexity (cf.[He and Chen,2020]). Hence, a neural network architecture that394

possesses sufﬁcient expressibility is necessary to handle the global embedding case, in particular for high-395

dimensional cases such as micropolar elasticity or multiphysics problems. Other feasible global manifold396

Manifold embedding data-driven mechanics xv

approaches (which could potentially bypass this complexity in an ofﬂine manner but still leverage the397

simplicity of the Euclidean space) may require the constructions of atlas and coordinate charts (or patches)398

such that the co-domains of each chart, which are themselves Euclidean space intersecting with each other,399

can be collectively used to represent the manifold (see, for instance, Lin and Zha [2008] for a list of options).400

These alternatives are, nevertheless, out of the scope of this study.401

5.2 Illustrative examples 2402

Here, we walk through the details of the introduced local projection method via the simple 1D example403

used in Section 3.2.404

Fig. 7: Training performance of the invertible neural network used for the database shown in Fig. 2(a).

Recall that we construct the embedded vector space of the constitutive manifold in an ofﬂine manner405

based on the data points in an available database. The database we use is similar to the previous example,406

see Fig. 2(a). We utilize one invertible coupling layer with the internal function parameterized by an elu407

MLP [Clevert et al.,2015]. This MLP has three hidden layers. Each layer consists of 5 units initialized by the408

uniform Kaiming approach [He et al.,2015]. The objective is minimized by the Adam method [Kingma and409

Ba,2014] with the initial learning rate of 5e-3 in a full-batch manner. To enhance the training step, we use410

ReduceLROnPlateau learning scheduler of PyTorch library Paszke et al. [2019] to reduce the learning411

rate by the factor of 0.91 every 50 iterations after the ﬁrst 2000 iterations. The minimum learning rate is412

set to 1e-6. These hyperparameters are tuned manually with satisfactory results as shown in 7. A more413

rigorous hyperparameter tunning might, to some degrees, further improve the performance, but such an414

endeavor is not the focus of this study [Bardenet et al.,2013,Fuchs et al.,2021,Heider et al.,2021].415

Figure 7summarizes the training performance. As this result suggest, the invertible architecture ﬁnds a416

near-optimal mapping function that can approximately satisfy the introduced loss function in Eq. (25) with417

the error in the order of O(−6). For a better comparison, the mapped database in the constructed vector418

space is plotted in Fig. 2(b). The linearity of the data expressed in the coordinates of the embedded space419

is obvious.420

As discussed earlier, if a query point does not belong to the data manifold, it will be out of the con-421

structed hyperplane. Hence, we deﬁne a Euclidean projection to project the mapped query point onto the422

constructed hyperplane, see Eqs. (16) and (17). To show this and depict the effect of the mapping function,423

we present the deformation of a regular grid in the strain-stress space caused by the mapping function F424

in Fig. 8. The mapping function can be thought as a deformation of the domain of the actual data points425

(circular markers in Fig. 8(a)) such that the resultant deformed conﬁguration becomes a hyperplane, i.e., a426

straight line in 1D setup; see circular markers in Fig. 8(b). However, points out of the data manifold, i.e.,427

square markers in 8(a), will not be placed on the constructed hyperplane.428

xvi Bahador Bahmani, WaiChing Sun

(a) regular grid points in strain-stress space (b) grid deformation after mapping

Fig. 8: (a) a regular grid in the strain-stress space, (b) the deformation of this grid in the mapped domain.

We show the steps of our local projection method at the query time in Fig. 9, see Algorithm 3. In Fig. 9(a)429

the similar equilibrium state (depicted by a black square marker) to the problem in Fig. 3is chosen to be430

projected onto the data space. First, we map the query point, using Eqs. (20) and (21), onto the constructed431

vector space by the trained invertible neural network, see the black square marker in Fig. 9(b). Then, we432

ﬁnd the closest point on the vector space to the mapped query point by Eqs. (16) and (17), see the red433

star marker in Fig. 9(b). Finally, we map back to the actual data space via the inverse operations Eqs. (22)434

and (23), see red star marker in Fig. 9(c). As the ﬁnal result suggests, the proposed local projection can435

ﬁnd the closest material point in the interpolation region of the data space and admits the underlying436

manifold. Notice that, even qualitatively, the found closest point is closer than the found points by the437

nearest neighbor approach, c.f. Fig. 3, and still the found point belongs to the underlying data generator438

manifold (black curve line in Fig. 9).439

For a better comparison with the nearest neighbor method, we apply our introduced local projection440

method to the similar query points used in Fig. 4(a-c). The result is indicated in Fig. 4(d) where the found441

local projections by our method provide a smooth transition from one point to the other compared with Fig.442

4(a-c) where there were only ﬁve possible discrete choices. Our scheme can accurately interpolate between443

data points such that the projected values belong to the underlying data manifold.444

5.3 Illustrative examples 2: a sanity check through the lens of convex interpolation on the manifold445

Here, we examine the linearity of the constructed vector space by its convex interpolation capability. We446

claim that if the constructed space is truly vector space and the mapping function truly respects the un-447

derlying manifold, then any convex interpolation between two arbitrary points on the constructed space448

should result in a point on the actual manifold after the backward map. A convex interpolation between449

two points ˆzsand ˆzeis deﬁned as follows:450

ˆzα=αˆzs+ (1−α)ˆze;α∈(0, 1). (26)

We examine this property in Fig. 10. Start and end points ˆzsand ˆzeare shown by blue square markers.451

A red star marker depicts the interpolated point in the constructed vector space. The interpolated point452

in the actual data space is depicted as a blue star marker. As this experiment shows, the interpolation in453

the constructed vector space is equivalent to the interpolation over the underlying manifold. However, the454

regular interpolation in the actual data space results in points out of the underlying data manifold.455

Manifold embedding data-driven mechanics xvii

(a) step 0: requested query (b) step 1: forward map to a vector space

Fig. 9: Summary of the steps for the introduced local projection: (a) a query point shown by a black square

marker is requested to be projected onto the database. (b) the trained neural network is ﬁrst applied to

embed the database and the query point to the space that has vector space properties for database points.

Moreover, in this step, the mapped query point should be adjusted to the constructed vector space for the

database, see red star point in (b). (c) the adjusted point in (b) is ﬁnally mapped back to the real data space

via the inverse operation. The black curve line shows the underlying material manifold. Colored circle

points belong to the database.

Remark 8 The performance of a trained neural network (e.g., number of neurons, number of layers, type456

of activation) depends on the hyperparameters. The neural network embedding is no exception. In this457

work, the tuning of the hyperparameters is done manually and is sufﬁcient for our purpose, as evidenced458

by the loss function value at the end of the training. Other feasible and more rigorous approaches to ﬁne-459

tuning the best combination of parameters can be done by, for instance, the greedy search, random search460

[Bergstra and Bengio,2012], random forest [Probst et al.,2019], Bayesian optimization [Klein et al.,2017],461

metaheuristic or deep reinforcement learning [Fuchs et al.,2021,Heider et al.,2021]. Applying state-of-the-462

art tuning techniques for hyperparameters is out of the scope of this study, but the relevant information463

can be found in the aforementioned literature.464

6 Numerical Examples465

In this section, we benchmark four numerical problems to examine the accuracy and robustness of the pro-466

posed scheme in various scenarios. Importantly, we compare the efﬁciency of this method with the original467

distance minimization scheme in the limited data regime. First, in Section 6.1, we solve a 1D bar problem468

for two databases with different availability of data. Moreover, we compare the robustness of the proposed469

xviii Bahador Bahmani, WaiChing Sun

(a) α=0.2 (b) α=0.4

Fig. 10: Interpolation capability in the regular data space and the constructed vector space for different

values of αdeﬁned in Eq. (26). Interpolated points (red star markers) in the constructed vector space are

on the underlying material manifold. However, not surprisingly, the interpolation in the actual data space

results in out of manifold points (blue star markers).

invertible network with its vanilla autoencoder variant. Second, we use the same material database to470

solve a 3D truss system under loading-unloading conditions in Section 6.2. In this problem, we point to471

a circumstance where the original distance minimization may predict a spurious dissipation mechanism472

for an elastic material. Third, we compare the efﬁciency of both methods for a plane strain problem under473

small-strain assumption that possesses stress concentration due to the geometrical imperfection in Section474

6.3. In Section 6.4, we study the application of the proposed method for an anisotropic material under the475

ﬁnite-strain condition. Moreover, in Appendix C, we showcase and validate the application of the pro-476

posed method for a heat conduction problem which is studied in the literature as well. The databases will477

be publicly available for third-party validation exercises (upon acceptance of this manuscript).478

6.1 Nonlinear 1D bar code veriﬁcation479

The purpose of this example is to verify the implementation of the model. As pointed out by, for instance,480

Roache [2002], the method of manufactured solution (MMS) employed here for code veriﬁcation is a purely481

mathematical exercise where one examines whether the computer model is capable of robustly replicating482

a manufactured solution of a boundary value problem. Satisfying the manufactured solution is a necessary483

but not sufﬁcient for a successful modeling effort. In this example, our goal is to compare the accuracy and484

robustness of our proposed method and the original distance minimizing method [Kirchdoerfer and Ortiz,485

2016] by applying them to solve the same boundary value problem.486

Manifold embedding data-driven mechanics xix

The problem of interest is a classical 1D nonlinear elasticity problem under the quasi-static condition,487

as illustrated in Fig. 11. It contains a homogeneous bar composed of a nonlinear elastic material subjected488

to an external load. The material constitutive behavior is assumed to follow Eq. (27) deﬁned as:489

σ(e) = αmtanh(αse), (27)

where αmand αsare material parameters. The bar length is set to L=1m. The applied force at the right490

end is set to F=833.6546 N, and the left end is ﬁxed from the movement. The cross-sectional area of491

the bar is constant and equal to 1 mm2. The body force applied on the 1D bar is b(x) = −0.02αsαm[1−492

tanh2(0.02αsx)]. The governing equations in this problem follow the balance of linear momentum in the493

one-dimensional domain. The corresponding analytical MMS solution reads,

<latexit sha1_base64="g3vERbzU3kfXibT9Kek4KMf3Qbg=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKQL0FvXhMwDwgWcLspDcZMzu7zMyKIeQLvHhQxKuf5M2/cZLsQRMLGoqqbrq7gkRwbVz328mtrW9sbuW3Czu7e/sHxcOjpo5TxbDBYhGrdkA1Ci6xYbgR2E4U0igQ2ApGtzO/9YhK81jem3GCfkQHkoecUWOl+lOvWHLL7hxklXgZKUGGWq/41e3HLI1QGiao1h3PTYw/ocpwJnBa6KYaE8pGdIAdSyWNUPuT+aFTcmaVPgljZUsaMld/T0xopPU4CmxnRM1QL3sz8T+vk5rwyp9wmaQGJVssClNBTExmX5M+V8iMGFtCmeL2VsKGVFFmbDYFG4K3/PIqaV6UvUr5ul4pVW+yOPJwAqdwDh5cQhXuoAYNYIDwDK/w5jw4L86787FozTnZzDH8gfP5A+oZjQg=</latexit>

<latexit sha1_base64="TiLvefjg1Ce6pszniLxk2n8kbRE=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKQL0FBfGYgHlAsoTZSW8yZnZ2mZkVQsgXePGgiFc/yZt/4yTZgyYWNBRV3XR3BYng2rjut5NbW9/Y3MpvF3Z29/YPiodHTR2nimGDxSJW7YBqFFxiw3AjsJ0opFEgsBWMbmd+6wmV5rF8MOME/YgOJA85o8ZK9bteseSW3TnIKvEyUoIMtV7xq9uPWRqhNExQrTuemxh/QpXhTOC00E01JpSN6AA7lkoaofYn80On5MwqfRLGypY0ZK7+npjQSOtxFNjOiJqhXvZm4n9eJzXhlT/hMkkNSrZYFKaCmJjMviZ9rpAZMbaEMsXtrYQNqaLM2GwKNgRv+eVV0rwoe5Xydb1Sqt5kceThBE7hHDy4hCrcQw0awADhGV7hzXl0Xpx352PRmnOymWP4A+fzB55RjNY=</latexit>

<latexit sha1_base64="pj7Vh9GyVkna0pxsEqh7tiQpBAs=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe4koHZBGwuLBMwHJEfY28wla/b2jt09IYT8AhsLRWz9SXb+GzfJFZr4YODx3gwz84JEcG1c99vJra1vbG7ltws7u3v7B8XDo6aOU8WwwWIRq3ZANQousWG4EdhOFNIoENgKRrczv/WESvNYPphxgn5EB5KHnFFjpfp9r1hyy+4cZJV4GSlBhlqv+NXtxyyNUBomqNYdz02MP6HKcCZwWuimGhPKRnSAHUsljVD7k/mhU3JmlT4JY2VLGjJXf09MaKT1OApsZ0TNUC97M/E/r5Oa8MqfcJmkBiVbLApTQUxMZl+TPlfIjBhbQpni9lbChlRRZmw2BRuCt/zyKmlelL1K+bpeKVVvsjjycAKncA4eXEIV7qAGDWCA8Ayv8OY8Oi/Ou/OxaM052cwx/IHz+QOnaYzc</latexit>

b(x)

<latexit sha1_base64="AEJihnYJZfe99dgXbOVoBhgmpzE=">AAAB63icdVDLSgMxFM3UV62vqks3wSLUzZCUjtZd0Y3LCvYB7VAyaaYNTWaGJCOWob/gxoUibv0hd/6N6UNQ0QMXDufcy733BIng2iD04eRWVtfWN/Kbha3tnd294v5BS8epoqxJYxGrTkA0EzxiTcONYJ1EMSIDwdrB+Grmt++Y0jyObs0kYb4kw4iHnBIzk4Ly/Wm/WEIuOvOquAaR6yFcw54lFQ8jVIHYRXOUwBKNfvG9N4hpKllkqCBadzFKjJ8RZTgVbFropZolhI7JkHUtjYhk2s/mt07hiVUGMIyVrcjAufp9IiNS64kMbKckZqR/ezPxL6+bmrDmZzxKUsMiulgUpgKaGM4ehwOuGDViYgmhittbIR0RRaix8RRsCF+fwv9Jq+LiqntxUy3VL5dx5MEROAZlgME5qINr0ABNQMEIPIAn8OxI59F5cV4XrTlnOXMIfsB5+wS4v44P</latexit>

Fig. 11: One-dimensional bar with length L, body force b(x), and applied force F. We use 50 uniform ﬁnite

elements with linear basis functions to discretize the domain.

494

u(x) = 0.01x2;x∈[0, L], (28)

which is obtained for the constitutive law expressed in Eq. (27). This constitutive law is assumed to be the495

ground truth for veriﬁcation purposes only.496

6.1.1 Limited, complete database497

The material database is a set of data points synthesized from the model in Eq. (27) with parameters αm=498

1000 MPa and αs=60. The database is populated by sampling 41 equally spaced data points in the range499

e∈[−0.03, 0.03]. For comparison purposes, we refer it as the complete database.500

The original distance minimization method directly uses the database in its local optimization step.501

However, the method introduced in this paper requires an appropriate mapping F:R27→ R2to perform502

operations between the ambient data space and the mapped vector space, as explained in Sections 4and 5.503

This mapping is found in an ofﬂine manner by a single invertible layer which has three hidden elu layers504

[Clevert et al.,2015] with ﬁve hidden units per layer. Neural network weights and biases are initialized by505

the uniform Kaiming approach [He et al.,2015].506

The neural network parameters are found by ADAM optimizer [Kingma and Ba,2014] with the initial507

learning rate set at 0.05. We use ReduceLROnPlateau learning scheduler of PyTorch library Paszke508

et al. [2019] to adjust the learning rate every 50 iterations after the ﬁrst 2000 iterations. The learning rate509

reduction factor is set to 0.91 with the minimum learning rate 1e-6. Before training, strain and stress data510

points are linearly normalized based on their maximum and minimum values to be positive and less than511

or equal to 1. The training performance is shown in Fig. 12.512

Figure 13 plots the raw data points and the mapped one which has the vector space properties after513

training the neural network map function. The results conﬁrm that the proposed neural network architec-514

ture is able to ﬁnd an appropriate bijective function to map forward and backward between the original515

data space and its vector space counterpart.516

We compare the distribution of displacement, strain, and stress along the bar obtained from the em-517

bedding method and the original distance minimization method in Fig. 14. Generally speaking, both ap-518

proaches capture the general trends of the exact solution with the embedding method showing slightly519

more accurate predictions on the displacement ﬁeld. A major point of the departure between the two520

methods is the predicted strain ﬁeld where the original distance minimization method exhibits spurious521

oscillations in the strain ﬁeld, while the embedding method does not. This spurious non-smoothness has522

xx Bahador Bahmani, WaiChing Sun

Fig. 12: Mean squared error (mse) of the linearity condition violation during the training epochs for complete

data shown in Fig. 13(a). The training is performed in a full-batch manner.

(a) material database (b) mapped material database

Fig. 13: (a) material database (b) mapped material database into a vector space using the invertible neural

network. Colors show the data number.

been observed previously in the numerical solutions obtained from the model-free paradigm, such as Car-523

rara et al. [2020] and Bahmani and Sun [2021a], and is related to the discrete nature of the point cloud. Note524

that the spurious non-smoothness exhibited in the original distance minimization can also be suppressed525

by increasing the density of the data points in the parametric space. However, this non-smoothness may526

also become more severe when the available data is sparse. While limiting the data may increase the dif-527

ﬁculty in generating the correct inductive bias (see Williams et al. [2021] for a more in-depth discussion),528

the manifold embedding may overcome the spurious non-smoothness due to the continuous nature of the529

hyperplane.530

As a support for the previous claim, we plot the strain-stress state at each bar element for both schemes531

in Fig. 15. The found material states by the proposed scheme accurately resemble the underlying generative532

function that populates the material database. This observation indicates that the proposed scheme is able533

to indirectly learn and recover the underlying geometric structure of the material behavior.534

6.1.2 Limited, incomplete database535

To validate the robustness of the proposed scheme, we eliminate some data points from the nonlinear536

regions of the previous database and call the new database incomplete, shown in Fig. 16(a). We retrain the537

neural network map function for the new database, and the training performance is reported in Fig. 17; the538

architecture and hyper-parameters are kept the same as the ones used for the complete database.539

Manifold embedding data-driven mechanics xxi

(a) manifold embedding projection

(this work)

(b) manifold embedding projection

(this work)

(d) nearest neighbour projection (e) nearest neighbour projection (f) nearest neighbour projection

Fig. 14: A comparison between solution ﬁelds obtained by the proposed scheme and nearest neighbor

projection onto the database. The strain ﬁeld predictions by the proposed method are more accurate and

smooth; see (b) and (e).

(a) manifold embedding projection (this work) (b) nearest neighbour projection

Fig. 15: Strain-stress values obtained via (a) the introduced projection and (b) the nearest neighbor pro-

jection. The proposed scheme can identify material states that genuinely belong to the underline hidden

material manifold.

The predicted displacement, strain, and stress ﬁelds along the bar are shown in Fig. 18 for both meth-540

ods. As the results suggest, the accuracy of the proposed method in this paper is still satisfactory even541

for such an incomplete and limited database. However, as expected, the accuracy of the original distance542

minimization is considerably reduced, especially for displacement and strain ﬁelds (c.f. Fig. 14(d-e)).543

Similarly, the found strain-stress states in the entire domain are plotted as dot points in Fig. 19 to544

demonstrate the capabilities of the proposed manifold learning algorithm in recovering the embedded545

space of the underlying hidden constitutive manifold. Comparisons of results in Fig. 19(a) with Fig. 15(a)546

xxii Bahador Bahmani, WaiChing Sun

(a) material database (b) mapped material database

Fig. 16: (a) incomplete material database (b) mapped material database into a vector space using the invert-

ible neural network. Colors show the data number.

Fig. 17: Mean squared error (mse) of the linearity condition violation during the training epochs for incom-

plete data shown in Fig. 16(a). The training is performed in a full-batch manner.

reveal that the performance of the proposed manifold method is not signiﬁcantly affected by the missing547

data. On the other hand, the predicted state values obtained via the original distance minimization method548

are shown to be very sensitive to the missing data as evidenced in Fig. 19(b) and Fig. 15(b).549

This numerical experiment indicates that the knowledge of the geometry of the manifold gained from550

the neural network may improve the robustness of the data-driven predictions. Of course, the quality of the551

projections may still depend on how the missing data distributes, the smoothness of the data, and whether552

losing this data may otherwise signiﬁcantly alter the learned mapping between the constitutive manifold553

and the embedded hyperplane. It should be noticed that, while we did not implement the manifold learn-554

ing alternatives, we believe that the spurious oscillation pattern in the strain ﬁeld exhibited in Fig. 18(e)555

can also be suppressed when a global manifold is constructed [Kanno,2021], a locally embedding space is556

identiﬁed [He et al.,2021] or the data density is sufﬁciently high such that the projection is within a very557

small distance.558

6.1.3 Comparison with the vanilla autoencoder architecture559

Here, we aim to check the reconstruction error and the bijectivity constraint of the proposed invertible ar-560

chitecture compared with a vanilla autoencoder architecture. The details of the autoencoder map function561

and its optimization statement are described at the beginning of Section 5.562

Encoder and decoder branches of the autoencoder framework have three hidden layers with three hid-563

den units per layer. For a fair comparison between these two frameworks, this particular network conﬁgu-564

Manifold embedding data-driven mechanics xxiii

(a) manifold embedding projection

(this work)

(b) manifold embedding projection

(this work)

(d) nearest neighbour projection (e) nearest neighbour projection (f) nearest neighbour projection

Fig. 18: A comparison between solution ﬁelds obtained by the proposed scheme and nearest neighbor

projection. Even in the incomplete data scenario, the proposed scheme can ﬁnd more accurate responses

compared to the original distance minimization scheme.

(a) manifold embedding projection (this work) (b) nearest neighbour projection

Fig. 19: Strain-stress values obtained at each ﬁnite element via (a) the introduced projection and (b) the

nearest neighbor projection. The proposed scheme can identify material states that genuinely belong to the

underline hidden material manifold, even in the incomplete data-set scenario.

ration is chosen to be similar to the invertible internal network with almost the same number of unknown565

parameters. The total number of unknown parameters for the autoencoder and invertible networks are 82566

and 76, respectively. The rest of the hyper-parameters are similar to the invertible network reported earlier567

in subsection 6.1.1.568

Figure 20 shows the autoencoder training performance. With the same number of gradient descent569

iterations, the obtained error of the linearity constraint is almost similar to the invertible architecture (c.f.570

Fig. 12). However, the autoencoder network’s reconstruction error (backward map) is signiﬁcant relative to571

xxiv Bahador Bahmani, WaiChing Sun

Fig. 20: Mean squared error (mse) of the reconstruction and linearity constraint errors during the training

epochs for complete data shown in Fig. 16(a) using the vanilla autoencoder framework. The training is

performed in a full-batch manner. The number of training iterations is similar to the invertible network

training shown in Fig. 12.

the machine precision error of the invertible network, although it is small in the absolute sense. To compare572

the bijectivity property and possible accumulated round of errors, we study the reconstruction stability and573

accuracy of the trained networks during 200 consecutive forward and backward mappings for a batch of574

100 data points generated randomly from the constitutive manifold and not seen in the training. The results575

are plotted in Fig. 21. Not surprisingly, the accuracy of the invertible network remains constant around the576

machine precision, while the autoencoder accuracy rapidly decreases up until a saturation level. Notice577

that this experiment of consecutive backward-forward mappings is designed as a validation exercise to578

test the robustness of the forward and backward mappings and to detect any potential information loss579

when the forward and backward mapping occurs.580

(a) introduced invertible architecture (b) vanilla autoencoder architecture

Fig. 21: Comparing the stability of two architectures after 200 forward and backward mappings for 100

randomly generated data points not used in the network training. The reconstruction error of the tailored

architecture (a) is almost negligible and remains stable during continuously forward-backward mappings.

Since the invertibility of the forward and backward mapping may prevent the information loss, it is581

a property desirable for the cases where data set does not contain noise or error. This could be the case582

when the data are generated from direct numerical simulations, and the prediction is made for the consti-583

tutive responses of the corresponding effective medium. On the other hands, there are also circumstances584

in which the invertibility is purposely disabled in order to denoising data (e.g., data compression via the585

Manifold embedding data-driven mechanics xxv

singular value decomposition). In the case where data are acquired from experiments or sensors may con-586

tain both noise and outliers, these noises and outliers may increase the difﬁculty of the embedding through587

the neural network training. Presumably, this issue can be circumvented by de-noising or ﬁltering proce-588

dures performed before the embedding (cf. Lyu et al. [2019]). Interested readers may also be referred to,589

for instance, He et al. [2021] in which autoencoder is used. In this setting, the encoder is used to compress590

data into latent space and therefore ﬁlter noises, while the decoder is used to reconstruct the de-noised591

data such that the processed data is smooth and free of oscillation.592

6.2 Nonlinear truss system593

<latexit sha1_base64="VJdGHjdd+0fH0439rdnOjiixVJQ=">AAAB6HicdVDLSsNAFJ3UV62vqks3g0VwFSbaYLMriOCyBfuANpTJdNKOnUzCzESooV/gxoUibv0kd/6Nk7aCih64cDjnXu69J0g4UxqhD6uwsrq2vlHcLG1t7+zulfcP2ipOJaEtEvNYdgOsKGeCtjTTnHYTSXEUcNoJJpe537mjUrFY3OhpQv0IjwQLGcHaSM37QbmCbOR6roMgsl3keOc58bxa1XWhY6M5KmCJxqD83h/GJI2o0IRjpXoOSrSfYakZ4XRW6qeKJphM8Ij2DBU4osrP5ofO4IlRhjCMpSmh4Vz9PpHhSKlpFJjOCOux+u3l4l9eL9Vhzc+YSFJNBVksClMOdQzzr+GQSUo0nxqCiWTmVkjGWGKiTTYlE8LXp/B/0j6znartNauV+tUyjiI4AsfgFDjgAtTBNWiAFiCAggfwBJ6tW+vRerFeF60FazlzCH7AevsEXmSNWw==</latexit>

<latexit sha1_base64="Uv5Rfynn1qYC/96l9RBuDQXKlY8=">AAAB6HicdVDLSsNAFJ34rPVVdelmsAiuwkQbbHYFEVy2YB/QhjKZTtqxk0mYmQgh9AvcuFDErZ/kzr9x0lZQ0QMXDufcy733BAlnSiP0Ya2srq1vbJa2yts7u3v7lYPDjopTSWibxDyWvQArypmgbc00p71EUhwFnHaD6VXhd++pVCwWtzpLqB/hsWAhI1gbqZUNK1VkI9dzHQSR7SLHuyiI59VrrgsdG81RBUs0h5X3wSgmaUSFJhwr1XdQov0cS80Ip7PyIFU0wWSKx7RvqMARVX4+P3QGT40ygmEsTQkN5+r3iRxHSmVRYDojrCfqt1eIf3n9VId1P2ciSTUVZLEoTDnUMSy+hiMmKdE8MwQTycytkEywxESbbMomhK9P4f+kc247Ndtr1aqN62UcJXAMTsAZcMAlaIAb0ARtQAAFD+AJPFt31qP1Yr0uWles5cwR+AHr7RNc4I1a</latexit>

<latexit sha1_base64="PbI/TGYVOToDQ84nquPGP7QbT0E=">AAAB6HicdVDLSsNAFJ3UV62vqks3g0VwFSbaYLMriOCyBfuANpTJdNKOnUzCzEQsoV/gxoUibv0kd/6Nk7aCih64cDjnXu69J0g4UxqhD6uwsrq2vlHcLG1t7+zulfcP2ipOJaEtEvNYdgOsKGeCtjTTnHYTSXEUcNoJJpe537mjUrFY3OhpQv0IjwQLGcHaSM37QbmCbOR6roMgsl3keOc58bxa1XWhY6M5KmCJxqD83h/GJI2o0IRjpXoOSrSfYakZ4XRW6qeKJphM8Ij2DBU4osrP5ofO4IlRhjCMpSmh4Vz9PpHhSKlpFJjOCOux+u3l4l9eL9Vhzc+YSFJNBVksClMOdQzzr+GQSUo0nxqCiWTmVkjGWGKiTTYlE8LXp/B/0j6znartNauV+tUyjiI4AsfgFDjgAtTBNWiAFiCAggfwBJ6tW+vRerFeF60FazlzCH7AevsEW1yNWQ==</latexit>

<latexit sha1_base64="TXtP99XjcK5MUTDNSd28TIvyOrY=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKQL0FRPDgIQHzgGQJs5PeZMzs7DIzK4SQL/DiQRGvfpI3/8ZJsgdNLGgoqrrp7goSwbVx3W8nt7a+sbmV3y7s7O7tHxQPj5o6ThXDBotFrNoB1Si4xIbhRmA7UUijQGArGN3M/NYTKs1j+WDGCfoRHUgeckaNler3vWLJLbtzkFXiZaQEGWq94le3H7M0QmmYoFp3PDcx/oQqw5nAaaGbakwoG9EBdiyVNELtT+aHTsmZVfokjJUtachc/T0xoZHW4yiwnRE1Q73szcT/vE5qwit/wmWSGpRssShMBTExmX1N+lwhM2JsCWWK21sJG1JFmbHZFGwI3vLLq6R5UfYq5et6pVS9zeLIwwmcwjl4cAlVuIMaNIABwjO8wpvz6Lw4787HojXnZDPH8AfO5w+oUIzf</latexit>

<latexit sha1_base64="9shFCHGu+5DBBK0jdPBREWzNAiM=">AAAB6HicdVDJSgNBEK2JW4xb1KOXxiB4GmbGIcstIIrHBMwCyRB6Oj1Jm56F7h4hhHyBFw+KePWTvPk39iQRVPRBweO9Kqrq+QlnUlnWh5FbW9/Y3MpvF3Z29/YPiodHbRmngtAWiXksuj6WlLOIthRTnHYTQXHoc9rxJ5eZ37mnQrI4ulXThHohHkUsYAQrLTWvB8WSZdaqZcctI8u0rIrt2BlxKu6Fi2ytZCjBCo1B8b0/jEka0kgRjqXs2VaivBkWihFO54V+KmmCyQSPaE/TCIdUerPFoXN0ppUhCmKhK1JooX6fmOFQymno684Qq7H87WXiX14vVUHVm7EoSRWNyHJRkHKkYpR9jYZMUKL4VBNMBNO3IjLGAhOlsynoEL4+Rf+TtmParllruqX61SqOPJzAKZyDDRWoww00oAUEKDzAEzwbd8aj8WK8LltzxmrmGH7AePsE/0eNGw==</latexit>

<latexit sha1_base64="PkCXy9HQ5kYM5SyzhdCNghEIIno=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoN4KInhswdZCG8pmu2nXbjZhdyKU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYkUBl332ymsrW9sbhW3Szu7e/sH5cOjtolTzXiLxTLWnYAaLoXiLRQoeSfRnEaB5A/B+GbmPzxxbUSs7nGScD+iQyVCwShaqYn9csWtunOQVeLlpAI5Gv3yV28QszTiCpmkxnQ9N0E/oxoFk3xa6qWGJ5SN6ZB3LVU04sbP5odOyZlVBiSMtS2FZK7+nshoZMwkCmxnRHFklr2Z+J/XTTG88jOhkhS5YotFYSoJxmT2NRkIzRnKiSWUaWFvJWxENWVosynZELzll1dJ+6Lq1arXzVqlfpvHUYQTOIVz8OAS6nAHDWgBAw7P8ApvzqPz4rw7H4vWgpPPHMMfOJ8/5PCNBw==</latexit>

<latexit sha1_base64="TZrECy5T5zB+5j4Z4S41sMhchYo=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKQL0FRPGYgHlAsoTZSW8yZnZ2mZkVQsgXePGgiFc/yZt/4yTZgyYWNBRV3XR3BYng2rjut5NbW9/Y3MpvF3Z29/YPiodHTR2nimGDxSJW7YBqFFxiw3AjsJ0opFEgsBWMbmZ+6wmV5rF8MOME/YgOJA85o8ZK9bteseSW3TnIKvEyUoIMtV7xq9uPWRqhNExQrTuemxh/QpXhTOC00E01JpSN6AA7lkoaofYn80On5MwqfRLGypY0ZK7+npjQSOtxFNjOiJqhXvZm4n9eJzXhlT/hMkkNSrZYFKaCmJjMviZ9rpAZMbaEMsXtrYQNqaLM2GwKNgRv+eVV0rwoe5Xydb1Sqt5mceThBE7hHDy4hCrcQw0awADhGV7hzXl0Xpx352PRmnOymWP4A+fzB584jNk=</latexit>

<latexit sha1_base64="GMjsi4K9wtlR4iI3iQwpcWNdTeQ=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoN4KInisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpn57SeujYjVI04S7kd0qEQoGEUrPWDf7ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVa9Wvb6vVeq3eRxFOIFTOAcPLqEOd9CAJjAYwjO8wpsjnRfn3flYtBacfOYY/sD5/AEI242q</latexit>

2t0

<latexit sha1_base64="MxkyvOJfKUfLJIijPn7LTG9A1xI=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lKQb0VRPBYwX5AG8pmu2mX7iZhdyKU0r/gxYMiXv1D3vw3btoctPXBwOO9GWbmBYkUBl332ylsbG5t7xR3S3v7B4dH5eOTtolTzXiLxTLW3YAaLkXEWyhQ8m6iOVWB5J1gcpv5nSeujYijR5wm3Fd0FIlQMIqZVMOBOyhX3Kq7AFknXk4qkKM5KH/1hzFLFY+QSWpMz3MT9GdUo2CSz0v91PCEsgkd8Z6lEVXc+LPFrXNyYZUhCWNtK0KyUH9PzKgyZqoC26kojs2ql4n/eb0Uw2t/JqIkRR6x5aIwlQRjkj1OhkJzhnJqCWVa2FsJG1NNGdp4SjYEb/XlddKuVb169eahXmnc5XEU4QzO4RI8uIIG3EMTWsBgDM/wCm+Ocl6cd+dj2Vpw8plT+APn8wd6IY3m</latexit>

3t0

<latexit sha1_base64="GDQAXcnbi6LDM9MTf2UOEV9yqxk=">AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oN4KInisYD+gDWWz3bRLN5uwOxFK6V/w4kERr/4hb/4bN20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvs389hPXRsTqEScJ9yM6VCIUjGImXWLf7ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+lOqUTDJZ6VeanhC2ZgOeddSRSNu/On81hk5s8qAhLG2pZDM1d8TUxoZM4kC2xlRHJllLxP/87ophtf+VKgkRa7YYlGYSoIxyR4nA6E5QzmxhDIt7K2EjaimDG08JRuCt/zyKmldVL1a9eahVqnf5XEU4QRO4Rw8uII63EMDmsBgBM/wCm9O5Lw4787HorXg5DPH8AfO5w97qI3n</latexit>

(a) (b)

¯

<latexit sha1_base64="FFXuJK382iH9+2MeXbJMkxmtwQ8=">AAAB73icbVDLSgNBEOyNrxhfUY9eBoPgxbArAfUWEMVjBPOAZAmzk0kyZHZ2nekVwpKf8OJBEa/+jjf/xkmyB00saCiquunuCmIpDLrut5NbWV1b38hvFra2d3b3ivsHDRMlmvE6i2SkWwE1XArF6yhQ8lasOQ0DyZvB6HrqN5+4NiJSDziOuR/SgRJ9wShaqXXWCahObyfdYsktuzOQZeJlpAQZat3iV6cXsSTkCpmkxrQ9N0Y/pRoFk3xS6CSGx5SN6IC3LVU05MZPZ/dOyIlVeqQfaVsKyUz9PZHS0JhxGNjOkOLQLHpT8T+vnWD/0k+FihPkis0X9RNJMCLT50lPaM5Qji2hTAt7K2FDqilDG1HBhuAtvrxMGudlr1K+uq+UqjdZHHk4gmM4BQ8uoAp3UIM6MJDwDK/w5jw6L8678zFvzTnZzCH8gfP5A8ckj9U=</latexit>

<latexit sha1_base64="7v8tYHAYhPlWzOg1mRtjduY1bOw=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoN4KonisYD+gDWWy3bRLN5uwuxFK6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkRwbVz32ymsrW9sbhW3Szu7e/sH5cOjlo5TRVmTxiJWnQA1E1yypuFGsE6iGEaBYO1gfDPz209MaR7LRzNJmB/hUPKQUzRWavcCVNndtF+uuFV3DrJKvJxUIEejX/7qDWKaRkwaKlDrrucmxs9QGU4Fm5Z6qWYJ0jEOWddSiRHTfjY/d0rOrDIgYaxsSUPm6u+JDCOtJ1FgOyM0I73szcT/vG5qwis/4zJJDZN0sShMBTExmf1OBlwxasTEEqSK21sJHaFCamxCJRuCt/zyKmldVL1a9fqhVqnf5nEU4QRO4Rw8uIQ63EMDmkBhDM/wCm9O4rw4787HorXg5DPH8AfO5w9cxY+e</latexit>

Fig. 22: (a) Truss system under (b) quasi-static cyclic loading F. Horizontal axis in (b) indicates the step

number. We set ¯

F=3 kN, t0=20, and the entire loading-unloading takes 60 steps.

In this problem, we compare the efﬁciency, robustness, and accuracy of the proposed method against594

the nearest neighbor projection for a 3D truss structure that undergoes quasi-static, elastic loading-595

unloading, as shown in Fig. 22. The structure is loading at the top node by a linearly increasing compressive596

force to reach the maximum force ¯

F, then the compressive force is linearly unloaded up to a maximum ten-597

sile force ¯

Fas shown in Fig. 22(b). The continuum elasticity equations can be straightforwardly simpliﬁed598

for 3D truss elements, hence the details of the derivation are omitted for the sake of brevity; interested599

readers can follow reference books such as Fish and Belytschko [2007], Belytschko et al. [2014].600

The material database is similar to the complete data used in the previous problem (see Fig. 13), and we601

use the same neural network map function as the previous problem. The global optimization parameters602

and local ones (in the case of the original distance minimization method) are the same as the previous603

problem, i.e., C=S−1=42694.67MPa.604

Figure 23 depicts the force-displacement plots obtained by the proposed data-driven scheme and the605

original distance minimization method. Interestingly, the original distance minimization method shows606

spurious hysteresis behavior in the unloading stage which could be wrongly considered as an irreversible607

mechanism symptom by a not experienced practitioner, while the proposed scheme in this paper accurately608

predicts the smooth elastic behavior.609

The strain-stress pairs in each bar element in the truss system during the entire loading history are plot-610

ted in Fig. 24 to investigate the offset between the estimated material states (dot points) and the underlying611

material constitutive manifold (red line). As these plots suggest, the proposed scheme ﬁnds the material612

states that are truly on the underlying material manifold. However, the original distance minimization613

method cannot precisely recover the nonlinear behavior. Notice that this performance issue of the original614

distance minimization is not unexpected as it is known as a data demanding method due to its minimal615

assumption about the constitutive model [Bahmani and Sun,2021a]. In other words, it can converge to the616

true behavior in the big data regimes [Kirchdoerfer and Ortiz,2016,Conti et al.,2018].617

The stress values in each bar element for the maximum compressive and tensile applied forces are618

shown in Fig. 25 which are obtained by three solvers: classical model-based method, the introduced pro-619

jection method herein, and the original distance minimization method. There is a good agreement between620

xxvi Bahador Bahmani, WaiChing Sun

(a) manifold embedding projection (this work) (b) nearest neighbour projection

Fig. 23: force-displacement relation for the top node during the loading history via (a) the introduced

projection and (b) the nearest neighbor projection. The introduced approach does not suffer from a possible

spurious history dependence for an elastic material, in the limited data regime.

(a) introduced projection (b) nearest neighbour projection

Fig. 24: Strain-stress values obtained at each ﬁnite element during the loading history via (a) the introduced

projection and (b) the nearest neighbor projection. The founded material states by the introduced scheme

(a) follow the underlying, hidden material manifold, especially in the nonlinear regions.

the model-based method and the method introduced in this paper, while the original distance minimiza-621

tion method predicts inaccurate stress values in some elements; e.g., see left and right elements of the top622

Pyramid.623

6.3 Nonlinear elasticity624

Here, we show the effectiveness of the proposed method compared with the original distance minimization625

method in dealing with 2D nonlinear elasticity problems. The problem domain and boundary conditions626

are depicted in Fig. 26 where the bottom is ﬁxed from horizontal and vertical movements and a uniform627

vertical displacement uy=0.1m is applied over the top edge.628

We synthesize a database consisting of 1000 data points sampled from the following strain energy629

[Nguyen et al.,2020,Bahmani and Sun,2021a]:630

ψ(e) = 1

2(1+2ekk −2 log (1+ekk )) + 1

2(log (1+ekk ))2+eki eik. (29)

Manifold embedding data-driven mechanics xxvii

(a) model-based (b) manifold embedding projection

(this work)

(d) model-based (e) manifold embedding projection

(this work)

(f) nearest neighbour projection

Fig. 25: A comparison between solution ﬁelds obtained by the proposed scheme and nearest neighbor

projection onto the database. In (c) and (f), the brute-force distance minimization scheme provides wrong

stress states for the top left and right bars.

<latexit sha1_base64="QNDxlbr9HGGp5Q+ESWvFbwaiu/s=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe4koHYBGwuLiOYDkiPsbfaSJXt7x+6ccIT8BBsLRWz9RXb+GzfJFZr4YODx3gwz84JECoOu++0U1tY3NreK26Wd3b39g/LhUcvEqWa8yWIZ605ADZdC8SYKlLyTaE6jQPJ2ML6Z+e0nro2I1SNmCfcjOlQiFIyilR7SftYvV9yqOwdZJV5OKpCj0S9/9QYxSyOukElqTNdzE/QnVKNgkk9LvdTwhLIxHfKupYpG3PiT+alTcmaVAQljbUshmau/JyY0MiaLAtsZURyZZW8m/ud1Uwyv/IlQSYpcscWiMJUEYzL7mwyE5gxlZgllWthbCRtRTRnadEo2BG/55VXSuqh6ter1fa1Sv8vjKMIJnMI5eHAJdbiFBjSBwRCe4RXeHOm8OO/Ox6K14OQzx/AHzucPeyCN+w==</latexit>

<latexit sha1_base64="LzoXXbscziHCxLI6iaXqUSJN9RA=">AAAB6HicdVDJSgNBEK2JW4xb1KOXxiB4GmbikOUW0IPHBMwCyRB6Oj1Jm56F7h4xDPkCLx4U8eonefNv7EkiqOiDgsd7VVTV82LOpLKsDyO3tr6xuZXfLuzs7u0fFA+POjJKBKFtEvFI9DwsKWchbSumOO3FguLA47TrTS8zv3tHhWRReKNmMXUDPA6ZzwhWWmrdD4sly6zXKmWngizTsqp22c5IuepcOMjWSoYSrNAcFt8Ho4gkAQ0V4VjKvm3Fyk2xUIxwOi8MEkljTKZ4TPuahjig0k0Xh87RmVZGyI+ErlChhfp9IsWBlLPA050BVhP528vEv7x+ovyam7IwThQNyXKRn3CkIpR9jUZMUKL4TBNMBNO3IjLBAhOlsynoEL4+Rf+TTtm0HbPeckqNq1UceTiBUzgHG6rQgGtoQhsIUHiAJ3g2bo1H48V4XbbmjNXMMfyA8fYJStGNTA==</latexit>

<latexit sha1_base64="s0esQyEeOWHblNEo/mIkxPq+Wls=">AAAB6HicdVDLSsNAFJ34rPVVdelmsAiuQlJDH7uCLly2YB/QhjKZ3rRjJw9mJkII/QI3LhRx6ye582+ctBVU9MCFwzn3cu89XsyZVJb1Yaytb2xubRd2irt7+weHpaPjrowSQaFDIx6JvkckcBZCRzHFoR8LIIHHoefNrnK/dw9Csii8VWkMbkAmIfMZJUpL7XRUKltmo16tOFVsmZZVsyt2Tio159LBtlZylNEKrVHpfTiOaBJAqCgnUg5sK1ZuRoRilMO8OEwkxITOyAQGmoYkAOlmi0Pn+FwrY+xHQleo8EL9PpGRQMo08HRnQNRU/vZy8S9vkCi/7mYsjBMFIV0u8hOOVYTzr/GYCaCKp5oQKpi+FdMpEYQqnU1Rh/D1Kf6fdCum7ZiNtlNuXq/iKKBTdIYukI1qqIluUAt1EEWAHtATejbujEfjxXhdtq4Zq5kT9APG2ydMVY1N</latexit>

Fig. 26: Plane strain square with length 1m and a circular imperfection at its center with radius 0.15m. The

domain is discretized by 545 triangular ﬁnite elements.

Plane strain condition is assumed and a regular grid is used to sample the strain components in the ranges631

−0.335 ≤e11 ≤0.0155, 0.12 ≤e22 ≤1, and −0.03 ≤e12 ≤0.03.632

The mapping F:R67→ R6from the ambient space to the mapped space is performed by a single633

invertible layer consisting of three hidden elu layers with ten hidden units per layer. Neural network634

weights and biases are initialized by the uniform Kaiming approach.635

The neural network parameters are found by ADAM optimizer with an initial learning rate=0.002. We636

use ReduceLROnPlateau learning scheduler to adjust the learning rate every 50 iterations after the ﬁrst637

xxviii Bahador Bahmani, WaiChing Sun

Fig. 27: Mean squared error (mse) of the linearity condition violation during the training epochs for the 2D

nonlinear elasticity data. The training is performed in a full-batch manner with random shufﬂing at each

epoch.

1000 iterations. The learning rate reduction factor is set to 0.91 with the minimum learning rate 1e-6. Before638

training, strain and stress data points are linearly normalized based on their maximum and minimum639

values to be positive and less than or equal to 1. The training performance is shown in Fig. 27.640

The Cparameter in the global optimization step is set to the Hessian of the strain energy at zero strain,641

and Stensor is set to its inverse. The same parameters are used in the local optimization for the original642

distance minimization method.643

In Fig. 28, we show the predicted displacement, strain, and stress contours by the introduced manifold644

method in this paper, the original distance minimization method, and the classical model-based method.645

Although the improvement in the displacement ﬁeld predictions of the manifold approach over the nearest646

neighbor projection is marginal, the manifold method considerably enhances the strain and stress ﬁeld647

predictions, both in terms of the magnitude of the errors and the symmetry of the solutions.648

The strain ﬁeld obtained via the original distance minimization method (shown in Fig. 28 indicates649

that the discrete search does not yield a strain ﬁeld that even qualitatively describes the overall charac-650

teristics of the problem, e.g., strain concentration around the hole imperfection. Notice that this is not651

unexpected since the amount of data used in this problem is very limited; 1000 data points to sample a652

6-dimensional phase space is insufﬁcient for the original distance minimization scheme. Furthermore, it653

is also not difﬁcult to see that the solutions obtained from the nearest neighbor projection may lose the654

symmetry due to the bias induced by the limited choice of data pointed. This limitation can be overcome655

or at least suppressed by the usage of embedded space for projection where each query point is projected656

onto a hyperplane instead of a point from the material database.657

The proposed method is also conceptually different from the classical constitutive laws approach in658

which we merely implicitly leverage the generalization power of deep neural networks in high-dimensional659

space (cf. Balestriero et al. [2021], Belkin et al. [2019]) to gain knowledge on the manifold and utilize this660

acquired knowledge to generate a more precise notion of distance. We did not explicitly introduce a speciﬁc661

form or equations to curve-ﬁtting the data. As such, the resultant paradigm remains general and applicable662

for different mechanics problems.663

In Fig. 29, for the nearest neighbor projection method, the amount of data is gradually increased until664

we reach to almost similar error (for nodal displacements compared with classical model-based ﬁnite el-665

ement as the ground-truth) obtained by the global embedding projection method that utilizes a database666

consisting of only 103data points. Notice that (1) the classical approach needs 3 orders of magnitude more667

data than the proposed scheme to obtain a similar error in the displacement ﬁeld prediction. (2) assuming668

the availability of this amount of data, the simulation time (see Remark 9) is 1 order of magnitude more669

than the proposed scheme here. (3) the proposed scheme is almost insensitive to the random initialization,670

which increases robustness.671

Manifold embedding data-driven mechanics xxix

Displacement Norm √3J2of Strain Tensor von Mises Stress

Global Em-

bedding

Projection

(this study)

Nearest

Neighbor

Projection

Classical

FEM

Fig. 28: A comparison between solution ﬁelds obtained by the two data-driven methods, i.e., global embed-

ding (this study) and nearest neighbor projections, versus classical model-based nonlinear FEM (as ground

truth). The simulations are performed under the plane strain condition.

Remark 9 The simulation time in this problem refers to the actual boundary value problem simulation672

(Algorithm 1) obtained by the same ﬁnite element solver. The only difference is the scheme used in the673

local projection step. The solver is written in Python and was run on a MacBook laptop with a Quad-Core674

Intel Core i5 processor running at 1.4 GHz using 16 GB of RAM.675

6.4 Finite-strain anisotropic elasticity676

In this example, we showcase the application of the proposed method for transverse isotropic behavior677

in the ﬁnite-strain regime. As shown in Fig. 30, the uni-axial test is simulated for a synthesized material678

database populated from the transversely isotropic St. Venant hyperelasticity model [Bonet and Burton,679

1998] with the following strain energy functional:680

ψ=1

2λEI I EJJ +µEI J EI J + (α+β(IC−3) + γ(IVC−1))(IVC−1)−1

2α(VC−1)(30)

where IC=CKK is the ﬁrst invariant of the right Cauchy-Green deformation tensor C=FT·F, and681

λ,µ,α,β, and γare material properties. The forth and ﬁfth pseudo invariants of Care deﬁned as IVC=682

A·CA, and VC=A·C2A, respectively, where the unit vector Arepresents the principal ﬁber orientation of683

orthotropy in the reference undeformed conﬁguration. The database is generated by a uniform sampling of684

strains in ranges EXX ∈[−0.36, 0.33 ],EYY ∈[0.21, 0.75], and EXY ∈[−0.85, 0.1]with the number of samples685

41 ×41 ×21. In the data generation process, we use α=−0.4 GPa, β=−0.025 GPa, γ=0.025 GPa,686

µ=0.4 GPa, λ=0.6 GPa; these material properties correspond to an orthotropic material with the Elastic687

modulus in the ﬁber direction two times of the matrix material and the same Poisson’s ratio 0.25 for both688

ﬁber and matrix constituents [Bonet and Burton,1998]. The ﬁber orientation is set to 45 degrees.689

The invertible neural network has a coupling layer with three internal layers and 30 hidden neurons690

per layer. The initial ADAM learning rate is set to 0.005, and the learning decay scheduler is activated after691

500 epochs. The rest of the hyperparameters is kept the same as in the previous example. The training692

performance plots are shown in Fig. 31.693

Displacement and Lagrange multiplier ﬁelds in the weak statements, i.e., Eqs. (10) and (11), are dis-694

cretized with the standard linear basis functions via the Bubnov-Galerkin method. 812 uniform triangular695

xxx Bahador Bahmani, WaiChing Sun

Fig. 29: simulation time (left axis) and displacement error (right axis) comparisons between the global

embedding projection (GEP) and nearest neighbor projection (NNP) methods. The horizontal axis demon-

strates the different amounts of data used in the NNP simulations. Vertical bars at each point show 1.5

standard deviations among 10 randomly initialized simulations for a ﬁxed database. The markers indicate

the average among these 10 simulations. For the GEP method, we use the database with 103data points,

however, 10 randomly initialized simulations are also executed for this method. The average simulation

time and displacement error for the GEP method are plotted by dashed black and red lines, respectively.

The ﬁnal solution by the GEP method is almost insensitive to the initialization (zero standard deviation

in calculated errors). However, the mean and standard deviation values of simulation time for the GEP

method are 10.7 and 0.3 seconds, respectively.

elements discretize the undeformed reference domain, and the single point quadrature rule is used to cal-696

culate the underlying integrations. The simulations are conducted for 20 loading steps starting from zero697

to 0.7mm applied vertical displacement. The parameter CE= (SS)−1is set equal to the Hessian tensor of698

the isotropic St. Venant model with λ=0.6 GP and µ=0.4 GP.699

The vertical displacement versus reaction force curve on the loading side during the simulations is700

plotted in Fig. 32. There is a good agreement between both data-driven methods and the benchmark.701

However, the GEP method predictions are slightly better than the NNP method, especially in the initial702

loading steps. The benchmark solution is obtained by the classical model-based FEM.703

Figure 33 compares strain and stress responses at the last loading step. The non-homogeneous defor-704

mation of this anisotropic material is captured accurately for both methods compared with the benchmark.705

However, the stress ﬁeld prediction is less accurate for the NNP method. Accurate stress and strain ﬁelds706

are necessary for failure or buckling analysis since it directly affects the localization zone and stress or707

strain redistribution in the post localization regime. Notice that the observed deﬁciency can be resolved if708

the amount of data is increased.709

7 Conclusions710

We introduce a data-driven approach that employs the invertible neural network to embed nonlinear con-711

stitutive manifolds onto a hyperplane. This treatment provides a distance measure more consistent with712

the intrinsic property of the material data and therefore enables more robust data-driven predictions when713

available data is limited. By mapping data points onto a hyperplane, the distance minimization algorithm714

may leverage the ﬂatness and the continuous nature of the hyerplane to signiﬁcantly simplify the projec-715

tion step of the data-driven paradigm and bypass the cumbersome combinatorial search that may bear an716

increasing cost with the increasing amount of data.717

The speciﬁc choice of norms equipped by the phase space of material data can be considered as an718

inductive bias (assumption(s) that is used to predict the output of given input it has not encountered) that719

Manifold embedding data-driven mechanics xxxi

<latexit sha1_base64="e2mXgMUB2j4/GY279fWmaXCKjVk=">AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKQHMLeMkxAfOAZAmzk04yZnZ2mZkVwpIv8OJBEa9+kjf/xkmyB00saCiquunuCmLBtXHdbye3tb2zu5ffLxwcHh2fFE/P2jpKFMMWi0SkugHVKLjEluFGYDdWSMNAYCeY3i/8zhMqzSP5YGYx+iEdSz7ijBorNeuDYsktu0uQTeJlpAQZGoPiV38YsSREaZigWvc8NzZ+SpXhTOC80E80xpRN6Rh7lkoaovbT5aFzcmWVIRlFypY0ZKn+nkhpqPUsDGxnSM1Er3sL8T+vl5jRnZ9yGScGJVstGiWCmIgsviZDrpAZMbOEMsXtrYRNqKLM2GwKNgRv/eVN0r4pe5VytVkp1apZHHm4gEu4Bg9uoQZ1aEALGCA8wyu8OY/Oi/PufKxac042cw5/4Hz+AJ6kjM8=</latexit>

<latexit sha1_base64="/NziUWKRXlf8jEz2PuI/QNCejSo=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe4koOkCNhYWCZgPSI6wt5lL1uztHbt7Qgj5BTYWitj6k+z8N26SKzTxwcDjvRlm5gWJ4Nq47reT29jc2t7J7xb29g8Oj4rHJy0dp4phk8UiVp2AahRcYtNwI7CTKKRRILAdjG/nfvsJleaxfDCTBP2IDiUPOaPGSo37frHklt0FyDrxMlKCDPV+8as3iFkaoTRMUK27npsYf0qV4UzgrNBLNSaUjekQu5ZKGqH2p4tDZ+TCKgMSxsqWNGSh/p6Y0kjrSRTYzoiakV715uJ/Xjc14Y0/5TJJDUq2XBSmgpiYzL8mA66QGTGxhDLF7a2EjaiizNhsCjYEb/XlddK6KnuVcrVRKdWqWRx5OINzuAQPrqEGd1CHJjBAeIZXeHMenRfn3flYtuacbOYU/sD5/AGktIzT</latexit>

<latexit sha1_base64="NUPSM0GHcC1Ba3qZAkvi/cyOw5c=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0N4KXjxWtB/ShrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38z89hPXRsTqAScJ9yM6VCIUjKKV7tP+Y79ccavuHGSVeDmpQI5Gv/zVG8QsjbhCJqkxXc9N0M+oRsEkn5Z6qeEJZWM65F1LFY248bP5qVNyZpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0SjYEb/nlVdK6qHqX1drdZaVey+Mowgmcwjl4cAV1uIUGNIHBEJ7hFd4c6bw4787HorXg5DPH8AfO5w9E6Y3I</latexit>

<latexit sha1_base64="H/+C5vmEfWMmKGxRFiein5DwD9w=">AAAB6HicdVDJSgNBEO2JW4xb1KOXxiB4GmbimOUW8OIxAbNIMoSeTk3Spmehu0cIQ77AiwdFvPpJ3vwbe5IIKvqg4PFeFVX1vJgzqSzrw8itrW9sbuW3Czu7e/sHxcOjjowSQaFNIx6JnkckcBZCWzHFoRcLIIHHoetNrzK/ew9Csii8UbMY3ICMQ+YzSpSWWrfDYsky67VK2algy7Ssql22M1KuOhcOtrWSoYRWaA6L74NRRJMAQkU5kbJvW7FyUyIUoxzmhUEiISZ0SsbQ1zQkAUg3XRw6x2daGWE/ErpChRfq94mUBFLOAk93BkRN5G8vE//y+onya27KwjhRENLlIj/hWEU4+xqPmACq+EwTQgXTt2I6IYJQpbMp6BC+PsX/k07ZtB2z3nJKjctVHHl0gk7RObJRFTXQNWqiNqII0AN6Qs/GnfFovBivy9acsZo5Rj9gvH0CF1KNHg==</latexit>

<latexit sha1_base64="tScGRYT/9PmRSwGoiLFR97pPlFI=">AAAB6HicdVDLSsNAFJ3UV62vqks3g0VwFZIY+tgV3LhswT6gDWUyvWnHTh7MTIQS+gVuXCji1k9y5984aSuo6IELh3Pu5d57/IQzqSzrwyhsbG5t7xR3S3v7B4dH5eOTroxTQaFDYx6Lvk8kcBZBRzHFoZ8IIKHPoefPrnO/dw9Csji6VfMEvJBMIhYwSpSW2v1RuWKZjXrVcavYMi2rZjt2Tpyae+ViWys5KmiN1qj8PhzHNA0hUpQTKQe2lSgvI0IxymFRGqYSEkJnZAIDTSMSgvSy5aELfKGVMQ5ioStSeKl+n8hIKOU89HVnSNRU/vZy8S9vkKqg7mUsSlIFEV0tClKOVYzzr/GYCaCKzzUhVDB9K6ZTIghVOpuSDuHrU/w/6Tqm7ZqNtltpuus4iugMnaNLZKMaaqIb1EIdRBGgB/SEno0749F4MV5XrQVjPXOKfsB4+wQVgY0c</latexit>

⇥

<latexit sha1_base64="F3G8HQPddQdqLvy1yHbiGdQxZrA=">AAAB7XicdVDLSsNAFJ3UV62vqks3g0VwFZIY+tgV3Lis0Be0oUymk3bsZCbMTIQS+g9uXCji1v9x5984aSuo6IELh3Pu5d57woRRpR3nwypsbG5t7xR3S3v7B4dH5eOTrhKpxKSDBROyHyJFGOWko6lmpJ9IguKQkV44u8793j2Rigre1vOEBDGacBpRjLSRusP2lGg0Klccu1Gven4VOrbj1FzPzYlX86986BolRwWs0RqV34djgdOYcI0ZUmrgOokOMiQ1xYwsSsNUkQThGZqQgaEcxUQF2fLaBbwwyhhGQpriGi7V7xMZipWax6HpjJGeqt9eLv7lDVId1YOM8iTVhOPVoihlUAuYvw7HVBKs2dwQhCU1t0I8RRJhbQIqmRC+PoX/k65nu77duPUrzcY6jiI4A+fgErigBprgBrRAB2BwBx7AE3i2hPVovVivq9aCtZ45BT9gvX0C1DGPTQ==</latexit>

Fig. 30: Uni-axial test domain with length L=1mm and height H=2mm. The domain is assumed to be

reinforced by ﬁbrous materials with orientation Θdegree. The applied load is set to uY=0.7mm, and the

sample remains in plane-strain condition during the experiment.

Fig. 31: Mean squared error (mse) of the linearity condition violation during the training epochs for the

anisotropic hyperelastic data when the ﬁber orientation is 45 degrees. The training is performed with batch

sizes of 2000 and random shufﬂing at each epoch.

affects the quality of the predictions. Not surprisingly, inductive bias does not only persists in training720

neural network constitutive law, symbolic regression but is often introduced in hand-crafting a constitu-721

tive law (e.g., Occam’s razor, assumptions of smoothness, convexity), even though it is not always precisely722

recognized and consciously understood in the literature. The manifold embedding enables us to refocus723

on generating the inductive bias that helps us interpreting and examining the geometry of the data, while724

taking advantage of the simplicity of the Euclidean distance measure only available in the Euclidean space.725

This apparently subtle difference is nevertheless a crucial step for a wide spectrum of tasks highly relevant726

for physics and mechanics predictions because insights on the geometry of the data may play an impor-727

tant role in the quality and even correctness of these predictions. Examples of these applications include728

uncertainty quantiﬁcation [Giovanis and Shields,2018], de-noising [Lyu et al.,2019,Fefferman et al.,2020],729

and predictions with missing data [Mishne et al.,2019].730

There are also a few other research directions that are important but have not yet been explored in this731

paper, such as the construction of isometric embedding to directly measure the geodesic distance between732

data points, the search of optimal hyperparameters for the ofﬂine training, and the de-noising strategy733

on manifolds. Note that the proposed global embedding method is not only useful for delivering model-734

xxxii Bahador Bahmani, WaiChing Sun

Fig. 32: Displacement-force curve for different methods during 20 loading steps in the uni-axial test.

free simulation results, but can be used as a mean to encrypt material data in the mapped database. In735

such a case, the trained neural network can be used as a key to unlock the material data into interpretable736

data whereas the true identity of the materials can be masked in the mapped database. The manifold737

embedding can be used for applications that require proprietary data, or data of privacy concerns. These738

research directions will be pursued in the future but are considered out of the scope of this work.739

8 Acknowledgments740

We thank the two anonymous reviewers for their insightful feedback and helpful suggestions. Fruitful dis-741

cussions with Ran Ma are gratefully acknowledged. The authors are supported by the National Science742

Foundation under grant contracts CMMI-1846875 and OAC-1940203, and the Dynamic Materials and In-743

teractions Program from the Air Force Ofﬁce of Scientiﬁc Research under grant contracts FA9550-21-1-0391744

and FA9550-21-1-0027. These supports are gratefully acknowledged. The views and conclusions contained745

in this document are those of the authors, and should not be interpreted as representing the ofﬁcial poli-746

cies, either expressed or implied, of the sponsors, including the U.S. Government. The U.S. Government is747

authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright748

notation herein.749

Appendix A Manifold embedding in machine learning750

We aim to leverage the expressibility of neural networks to ﬁnd a global Euclidean embedding with desir-751

able properties. Hence, the ﬁrst important question is the existence of such an embedding.752

Whitney embedding theorem [Whitney,1936,1944] proves the existence of a function F:M 7→ Rn

753

that maps a sufﬁciently smooth manifold Mwith intrinsic dimension mto a vector (Euclidean) space of754

dimension n≥nwhitney where nwhitney =2m, provided that Mis not a real projective space. Nash proves755

the existence of such a mapping even with the isometry (distance-preserving) restriction for n≥nnash

756

where nnash >nwhitney, known as Nash–Kuipe embedding theorem [Nash,1954,Kuiper,1955]. G ¨

unther757

ﬁnds a tighter lower bound on the dimensionality of the Euclidean space nnash ≥max{m(m+1)/2, m(m+758

3)/2 +5}in the Nash embedding set-up [G¨

unther,1991].759

Although these theoretical results show the existence of such mappings, the remaining question is the760

feasibility of ﬁnding such mappings from an algorithmic perspective. [Baraniuk and Wakin,2009,Clark-761

son,2008] propose algorithms (with theoretical guarantees) based on the random projection methods to762

achieve an approximate isometric mapping. Also, some previous studies empirically show the feasibil-763

ity of ﬁnding such mapping functions such as Isomap [Tenenbaum et al.,2000], Locally Linear Embed-764

ding [Roweis and Saul,2000], Hessian Eigenmapping [Donoho and Grimes,2003], Laplacian Eigenmaps765

Manifold embedding data-driven mechanics xxxiii

GEP (this study) NNP Benchmark

√3J2of Strain Tensor

Von-misses Stress

Fig. 33: A comparison between solution ﬁelds obtained by the two data-driven methods, i.e., global em-

bedding (this study) and nearest neighbor projections, versus classical model-based nonlinear FEM (as the

benchmark). The simulations are performed under the plane strain condition. The computational domain

is the same for all methods and is shown in the top right contour image.

[Belkin and Niyogi,2003], Local Tangent Space Alignment [Zhang and Zha,2004], Local Fisher’s Discrim-766

inant Method [Sugiyama,2007], Riemannian Manifold Learning [Lin and Zha,2008], and Local Manifold767

Learning-Based k-Nearest-Neighbor [Ma et al.,2010] among many others.768

The scope of our work remains in the Whitney embedding setup, which is less restrictive than Nash-769

Kuipe embedding that requires the isometry condition. Notice that, although the isometry property is an770

essential condition to preserve the geometrical structure, there is a trade-off between the dimensionality of771

the target Euclidean space and the isometry restriction, i.e., nnash >nwhitney. Also, constructing a proper772

mapping function that preserves metrics between two spaces is more challenging and ongoing research.773

A.1 Related works on machine learning for manifold data774

Courty et al. [2017] propose a method based on the Siamese architecture [Chopra et al.,2005] to approxi-775

mately embed the Wasserstein metric into the Euclidean metric. This method ﬁnds the backward map from776

the Euclidean metric to the initial Wasserstein metric in an autoencoder fashion [Hinton and Salakhutdi-777

nov,2006] that is approximately invertible. A similar idea is used for point cloud embedding to speed778

up the Wasserstein distance calculations [Kawano et al.,2020]. Xiao et al. [2018] embed sub-samples of779

datasets with an arbitrary metric into the Euclidean metric to address mode collapse for generative adver-780

sarial networks (GANs) [Goodfellow et al.,2014]. Bramburger et al. [2021] aim to parametrize the Poincar´

e781

map by a deep autoencoder architecture that enforces invertibility as a soft constraint.782

xxxiv Bahador Bahmani, WaiChing Sun

Otto and Rowley [2019], Lusch et al. [2018], Gin et al. [2021] use deep autoencoder architecture to ﬁnd a783

coordinate transformation that linearizes a nonlinear partial differential equation (PDE). [Lee and Carlberg,784

2020] use deep convolutional autoencoders to project a dynamical system onto a nonlinear manifold of785

fewer dimensions instead of the classical approach of ﬁnding a linear subspace. Kim et al. [2021] construct786

a reduced-order model based on the autoencoder architecture with shallow multilayer perceptrons (MLPs)787

to ﬁnd a proper linear subspace for a nonlinear PDE that describes advection-dominated phenomena. To788

the best of our knowledge, the current work is the ﬁrst paper that leverages the bijectivity of the invertible789

neural network to generate the desired embedding space for the data-driven paradigm.790

Appendix B Terminology for data-driven mechanics791

Deﬁnition 1 (vector space) A vector space is a set of objects that are closed under vector addition and792

scalar multiplication with commutativity, associativity, additive identity, additive inverse, multiplicative793

identity, and distributive properties axioms Axler [1997].794

Deﬁnition 2 (hyperplane) A hyperplane is a set of points (objects) p∈Rnthat satisfy c·p=0 for a non-795

zero point cand any arbitrary point pbelongs to the hyperplane. The vector space’s axioms (properties)796

can be easily veriﬁed for hyperplane; hence, it is a vector subspace. The vector cindicates the normal797

direction to the hyperplane.798

Deﬁnition 3 (equilibrium manifold) The set of all equilibrium states belongs to a manifold called equi-799

librium manifold. Particularly, in this work , we assume this manifold (denoted as C) is smooth and con-800

tinuous.801

Deﬁnition 4 (material database) A set of multi-dimensional data points (e.g., pair of strain-stress tensors802

in solid mechanics) collected from experiments or ﬁner-scale simulations, denoted as Dthat records the803

local constitutive responses.804

Deﬁnition 5 (mapped material database) A set of new points obtained after transforming (mapping) the805

points from the material database onto a hyperplane, which is denoted as ˆ

D. We use the terms transforma-806

tion and mapping interchangeably throughout this work.807

Deﬁnition 6 (material state) A state (denoted as z∗) (e.g., a particular pair of strain-stress data) that be-808

longs to the material database, i.e. z∗∈ D. In general, z∗is not necessarily admissible in the equilibrium809

manifold C.810

Deﬁnition 7 (mapped material state) A state (denoted as ˆz) that belongs to the mapped material database,811

i.e. ˆz∈ˆ

D. ˆzis admissible to a hyperplane generated from an invertible neural network that maps the812

material data point cloud onto a hyperplane, i.e. F:D → ˆ

Dwith the corresponding inverse mapping813

G:ˆ

D → D.814

Deﬁnition 8 (equilibrium state) A state (denoted as z) that satisﬁes the equilibrium equation, i.e. z∈ C.815

In general, zis not necessarily an element in the material database.816

Appendix C Nonlinear heat conduction benchmark817

In this section, we provide an additional nonlinear heat condition problem as a benchmark. In this problem,818

a square domain is composed of a material of nonlinear thermal conductivity where Dirichlet boundary819

condition is prescribed, as shown in Fig. 34. Due to the simplicity of this boundary value problem, it was820

used in previous work such as Nguyen et al. [2020] as to verify the data-free paradigm. As such, we adopt821

this example in the appendix to examine the performance of the manifold embedding solver in the limited822

data scenario and compare it with the classical one based on the nearest neighbor projection.823

At the steady-state, the heat conduction governing equation reads824

∇x·q(x,y) + s(x,y) = 0, (31)

Manifold embedding data-driven mechanics xxxv

Fig. 34: The square domain of length 1m for the heat conduction problem with pure Dirichlet boundary

and non-zero heat source. The domain is discretized by 226 triangular ﬁnite elements.

where q(x,y)and s(x,y)are the heat ﬂux, and the heat source/sink term, respectively. The manufactured825

solution for the temperature ﬁeld [Nguyen et al.,2020]826

T(x,y) = 1

2sin (2πx)y(1−y), (32)

satisﬁes the governing equations with the following source term827

s(x,y) = 2π2y(1−y)sin(2πx)[1−tanh2(πy(1−y)cos(2πx) )] + sin(2πx)[1−tanh2(1

2sin(2πx)(1−2y))],

(33)

when the constitutive model is chosen as828

q=tanh (∇xT). (34)

For the global projection formulation, we follow the model-free derivation of the heat conduction prob-829

lem [Nguyen et al.,2020]. The extension of manifold embedding projection for the heat conduction problem830

in the phase space of (∇xT−q)is straightforward and omitted for the sake of brevity.831

To populate the database from the assumed constitutive model in Eq. (34), a regular grid with size 202

832

is used for temperature gradient components in the ranges −1≤∂T/∂x≤1 and −1≤∂T/∂y≤1. The C833

parameter for global optimization is set to 0.42Iwhere Iis the second-order identity tensor, and Stensor834

is set to its inverse as suggested in [Nguyen et al.,2020,Bahmani and Sun,2021a]. The original distance835

minimization scheme requires similar parameters for the local optimization which is set equal to the global836

optimization parameters.837

For the proposed scheme, we should ﬁrst ﬁnd an appropriate map function F:R47→ R4to perform838

operations between the original data space and the mapped space with the vector space properties. This839

step is done ofﬂine by training an invertible neural network map function. For this database, we design an840

invertible neural network consisting of four elu layers, each has ten hidden units. The uniform Kaiming841

approach initializes the network weights and biases.842

ADAM optimizer iterates to ﬁnd close to optimal parameters for the mapping function with an initial843

learning rate=0.002. We use ReduceLROnPlateau learning scheduler to adjust the learning rate every 50844

iterations after the ﬁrst 500 iterations. The learning rate reduction factor is set to 0.91 with the minimum845

learning rate 1e-6. Before training, temperature gradient and heat ﬂux data points are linearly normalized846

based on their maximum and minimum values to be positive and less than or equal to 1. The training847

performance is shown in Fig. 35.848

As shown in Fig. 36, there is a good agreement between the proposed scheme and the ground-truth849

solution. The heat ﬂux predictions by the original distance minimization method are less accurate than the850

proposed method in the limited data regime. Notice that, the spurious oscillation of the heat ﬂux and the851

break of symmetry exhibited in the solution obtained from the original distance minimization scheme is852

attributed to the limited data available in the database. If the database is sufﬁciently large, both issues may853

become less severe and may converge to the benchmark solution. [Kirchdoerfer and Ortiz,2016,Leygue854

et al.,2018,Nguyen et al.,2020,Bahmani and Sun,2021a].855

xxxvi Bahador Bahmani, WaiChing Sun

Fig. 35: Mean squared error (mse) of the linearity condition violation during the training epochs for the 2D

nonlinear heat transfer data. Random mini-batches of the size 200 are used during the training.

References856

Lynton Ardizzone, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W Pellegrini, Ralf S Klessen, Lena857

Maier-Hein, Carsten Rother, and Ullrich K ¨

othe. Analyzing inverse problems with invertible neural net-858

works. arXiv preprint arXiv:1808.04730, 2018.859

Sheldon Jay Axler. Linear algebra done right, volume 2. Springer, 1997.860

Santiago Badia, Michael Parks, Pavel Bochev, Max Gunzburger, and Richard Lehoucq. On atomistic-to-861

continuum coupling by blending. Multiscale Modeling & Simulation, 7(1):381–406, 2008.862

Bahador Bahmani and WaiChing Sun. A kd-tree-accelerated hybrid data-driven/model-based approach863

for poroelasticity problems with multi-ﬁdelity multi-physics data. Computer Methods in Applied Mechanics864

and Engineering, 382:113868, 2021a.865

Bahador Bahmani and WaiChing Sun. Training multi-objective/multi-task collocation physics-informed866

neural network with student/teachers transfer learnings. arXiv preprint arXiv:2107.11496, 2021b.867

Pierre Baldi. Autoencoders, unsupervised learning, and deep architectures. In Proceedings of ICML workshop868

on unsupervised and transfer learning, pages 37–49. JMLR Workshop and Conference Proceedings, 2012.869

Randall Balestriero, Jerome Pesenti, and Yann LeCun. Learning in high dimension always amounts to870

extrapolation. arXiv preprint arXiv:2110.09485, 2021.871

Richard G Baraniuk and Michael B Wakin. Random projections of smooth manifolds. Foundations of com-872

putational mathematics, 9(1):51–77, 2009.873

R´

emi Bardenet, M´

aty´

as Brendel, Bal´

azs K´

egl, and Michele Sebag. Collaborative hyperparameter tuning.874

In International conference on machine learning, pages 199–207. PMLR, 2013.875

Jan Jetze Beitler, Ivan Sosnovik, and Arnold Smeulders. Pie: Pseudo-invertible encoder. arXiv preprint876

arXiv:2111.00619, 2021.877

Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data represen-878

tation. Neural computation, 15(6):1373–1396, 2003.879

Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning prac-880

tice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116(32):881

15849–15854, 2019.882

Ted Belytschko, Wing Kam Liu, Brian Moran, and Khalil Elkhodary. Nonlinear ﬁnite elements for continua883

and structures. John wiley & sons, 2014.884

Yoshua Bengio, Yann LeCun, et al. Scaling learning algorithms towards ai. Large-scale kernel machines, 34885

(5):1–41, 2007.886

James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of machine887

learning research, 13(2), 2012.888

Christopher M Bishop. Pattern recognition. Machine learning, 128(9), 2006.889

Manifold embedding data-driven mechanics xxxvii

(a) manifold embedding projection

(this work)

(b) manifold embedding projection

(this work)

(d) nearest neighbour projection (e) nearest neighbour projection (f) nearest neighbour projection

(g) model-based (h) model-based (i) model-based

Fig. 36: A comparison between solution ﬁelds (temperature, temperature gradient magnitude, and heat

ﬂux magnitude) obtained by (a-c) the proposed projection, (d-f) nearest neighbor projection, and (g-i) clas-

sical model-based nonlinear ﬁnite element solver. The introduced scheme improves heat ﬂux prediction.

J Bonet and AJ0944 Burton. A simple orthotropic, transversely isotropic hyperelastic constitutive equation890

for large strain computations. Computer methods in applied mechanics and engineering, 162(1-4):151–164,891

1998.892

Jason J Bramburger, Steven L Brunton, and J Nathan Kutz. Deep learning of conjugate mappings. arXiv893

preprint arXiv:2104.01874, 2021.894

Pietro Carrara, Laura De Lorenzis, Laurent Stainier, and Michael Ortiz. Data-driven fracture mechanics.895

Computer Methods in Applied Mechanics and Engineering, 372:113390, 2020.896

Sumit Chopra, Raia Hadsell, and Yann LeCun. Learning a similarity metric discriminatively, with ap-897

plication to face veriﬁcation. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern898

Recognition (CVPR’05), volume 1, pages 539–546. IEEE, 2005.899

xxxviii Bahador Bahmani, WaiChing Sun

Kenneth L Clarkson. Tighter bounds for random projections of manifolds. In Proceedings of the twenty-fourth900

annual symposium on Computational geometry, pages 39–48, 2008.901

Djork-Arn´

e Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning902

by exponential linear units (elus). arXiv preprint arXiv:1511.07289, 2015.903

Sergio Conti, Stefan M¨

uller, and Michael Ortiz. Data-driven problems in elasticity. Archive for Rational904

Mechanics and Analysis, 229(1):79–123, 2018.905

Nicolas Courty, R´

emi Flamary, and M´

elanie Ducoffe. Learning wasserstein embeddings. arXiv preprint906

arXiv:1710.07457, 2017.907

Miles Cranmer, Alvaro Sanchez Gonzalez, Peter Battaglia, Rui Xu, Kyle Cranmer, David Spergel, and908

Shirley Ho. Discovering symbolic models from deep learning with inductive biases. Advances in Neural909

Information Processing Systems, 33:17429–17442, 2020.910

Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation.911

arXiv preprint arXiv:1410.8516, 2014.912

Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint913

arXiv:1605.08803, 2016.914

David L Donoho and Carrie Grimes. Hessian eigenmaps: Locally linear embedding techniques for high-915

dimensional data. Proceedings of the National Academy of Sciences, 100(10):5591–5596, 2003.916

Robert Eggersmann, Trenton Kirchdoerfer, Stefanie Reese, Laurent Stainier, and Michael Ortiz. Model-free917

data-driven inelasticity. Computer Methods in Applied Mechanics and Engineering, 350:81–99, 2019.918

Robert Eggersmann, Laurent Stainier, Michael Ortiz, and Stefanie Reese. Efﬁcient data structures for919

model-free data-driven computational mechanics. Computer Methods in Applied Mechanics and Engineer-920

ing, 382:113855, 2021a.921

Robert Eggersmann, Laurent Stainier, Michael Ortiz, and Stefanie Reese. Model-free data-driven compu-922

tational mechanics enhanced by tensor voting. Computer Methods in Applied Mechanics and Engineering,923

373:113499, 2021b.924

Wolfgang Ehlers and Bernd Scholz. An inverse algorithm for the identiﬁcation and the sensitivity analysis925

of the parameters governing micropolar elasto-plastic granular material. Archive of Applied Mechanics, 77926

(12):911–931, 2007.927

Charles Fefferman, Sergei Ivanov, Matti Lassas, and Hariharan Narayanan. Reconstruction of a riemannian928

manifold from noisy intrinsic distances. SIAM Journal on Mathematics of Data Science, 2(3):770–808, 2020.929

Jacob Fish and Ted Belytschko. A ﬁrst course in ﬁnite elements. Wiley, 2007.930

Alexander Fuchs, Yousef Heider, Kun Wang, WaiChing Sun, and Michael Kaliske. Dnn2: A hyper-931

parameter reinforcement learning game for self-design of neural network based elasto-plastic consti-932

tutive descriptions. Computers & Structures, 249:106505, 2021.933

Cristian Guillermo Gebhardt, Dominik Schillinger, Marc Christian Steinbach, and Raimund Rolfes. A934

framework for data-driven structural analysis in general elasticity based on nonlinear optimization: The935

static case. Computer Methods in Applied Mechanics and Engineering, 365:112993, 2020.936

J Ghaboussi, JH Garrett Jr, and Xiping Wu. Knowledge-based modeling of material behavior with neural937

networks. Journal of engineering mechanics, 117(1):132–153, 1991.938

Craig Gin, Bethany Lusch, Steven L Brunton, and J Nathan Kutz. Deep learning models for global coordi-939

nate transformations that linearise pdes. European Journal of Applied Mathematics, 32(3):515–539, 2021.940

Dimitris G Giovanis and Michael D Shields. Uncertainty quantiﬁcation for complex systems with very941

high dimensional response using grassmann manifold variations. Journal of Computational Physics, 364:942

393–415, 2018.943

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron944

Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing945

systems, 27, 2014.946

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.947

Matthias G ¨

unther. Isometric embeddings of Riemannian manifolds. In Proceedings of the International948

Congress of Mathematicians, Vol. I, II (Kyoto, 1990), pages 1137–1143. Math. Soc. Japan, Tokyo, 1991.949

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectiﬁers: Surpassing human-950

level performance on imagenet classiﬁcation. In Proceedings of the IEEE international conference on computer951

vision, pages 1026–1034, 2015.952

Manifold embedding data-driven mechanics xxxix

Qizhi He and Jiun-Shyan Chen. A physics-constrained data-driven approach based on locally convex953

reconstruction for noisy database. Computer Methods in Applied Mechanics and Engineering, 363:112791,954

2020.955

Xiaolong He, Qizhi He, and Jiun-Shyan Chen. Deep autoencoders for physics-constrained data-driven956

nonlinear materials modeling. Computer Methods in Applied Mechanics and Engineering, 385:114034, 2021.957

Yousef Heider, Hyoung Suk Suh, and WaiChing Sun. An ofﬂine multi-scale unsaturated poromechanics958

model enabled by self-designed/self-improved neural networks. International Journal for Numerical and959

Analytical Methods in Geomechanics, 45(9):1212–1237, 2021.960

Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural net-961

works. science, 313(5786):504–507, 2006.962

Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal963

approximators. Neural networks, 2(5):359–366, 1989.964

Thomas JR Hughes, Gonzalo R Feij´

oo, Luca Mazzei, and Jean-Baptiste Quincy. The variational multiscale965

method—a paradigm for computational mechanics. Computer methods in applied mechanics and engineer-966

ing, 166(1-2):3–24, 1998.967

Ruben Iba ˜

nez, Domenico Borzacchiello, Jose Vicente Aguado, Emmanuelle Abisset-Chavanne, Elias Cueto,968

Pierre Ladeveze, and Francisco Chinesta. Data-driven non-linear elasticity: constitutive manifold con-969

struction and problem discretization. Computational Mechanics, 60(5):813–826, 2017.970

Rub´

en Ibanez, Emmanuelle Abisset-Chavanne, Jose Vicente Aguado, David Gonzalez, Elias Cueto, and971

Francisco Chinesta. A manifold learning approach to data-driven computational elasticity and inelastic-972

ity. Archives of Computational Methods in Engineering, 25(1):47–57, 2018.973

Yoshihiro Kanno. Data-driven computing in elasticity via kernel regression. Theoretical and Applied Me-974

chanics Letters, 8(6):361–365, 2018a.975

Yoshihiro Kanno. Simple heuristic for data-driven computational elasticity with material data involving976

noise and outliers: a local robust regression approach. Japan Journal of Industrial and Applied Mathematics,977

35(3):1085–1101, 2018b.978

Yoshihiro Kanno. A kernel method for learning constitutive relation in data-driven computational elastic-979

ity. Japan Journal of Industrial and Applied Mathematics, 38(1):39–77, 2021.980

Konstantinos Karapiperis, Michael Ortiz, and Jos´

e E Andrade. Data-driven nonlocal mechanics: Discover-981

ing the internal length scales of materials. Computer Methods in Applied Mechanics and Engineering, 386:982

114039, 2021.983

Keisuke Kawano, Satoshi Koide, and Takuro Kutsuna. Learning wasserstein isometric embedding for point984

clouds. In 2020 International Conference on 3D Vision (3DV), pages 473–482. IEEE, 2020.985

Youngkyu Kim, Youngsoo Choi, David Widemann, and Tarek Zohdi. A fast and accurate physics-informed986

neural network reduced order model with shallow masked autoencoder. Journal of Computational Physics,987

page 110841, 2021.988

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint989

arXiv:1412.6980, 2014.990

Trenton Kirchdoerfer and Michael Ortiz. Data-driven computational mechanics. Computer Methods in991

Applied Mechanics and Engineering, 304:81–101, 2016.992

Trenton Kirchdoerfer and Michael Ortiz. Data driven computing with noisy material data sets. Computer993

Methods in Applied Mechanics and Engineering, 326:622–641, 2017.994

Trenton Kirchdoerfer and Michael Ortiz. Data-driven computing in dynamics. International Journal for995

Numerical Methods in Engineering, 113(11):1697–1710, 2018.996

Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, and Frank Hutter. Fast bayesian optimization997

of machine learning hyperparameters on large datasets. In Artiﬁcial intelligence and statistics, pages 528–998

536. PMLR, 2017.999

Nicolaas H. Kuiper. On C1-isometric imbeddings. I. Indagationes Mathematicae (Proceedings), 58:545–556,1000

1955.1001

Liang Lan, Zhuang Wang, Shandian Zhe, Wei Cheng, Jun Wang, and Kai Zhang. Scaling up kernel svm on1002

limited resources: A low-rank linearization approach. IEEE Transactions on Neural Networks and Learning1003

Systems, 30(2):369–378, 2019.1004

Kookjin Lee and Kevin T Carlberg. Model reduction of dynamical systems on nonlinear manifolds using1005

deep convolutional autoencoders. Journal of Computational Physics, 404:108973, 2020.1006

xl Bahador Bahmani, WaiChing Sun

Adrien Leygue, Michel Coret, Julien R´

ethor´

e, Laurent Stainier, and Erwan Verron. Data-based derivation1007

of material response. Computer Methods in Applied Mechanics and Engineering, 331:184–196, 2018.1008

Tong Lin and Hongbin Zha. Riemannian manifold learning. IEEE Transactions on Pattern Analysis and1009

Machine Intelligence, 30(5):796–809, 2008.1010

Yang Liu, WaiChing Sun, and Jacob Fish. Determining material parameters for critical state plasticity1011

models based on multilevel extended digital database. Journal of Applied Mechanics, 83(1), 2016.1012

Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Deep learning for universal linear embeddings of1013

nonlinear dynamics. Nature communications, 9(1):1–10, 2018.1014

He Lyu, Ningyu Sha, Shuyang Qin, Ming Yan, Yuying Xie, and Rongrong Wang. Manifold denoising by1015

nonlinear robust principal component analysis. Advances in neural information processing systems, 32, 2019.1016

Li Ma, Melba M Crawford, and Jinwen Tian. Local manifold learning-based k-nearest-neighbor for hy-1017

perspectral image classiﬁcation. IEEE Transactions on Geoscience and Remote Sensing, 48(11):4099–4109,1018

2010.1019

Karel Matouˇ

s, Marc GD Geers, Varvara G Kouznetsova, and Andrew Gillman. A review of predictive1020

nonlinear theories for multiscale modeling of heterogeneous materials. Journal of Computational Physics,1021

330:192–220, 2017.1022

Gal Mishne, Eric Chi, and Ronald Coifman. Co-manifold learning with missing data. In International1023

Conference on Machine Learning, pages 4605–4614. PMLR, 2019.1024

John Nash. C1isometric imbeddings. Annals of mathematics, pages 383–396, 1954.1025

Lu Trong Khiem Nguyen and Marc-Andr´

e Keip. A data-driven approach to nonlinear elasticity. Computers1026

& Structures, 194:97–115, 2018.1027

Lu Trong Khiem Nguyen, Matthias Rambausek, and Marc-Andr´

e Keip. Variational framework for1028

distance-minimizing method in data-driven computational mechanics. Computer Methods in Applied Me-1029

chanics and Engineering, 365:112898, 2020.1030

Samuel E Otto and Clarence W Rowley. Linearly recurrent autoencoder networks for learning dynamics.1031

SIAM Journal on Applied Dynamical Systems, 18(1):558–593, 2019.1032

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen,1033

Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance1034

deep learning library. Advances in neural information processing systems, 32:8026–8037, 2019.1035

Philipp Probst, Marvin N Wright, and Anne-Laure Boulesteix. Hyperparameters and tuning strategies for1036

random forest. Wiley Interdisciplinary Reviews: data mining and knowledge discovery, 9(3):e1301, 2019.1037

Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein. On the expressive1038

power of deep neural networks. In international conference on machine learning, pages 2847–2854. PMLR,1039

2017.1040

Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep1041

learning framework for solving forward and inverse problems involving nonlinear partial differential1042

equations. Journal of Computational Physics, 378:686–707, 2019.1043

Patrick J Roache. Code veriﬁcation by the method of manufactured solutions. J. Fluids Eng., 124(1):4–10,1044

2002.1045

Sam T Roweis and Lawrence K Saul. Nonlinear dimensionality reduction by locally linear embedding.1046

science, 290(5500):2323–2326, 2000.1047

Masashi Sugiyama. Dimensionality reduction of multimodal labeled data by local ﬁsher discriminant1048

analysis. Journal of machine learning research, 8(5), 2007.1049

WaiChing Sun and Alejandro Mota. A multiscale overlapped coupling formulation for large-deformation1050

strain localization. Computational Mechanics, 54(3):803–820, 2014.1051

WaiChing Sun, Zhijun Cai, and Jinhyun Choo. Mixed arlequin method for multiscale poromechanics prob-1052

lems. International Journal for Numerical Methods in Engineering, 111(7):624–659, 2017.1053

Joshua B Tenenbaum, Vin De Silva, and John C Langford. A global geometric framework for nonlinear1054

dimensionality reduction. science, 290(5500):2319–2323, 2000.1055

Stephen Timoshenko. History of strength of materials: with a brief account of the history of theory of elasticity and1056

theory of structures. Courier Corporation, 1983.1057

Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing1058

robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine1059

learning, pages 1096–1103, 2008.1060

Manifold embedding data-driven mechanics xli

Kun Wang and WaiChing Sun. A semi-implicit discrete-continuum coupling method for porous media1061

based on the effective stress principle at ﬁnite strain. Computer Methods in Applied Mechanics and Engi-1062

neering, 304:546–583, 2016.1063

Kun Wang and WaiChing Sun. A multiscale multi-permeability poroplasticity model linked by recursive1064

homogenizations and deep learning. Computer Methods in Applied Mechanics and Engineering, 334:337–1065

380, 2018.1066

Kun Wang, WaiChing Sun, Simon Salager, SeonHong Na, and Ghonwa Khaddour. Identifying material pa-1067

rameters for a micro-polar plasticity model via x-ray micro-computed tomographic (ct) images: lessons1068

learned from the curve-ﬁtting exercises. International Journal for Multiscale Computational Engineering, 141069

(4), 2016.1070

Hassler Whitney. Differentiable manifolds. Annals of Mathematics, pages 645–680, 1936.1071

Hassler Whitney. The self-intersections of a smooth n-manifold in 2n-space. Annals of Mathematics, pages1072

220–246, 1944.1073

Francis Williams, Zan Gojcic, Sameh Khamis, Denis Zorin, Joan Bruna, Sanja Fidler, and Or Litany. Neural1074

ﬁelds as learnable kernels for 3d reconstruction. arXiv preprint arXiv:2111.13674, 2021.1075

Chang Xiao, Peilin Zhong, and Changxi Zheng. Bourgan: Generative networks with metric embeddings.1076

arXiv preprint arXiv:1805.07674, 2018.1077

Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient1078

surgery for multi-task learning. arXiv preprint arXiv:2001.06782, 2020.1079

Zhenyue Zhang and Hongyuan Zha. Principal manifolds and nonlinear dimensionality reduction via1080

tangent space alignment. SIAM journal on scientiﬁc computing, 26(1):313–338, 2004.1081

A Review on Data-Driven Constitutive Laws for Solids

Preprint

Full-text available

May 2024

This review article highlights state-of-the-art data-driven techniques to discover, encode, surrogate, or emulate constitutive laws that describe the path-independent and path-dependent response of solids. Our objective is to provide an organized taxonomy to a large spectrum of methodologies developed in the past decades and to discuss the benefits and drawbacks of the various techniques for interpreting and forecasting mechanics behavior across different scales. Distinguishing between machine-learning-based and model-free methods, we further categorize approaches based on their interpretability and on their learning process/type of required data, while discussing the key problems of generalization and trustworthiness. We attempt to provide a road map of how these can be reconciled in a data-availability-aware context. We also touch upon relevant aspects such as data sampling techniques, design of experiments, verification, and validation.

Data-driven micromorphic mechanics for materials with strain localization

Article

Full-text available

Aug 2024
COMPUT METHOD APPL M

This paper explores the role of generalized continuum mechanics, and the feasibility of model-free data-driven computing approaches thereof, in solids undergoing failure by strain localization. Specifically, we set forth a methodology for capturing material instabilities using data-driven mechanics without prior information regarding the failure mode. We show numerically that, in problems involving strain localization, the standard data-driven framework for Cauchy/Boltzmann continua fails to capture the length scale of the material, as expected. We address this shortcoming by formulating a generalized data-driven framework for micromorphic continua that effectively captures both stiffness and length-scale information, as encoded in the material data, in a model-free manner. These properties are exhibited systematically in a one-dimensional softening bar problem and further verified through selected plane-strain problems.

GPM-PeNN: A generalized plasticity model-based data-driven constitutive modeling framework using physics-encoded neural network

Article

Mar 2025
COMPUT METHOD APPL M

The deep finite element method: A deep learning framework integrating the physics-informed neural networks with the finite element method

Article

Mar 2025
COMPUT METHOD APPL M

Graph Neural Network Unveils the Spatiotemporal Evolution of Structural Defects in Sheared Granular Materials

Article

Dec 2024
INT J PLASTICITY

A Review on Data-Driven Constitutive Laws for Solids

Article

Nov 2024

Data‐Driven Modeling in Geomechanics

Chapter

Oct 2024

Konstantinos Karapiperis

Model-free chemomechanical interfaces: History-dependent damage under transient mass diffusion

Article

Nov 2024
COMPUT METHOD APPL M

Shujuan Hou

This paper presents a data-driven framework based on distance functional for chemo-mechanical cohesive interfaces to capture transient diffusion and resulting interfacial damage evolution. The framework eliminates reliance on complex constitutive models and phenomenological assumptions, especially avoiding the coupling tangent terms with a given material database. The interfacial chemical potential and its jumps are used to expand the phase space for describing the chemical states of interfaces. Subsequently, a novel chemo-mechanical distance norm, including traction-separation pair, interfacial chemical potential-concentration pair and corresponding chemical potential jump-flux pair, is presented. Momentum conservation law for interfacial tractions and mass conservation law for interfacial concentration are enforced via Lagrange multipliers. For tracking the history-dependent state of the interface damage, an internal variable, termed interface integrity, is introduced to condition the interfacial database and manage the subsets mapping strategies, following physically motivated evolution constraints. Numerical examples are conducted to investigate the efficiency and capability of the present framework. A monotonic loading test validates a good numerical convergence relative to the number of data points. Subsequent cyclic loading simulations, compared with the reference solutions, show that the algorithm is well suitable to predict the history-dependent interface damage evolution under complex loading paths. Finally, the chemo-mechanically coupled examples capture phenomena such as interfacial swelling and interface degradation induced by transient diffusion and stress-driven diffusion. The current work provides a promising tool for understanding chemo-mechanical behaviors of interfaces within heterogeneous composites.

Physics‐constrained symbolic model discovery for polyconvex incompressible hyperelastic materials

Article

Full-text available

May 2024

We present a machine learning framework capable of consistently inferring mathematical expressions of hyperelastic energy functionals for incompressible materials from sparse experimental data and physical laws. To achieve this goal, we propose a polyconvex neural additive model (PNAM) that enables us to express the hyperelastic model in a learnable feature space while enforcing polyconvexity. An upshot of this feature space obtained via the PNAM is that (1) it is spanned by a set of univariate basis functions that can be re‐parametrized with a more complex mathematical form, and (2) the resultant elasticity model is guaranteed to fulfill the polyconvexity, which ensures that the acoustic tensor remains elliptic for any deformation. To further improve the interpretability, we use genetic programming to convert each univariate basis into a compact mathematical expression. The resultant multi‐variable mathematical models obtained from this proposed framework are not only more interpretable but are also proven to fulfill physical laws. By controlling the compactness of the learned symbolic form, the machine learning‐generated mathematical model also requires fewer arithmetic operations than its deep neural network counterparts during deployment. This latter attribute is crucial for scaling large‐scale simulations where the constitutive responses of every integration point must be updated within each incremental time step. We compare our proposed model discovery framework against other state‐of‐the‐art alternatives to assess the robustness and efficiency of the training algorithms and examine the trade‐off between interpretability, accuracy, and precision of the learned symbolic hyperelastic models obtained from different approaches. Our numerical results suggest that our approach extrapolates well outside the training data regime due to the precise incorporation of physics‐based knowledge.

Predictive Multiscale Paradigm for Computational Design Certification

Chapter

Jan 2024

Data-Driven nonlocal mechanics: Discovering the internal length scales of materials

Article

Full-text available

Dec 2021
COMPUT METHOD APPL M

Nonlocal effects permeate most microstructured materials, including granular media, metals and foams. The quest for predictive nonlocal mechanical theories with well-defined internal length scales has been ongoing for more than a century since the seminal work of the Cosserats. We present here a novel framework for the nonlocal analysis of material behavior, which bypasses the need to define any internal length scale. This is achieved by extending the Data-Driven paradigm in mechanics, originally introduced for simple continua, into generalized continua. The problem is formulated directly on a material data set, comprised of higher-order kinematics and their conjugate kinetics, which are identified from experiments or inferred from lower scale computations. The case of a micropolar continuum is used as a vehicle to introduce the framework, which may also be adapted to strain-gradient and micromorphic media. Two applications are presented: a micropolar elastic plate with a hole, which is used to demonstrate the convergence properties of the method, and the shear banding problem of a triaxially compressed sample of quartz sand, which is used to demonstrate the applicability of the method in the case of complex history-dependent material behavior.

Deep autoencoders for physics-constrained data-driven nonlinear materials modeling

Article

Full-text available

Nov 2021
COMPUT METHOD APPL M

Physics-constrained data-driven computing is an emerging computational paradigm that allows simulation of complex materials directly based on material database and bypass the classical constitutive model construction. However, it remains difficult to deal with high-dimensional applications and extrapolative generalization. This paper introduces deep learning techniques under the data-driven framework to address these fundamental issues in nonlinear materials modeling. To this end, an autoencoder neural network architecture is introduced to learn the underlying low-dimensional representation (embedding) of the given material database. The offline trained autoencoder and the discovered embedding space are then incorporated in the online data-driven computation such that the search of optimal material state from database can be performed on a low-dimensional space, aiming to enhance the robustness and predictability with projected material data. To ensure numerical stability and representative constitutive manifold, a convexity-preserving interpolation scheme tailored to the proposed autoencoder-based data-driven solver is proposed for constructing the material state. In this study, the applicability of the proposed approach is demonstrated by modeling nonlinear biological tissues. A parametric study on data noise, data size and sparsity, training initialization, and model architectures, is also conducted to examine the robustness and convergence property of the proposed approach.

Efficient Data Structures for Model-free Data-Driven Computational Mechanics

Article

Full-text available

Aug 2021
COMPUT METHOD APPL M

The data-driven computing paradigm initially introduced by Kirchdoerfer & Ortiz (2016) enables finite element computations in solid mechanics to be performed directly from material data sets, without an explicit material model. From a computational effort point of view, the most challenging task is the projection of admissible states at material points onto their closest states in the material data set. In this study, we compare and develop several possible data structures for solving the nearest-neighbor problem. We show that approximate nearest-neighbor (ANN) algorithms can accelerate material data searches by several orders of magnitude relative to exact searching algorithms. The approximations are suggested by—and adapted to—the structure of the data-driven iterative solver and result in no significant loss of solution accuracy. We assess the performance of the ANN algorithm with respect to material data set size with the aid of a 3D elasticity test case. We show that computations on a single processor with up to one billion material data points are feasible within a few seconds execution time with a speed up of more than 106 with respect to exact k-d trees.

A kd-tree-accelerated hybrid data-driven/model-based approach for poroelasticity problems with multi-fidelity multi-physics data

Article

Full-text available

Jul 2021
COMPUT METHOD APPL M

We present a hybrid model/model-free data-driven approach to solve poroelasticity problems. Extending the data-driven modeling framework originated from \citet{kirchdoerfer2016data}, we introduce one model-free and two hybrid model-based/data-driven formulations capable of simulating the coupled diffusion-deformation of fluid-infiltrating porous media with different amounts of available data. To improve the efficiency of the model-free data search, we introduce a distance-minimized algorithm accelerated by a k-dimensional tree search. To handle the different fidelities of the solid elasticity and fluid hydraulic constitutive responses, we introduce a hybridized model in which either the solid and the fluid solver can switch from a model-based to a model-free approach depending on the availability and the properties of the data. Numerical experiments are designed to verify the implementation and compare the performance of the proposed model to other alternatives.

An offline multi-scale unsaturated poromechanics model enabled by self-designed/self-improved neural networks

Article

Full-text available

Jan 2021
INT J NUMER ANAL MET

Supervised machine learning via artificial neural networks (ANN) has gained significant popularity for many geomechanics applications that involves multi-phase flow and poromechanics. For unsaturated poromechanics problems, the multi-physics nature and the complexity of the hydraulic laws make it difficult to design the optimal setup, architecture, and hyper-parameters of the deep neural networks. This paper presents a meta-modeling approach that utilizes deep reinforcement learning (DRL) to automatically discover optimal neural network settings that maximize a pre-defined performance metric for the machine learning constitutive laws. This meta-modeling framework is cast as a Markov Decision Process (MDP) with well-defined states (subsets of states representing the proposed neural network (NN) settings), actions, and rewards. Following the selection rules, the artificial intelligence (AI) agent, represented in DRL via NN, self-learns from taking a sequence of actions and receiving feedback signals (rewards) within the selection environment. By utilizing the Monte Carlo Tree Search (MCTS) to update the policy/value networks, the AI agent replaces the human modeler to handle the otherwise time-consuming trial-and-error process that leads to the optimized choices of setup from a high-dimensional parametric space. This approach is applied to generate two key constitutive laws for the unsaturated poromechanics problems: (1) the path-dependent retention curve with distinctive wetting and drying paths. (2) The flow in the micropores, governed by an anisotropic permeability tensor. Numerical experiments have shown that the resultant ML-generated material models can be integrated into a finite element (FE) solver to solve initial-boundary-value problems as replacements of the hand-craft constitutive laws. Keywords Deep reinforcement learning, neural network settings, unsaturated porous media, retention curve, anisotropic permeability.

A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder

Article

Nov 2021
J COMPUT PHYS

Traditional linear subspace reduced order models (LS-ROMs) are able to accelerate physical simulations in which the intrinsic solution space falls into a subspace with a small dimension, i.e., the solution space has a small Kolmogorov n-width. However, for physical phenomena not of this type, e.g., any advection-dominated flow phenomena such as in traffic flow, atmospheric flows, and air flow over vehicles, a low-dimensional linear subspace poorly approximates the solution. To address cases such as these, we have developed a fast and accurate physics-informed neural network ROM, namely nonlinear manifold ROM (NM-ROM), which can better approximate high-fidelity model solutions with a smaller latent space dimension than the LS-ROMs. Our method takes advantage of the existing numerical methods that are used to solve the corresponding full order models. The efficiency is achieved by developing a hyper-reduction technique in the context of the NM-ROM. Numerical results show that neural networks can learn a more efficient latent space representation on advection-dominated data from 1D and 2D Burgers' equations. A speedup of up to 2.6 for 1D Burgers' and a speedup of 11.7 for 2D Burgers' equations are achieved with an appropriate treatment of the nonlinear terms through a hyper-reduction technique. Finally, a posteriori error bounds for the NM-ROMs are derived that take account of the hyper-reduced operators.

Manifold Denoising by Nonlinear Robust Principal Component Analysis

Article

Dec 2019
Adv Neural Inform Process Syst

This paper extends robust principal component analysis (RPCA) to nonlinear manifolds. Suppose that the observed data matrix is the sum of a sparse component and a component drawn from some low dimensional manifold. Is it possible to separate them by using similar ideas as RPCA? Is there any benefit in treating the manifold as a whole as opposed to treating each local region independently? We answer these two questions affirmatively by proposing and analyzing an optimization framework that separates the sparse component from the manifold under noisy data. Theoretical error bounds are provided when the tangent spaces of the manifold satisfy certain incoherence conditions. We also provide a near optimal choice of the tuning parameters for the proposed optimization formulation with the help of a new curvature estimation method. The efficacy of our method is demonstrated on both synthetic and real datasets.

DNN2: A hyper-parameter reinforcement learning game for self-design of neural network based elasto-plastic constitutive descriptions

Article

Jun 2021
COMPUT STRUCT

This contribution presents a meta-modeling framework that employs artificial intelligence to design a neural network that replicates the path-dependent constitutive responses of composite materials sampled by a numerical testing procedure of Representative Volume Elements (RVE). A Deep Reinforcement Learning (DRL) combinatorics game is invented to automatically search for the optimal set of hyper-parameters from a decision tree. Besides the typical hyper-parameters for ANN training, such as the network topology, the size and composition of the considered training data are incorporated as additional hyper-parameters to help investigate the amount of data necessary for training and validation. The proposed meta modeling framework is able to identify hyper-parameter configurations with a weighted trade-off between prediction accuracy and computational cost. The capabilities and limitations of the introduced framework are shown and discussed via several numerical examples. Moreover, the possibility of transferring the gained knowledge of hyper-parameters among different RVE is explored in numerical experiments.

Learning Wasserstein Isometric Embedding for Point Clouds

Conference Paper

Nov 2020

Model-free data-driven computational mechanics enhanced by tensor voting

Article

Jan 2021
COMPUT METHOD APPL M

The data-driven computing paradigm initially introduced by Kirchdoerfer and Ortiz (2016) is extended by incorporating locally linear tangent spaces into the data set. These tangent spaces are constructed by means of the tensor voting method introduced by Mordohai and Medioni (2010) which improves the learning of the underlying structure of a data set. Tensor voting is an instance-based machine learning technique which accumulates votes from the nearest neighbors to build up second-order tensors encoding tangents and normals to the underlying data structure. The here proposed second-order data-driven paradigm is a plug-in method for distance-minimizing as well as entropy-maximizing data-driven schemes. Like its predecessor (Kirchdoerfer and Ortiz, 2016), the resulting method aims to minimize a suitably defined free energy over phase space subject to compatibility and equilibrium constraints. The method’s implementation is straightforward and numerically efficient since the data structure analysis is performed in an offline step. Selected numerical examples are presented that establish the higher-order convergence properties of the data-driven solvers enhanced by tensor voting for ideal and noisy data sets.