Content uploaded by Omey Manyar
Author content
All content in this area was uploaded by Omey Manyar on Jan 10, 2022
Content may be subject to copyright.
A DIGITAL TWIN FOR AUTOMATED LAYUP OF
PREPREG COMPOSITE SHEETS
Yi-Wei Chen1, Rex Jomy Joseph1, Alec Kanyuck1, Shahwaz Khan1, Rishi K. Malhan1, Omey M. Manyar1, Zachary McNulty1, Bohan
Wang2, Jernej Barbiˇ
c2, and Satyandra K. Gupta1
1Center for Advanced Manufacturing, University of Southern California, Los Angeles, CA, USA.
2Department of Computer Science, University of Southern California, Los Angeles, CA, USA.
ABSTRACT
The composite sheet layup process involves stacking several lay-
ers of a viscoelastic prepreg sheet and curing the laminate to
manufacture the component. Demands for automating functional
tasks in the composite manufacturing processes have dramati-
cally increased in the past decade. A simulation system rep-
resenting a digital twin of the composite sheet can aid in the
development of such an autonomous system for prepreg sheet
layup. While Finite Element Analysis (FEA) is a popular ap-
proach for simulating flexible materials, material properties need
to be encoded to produce high-fidelity mechanical simulations.
We present a methodology to predict material parameters of a
thin-shell FEA model based on real-world observations of the
deformations of the object. We utilize the model to develop a
digital twin of a composite sheet. The method is tested on vis-
coelastic composite prepreg sheets and fabric materials such as
cotton cloth, felt and canvas. We discuss the implementation and
development of a high-speed FEA simulator based on the Veg-
aFEM library. By using our method to identify sheet material
parameters, the sheet simulation system is able to predict sheet
behavior within 5 cm of average error and have proven its capa-
bility for 10 fps real-time sheet simulation.
1 Introduction
Composites manufacturing methods form an integral part of the
aerospace sector. Recently the aerospace industry has gravitated
towards an increase in the use of composites. The use of com-
posites is projected to increase with a CAGR of about 10% year
on year [24]. This growth in the utilization of composites has
stirred a demand for automating functional tasks in the compos-
ite manufacturing processes. Out of the myriad of manufacturing
methods, pre-impregnated with resin (prepreg) composite sheet
layup method has been of key interest for the aerospace indus-
try. Prepreg sheets are advantageous due to their controlled vol-
ume fraction, simple and inexpensive tooling, and ease of han-
dling. These materials can be customized in terms of the types
of fibers, weave type, and number of plies, allowing for diversi-
fication of component production and a broad range of applica-
tions [12,26]. Although the multi-component material systems
delivered by prepreg sheets carry improved functionality, their
manufacturing method is still susceptible to defects and qual-
ity issues. These defects can be classified as air gaps, wrinkles,
bridging, etc. Currently these layups are predominantly executed
by skilled operators who place each ply (prepreg sheet) manually
on the tooling and apply localized pressure using their hands or
custom hand tools.
The high factor of human involvement in the layup pro-
cesses introduces a potential variability and error in each sub-
sequent layup. With an increase in demand for composite
parts, it becomes crucial to increase production rates and im-
prove quality to maintain process capability measures. This can
be achieved through automation the composite manufacturing
methods. Presently automated techniques such as Automated
Fiber Placement (AFP) and Automated Tape Layup (ATL) are
employed primarily for tooling which does not involve complex
features. Automating a prepreg layup process introduces chal-
lenges that can only be addressed if the prepreg sheet behav-
ior can be predicted accurately beforehand. For automated sheet
layup, rectifying a defect in-process can be arduous as prepregs
once draped are difficult to separate. Consequently, the separa-
tion can result in additional defects such as bridging, fiber mis-
alignment, and impairment of previously laid up base sheets. As
a result, executing a layup perfectly at once becomes a pivotal
criterion for an automated prepreg layup process. This necessi-
tates a need for a precise and low-latency simulation system that
can accurately predict prepreg sheet behavior during the auto-
mated layup process. Such a simulation system acts as a digital
twin of the prepreg, and can be utilized to plan automation tasks.
In previous work accomplished at the Center for Advanced
Manufacturing (CAM) at the University of Southern Califor-
nia (USC), Viterbi School of Engineering, the use of robots
1
for performing automated prepreg sheet layup has been demon-
strated [18–22]. Such a highly autonomous robotic cell needs es-
timation of the sheet parameters in-situ for optimal and efficient
planning. Along with accuracy, a high data transfer rate plays a
critical role. In this research, the aim is to study the identification
of different material parameters of a prepreg sheet, namely bend-
ing stiffness, tensile stiffness, shear stiffness, damping stiffness,
and surface density. These material parameters can help develop
a FEM simulation model that can estimate and accurately simu-
late prepreg sheet behavior.
For developing the thin-sheet FEA model, we used the Ve-
gaFEM library [30]. In this paper present a modeling technique
to accurately simulate prepreg sheets under fixed constraints 1.
An experimental methodology of recording sheet characteristics
that can aid in the determination of key material parameters is
presented as well. This recorded data is used to train a model
that can evaluate the parameters with an acceptable accuracy us-
ing the Vega FEM library. The evaluated parameters are subse-
quently used to construct a digital twin represented by a force,
damping and mass matrix with the ability to emulate prepreg
sheet behavior under external fixed constraints. The study then
focuses on model evaluation and testing for different conditions.
A detailed comparison of the parameter model predictions and
experimental data are presented as well. We test our methodol-
ogy on other materials such as cotton cloth, felt and canvas. Ad-
ditionally, we introduce a real-time sheet tracking system with
depth image sensing to track the sheet for in-process validation.
FIGURE 1: Left: the simulated prepreg under external forces and con-
straints. Right: the current robotic cell with two Kuka iiwa R7 robots
and one Kuka iiwa R14 robot.
The proposed in-situ simulation system can aid in the de-
velopment of an autonomous human-robot collaborative cell for
prepreg sheet layup. In order to test the modelling concept, we
developed a robotic cell setup and the sheet simulation as de-
picted in Fig. 1. Such a simulation system can particularly assist
1This paper is based on the work presented by the authors at the 2021 ASME
Manufacturing Science and Engineering Conference [7].
in the development of a grasp planning system that can aid in the
handling of the prepreg sheets by robots. The potential applica-
tions of this simulation system are also discussed in detail.
2 Related Work
Mechanical simulation of composite prepreg sheets is exten-
sively used in context of predicting the draping of prepreg over
a mold. The sheet is represented by a mesh and fiber alignment,
shear, and bending of the sheet can be predicted. A simple and
computationally fast approach is to use kinematic simulations
which only consider the mold geometry [2,34]. More accurate
models use elasticity theory to compute the strains within the
fabric [1]. Advanced finite element analysis (FEA) based models
are used to capture the deformation mechanics of the cloth [8].
In [17], a robot was used to place the flexible material on a dou-
bly curved guiding tool. They used FEA based models to predict
the conformity. While FEA models are more accurate, they are
often computationally expensive. In our work, we utilize a vari-
ant of FEA developed in the computer graphics community, pro-
viding a good trade-off between simulation speed, stability and
accuracy [32,33]. As a result, our method is not just reasonably
accurate, but also runs at interactive rates which permits us to
perform more design iterations.
A survey work done in [25] presents an overview of all the
techniques and models used to simulate fabrics. An overview of
recent advancements in automated composite draping was pre-
sented in [11]. Progressive drape model which are a hybrid
between kinematic and FEA simulations have also been pro-
posed [13,29]. Such methods improve the accuracy compared
to purely kinematics model but also reduce the computational
expense compared to FEA. Other commonly used models are
particle systems [4,5,14,27] which nonetheless suffer from an
accuracy loss due to their inherent particle discretization.
The parameters of the model need to be tuned to match real-
world thin sheet observations. For cloth, this can be done us-
ing Kawabata plots [16]. Pre-preg composite sheets are stur-
dier than cloth, however, necessitating precise displacement and
force measuring equipment, rendering Kawabata plots less appli-
cable. Instead we develop an approach which uses FEA simula-
tion and optical motion tracking to tune the material parameters
for a given sheet, without explicitly measuring any internal elas-
tic forces.
3 Simulation Model
We use thin-shell FEA to simulate the behavior of viscoelastic
prepreg materials. Our thin-shell mathematical simulation model
was described in computer graphics references [32,33], and we
summarize it in this chapter for completeness. We note that these
models were designed as a tradeoff between computational effi-
ciency and accuracy, and have not been previously applied to
real-world structures such as composite pre-preg. We stress that
accuracy alone is not sufficient for a successful material opti-
mization system, as the speed by which design iterations can be
explored also greatly affects one’s ability to obtain satisfactory
results. Our system runs interactively at 10 FPS, which enables
rapid exploration of the material parameter optimization space.
We incorporate layup domain knowledge, realistic constraint-
handling, and real-time tracking of sheet deformation, and op-
timize the material properties to match real-world prepreg obser-
vations.
The prepreg is represented as a triangulated mesh with x,y,z
displacements of the vertices treated as simulation degrees of
freedom of the model. The internal forces as a result of bending,
shear, and stretching of the mesh govern the behavior of the ma-
terial. Therefore, these internal forces need to be computed based
on laws of elasticity for mechanical simulations. In our approach
we also compute the bending and tensile-shear force Jacobians
to improve the computational efficiency of simulations.
3.1 Tensile and Shear Forces
Our algorithm takes the triangle elements of the mesh as an in-
put. We parameterize the 3D surface of the material using 2D
parameter space represented by parameters uand v. Correspond-
ing 3D weft and warp direction vectors Uand Vof the mesh can
be computed in terms of these parameters. These vectors need
not be orthonormal to each other after deformation. Consider
the undeformed and deformed states of a triangle element in the
mesh to illustrate the concept shown in Fig. 2. The 3D vertices of
the triangle P
a,P
b,P
care computed as a function of ua,va,ub,vb,
and uc,vc.
FIGURE 2: Undeformed(left) and Deformed(right) states of a triangle
in the mesh representing the sheet. The warp and weft vectors Vand U
are used to compute the tensile and shear strains.
The weft and warp vectors are represented by weighted sums
of the three parametric vertices of the triangle. We can formu-
late a linear system of six equations: ∑
i
ruiui=1, ∑
i
ruivi=
0, ∑
i
rui =0, ∑
i
rviui=0, ∑
i
rvivi=1, and ∑
i
rvi =0, where
i∈ {a,b,c}. The weights rui and rvi can be precomputed us-
ing the equations: rua =d−1(vb−vc),rva =d−1(uc−ub),rub =
d−1(vc−va),rvb =d−1(ua−uc),ruc =d−1(va−vb), and rvc =
d−1(ub−ua), where d=ua(vb−vc) + ub(vc−va) + uc(va−vb).
The system of six linear equations is solved to obtain the vectors
Uand Vgiven by the equation (1). Viscosity of the material is
given by the evolution rates or rate change of these vectors given
by the equation (2). The vectors are then used to compute the
Green-Lagrange strain tensor which consists of shear and tensile
strains. Rate of change of these strains are then derived. Equa-
tions (3,4,5, and 6) gives the representations.
U=∑
i∈{a,b,c}
ruiP
iV=∑
i∈{a,b,c}
rviP
i(1)
U0=∑
i∈{a,b,c}
ruiP0
iV0=∑
i∈{a,b,c}
rviP0
i(2)
εuu =1
2(UTU−1)ε0
uu =1
2(UTU0)(3)
εvv =1
2(VTV−1)ε0
uu =1
2(VTV0)(4)
εuv =1
2(UTV−VTU)(5)
ε0
uv =1
2(UTV0+VTU0)(6)
Deriving the weft, warp, and shear components of total elas-
tic energy of the triangle with respect to vertex position gives us
the force applied at the jth vertex of the triangle given by the
equation (7).
Fj=−|d|
2(σuu(ru j U) + σvv(rv j V) + σuv(ru j V+rv j U)) (7)
Stress tensor provides the values of the stresses σij used in
the equation (7). The relationship between stress and strain ten-
sor is given by σ=Eε+E0ε0where Eand E’ are the elastic and
viscosity stiffness matrices of the material and σand εare the
3D stress and strain vector. We then compute the force Jacobian
which is necessary for implementation and efficiency of numer-
ical techniques. The Jacobian for ith and jt h vertex where both
i,j∈ {a,b,c}is computed using the equation (8). If viscosity is
also considered, then another contribution given by equation (9)
has to be considered as well.
∂Fj
∂P
i=−|d|
2 ∑
m,n∈{uu,vv,uv}
∂ σm
∂ εn ∂ ε T
m
∂P
i
∂ εn
∂Pj!
+∑
m,n∈{uu,vv,uv}
σm ∂
∂P
i
∂ ε T
m
∂Pj!! (8)
∂Fj
∂P0
i=−|d|
2 ∑
m,n∈{uu,vv,uv}
∂ σm
∂ ε 0
n ∂ ε T
m
∂P
i
∂ εn
∂Pj!! (9)
The stiffness component governs how stress-strain relation-
ship affects the forces acting on the vertices of the triangle. The
new position of the vertices are then found by taking into account
internal and external forces into account during the simulation.
The force acting due to bending stress is also superimposed with
the tensile and shear forces to improve simulation accuracy. We
will now discuss the method to compute the bending force.
3.2 Bending Forces
The bending force is computed as a function of the hinge angle
between two adjacent triangles in the mesh; we adopt the math-
ematical model of [32]. Fig. 3shows the two normals n1and n2
corresponding to each triangle and to the most three neighbor-
ing triangles in the mesh. The angle between these normals or
the bending angle is the hinge angle θ. Consider the total bend-
ing energy Eb=∑
i
ψ(θi)as a function of θsummed over all
possible hinges iof the mesh. The function ψis an application
specific function of the bending angle θ. We can obtain the bend-
ing force by differentiating the energy with respect to the vertex
position xas F(x) = −∑
i
∇ψand the hessian can be obtained
as H(x) = ∑
i
ψ0Hess(θi) + ψ00∇θiT∇θi, where Hess(θi)is the
second-order derivative of θiwith respect to x. In this work, the
function ψ(θ)is given by the equation
ψ(θ) = k(2tan(θ
2)−2tan(¯
θ
2))2,θ∈(−π,π),(10)
where kis a constant dependent on material properties and ¯
θis
the angle at the rest configuration. Details on how to compute
the gradient and the hessian of the bending energy analytically
are given in [32]. At each timestep, we update the bending forces
and the Jacobian of the forces based on the vertex positions.
FIGURE 3: (Left) Two adjacent triangles in the mesh and the bending
angle θbetween them. (Right) Three neighboring triangles for a triangle
under consideration are shown.
4 Estimating Sheet Parameters
4.1 Overview
In this section, we discuss the methodology of estimating ma-
terial parameters for composite prepregs, cotton cloth, felt and
canvas. The experimentation was accomplished in three stages:
data acquisition, data and parameter training/testing, and param-
eter optimization. Data acquisition was conducted within a phys-
ical environment through a guided manipulation of the various
material sheets and collecting point cloud data of the position
of each sheet as it moved. The training data optimization was
conducted in a purely computational space, repetitively running
the acquired data through an optimizing simulation sequence to
characterize optimal parameter outputs. The outputs were then
run through the simulation testing sequence to produce the fi-
nal results. This can be seen in Fig. 4below. This method
was completed for all four material types separately. The pa-
rameter estimation approach will be discussed comprehensively
in Section-4.3.
FIGURE 4: Process Overview: The initial state of the sheet is defined
as the sheet configuration under initial boundary conditions. The releas-
ing/released sheet state is defined as the sheet behavior after releasing
one of the boundary conditions. After conducting physical experiments,
initial mesh and observed data in form of a mesh of the sheet are ob-
tained from two sheet states, respectively. The data is further fed to the
optimizer to acquire computed parameters.
4.2 Acquisition of Training and Testing Data
The data collection was conducted using a Hexagon RS5 laser
scanner and contact probe attached to the Romer Absolute Arm
(87-Series). The class 2M laser scanner is hand-operated and
generates point cloud data. All scanning equipment is shown be-
low in Fig 5. The sampling filter can be manually set to optimize
the percentage of points recorded and exposure time based on the
light in the sampling environment and color of the component be-
ing scanned to reduce noise and capture the target locations. The
settings can be seen in Table 1below.
FIGURE 5: (A) Romer Absolute Arm, (B) Laser Scanner, (C) Contact
Probe and (D) Contact probe variety
TABLE 1: Settings used on the Hexagon RS5 laser scanner
Settings Value
Max Capture Rate 752000 points/sec
Percentage 25%
Exposure Time 200 µs
Point Spacing 0.052 mm
Line Width 130 mm
Sampling Rate 51 Hz
The contact probe was calibrated and operated using a TESA
TKJ 3mm Ruby Ball Probe. The probe and laser were subject to
accuracy specifications, designated by the manufacturer in Table
2.
TABLE 2: Accuracy Specifications for Laser Scanner and Contact
Probe
Equipment Accuracy
Laser Scanner 0.028mm
Contact Probe 0.046mm
4.2.1 Sheet Preparation
To prepare each sheet for scanning, quarter-inch markers were
placed at even intervals along the sheet in a 17x17 grid, for a
total of 289 markers. The markers used were white 3M double-
sided foam tape squares, cut to the correct size. The markers
were raised from the surface, allowing for easier detection by the
scanner. One of the sample materials used was white in color
and the markers had to be colored in black to be picked up by the
scanner, otherwise the color difference was considered optimal
for scanning purposes with the settings previously mentioned.
The four locations of the clamps obscured the markers, and were
recorded separately via the contact probe for use within the sim-
ulation as a fixed point.
FIGURE 6: Sheets were prepared by attaching 289 quarter-inch mark-
ers to each sheet
4.2.2 Observed Data Generation
The data was collected during two trials for each materials tested.
Each edge of a sheet was grasped using four vertically-fixed
clamps, allowing the sheet to rest suspended between them, as
shown in Fig 7. The clamping locations were chosen at differ-
ing and unique distances from adjacent corners. The collection
period for each trial featured a five-stage setup, with each stage
introducing a new modification to sheet positioning by moving
the clamp with respect to the room, while retaining the clamping
position on the sheet. The translational movement of the clamp
was chosen arbitrarily, but all movements had a change in dis-
tance of no more than 350mm.
FIGURE 7: Clamps fixed the sheet configuration at four points. This is
defined as the initial state.
The first stage required placing the sheet under the initial
boundary conditions of the first stage, resulting in the sheet sus-
pended in a relaxed, horizontal position. The next three stages
involved isolated movement of only one grasping location to a
new position, with a general increase in the z-axis, coinciding
with x- and y-axis movement toward the center of the sheet. The
last stage for every trial was a release of the clamp, allowing the
material to settle into a hanging position. Within each stage, each
clamp changed location only once and was allowed to settle into
position, until no visible movement could be detected. At the
beginning of each stage, after clamp relocation had occurred, the
new clamp locations were collected via the contact probe and the
sheet was scanned via the laser scanner. This procedure can be
observed in Fig 8.
The point cloud data collected from each stage was pro-
cessed using PC-DMIS and exported as an XYZ file for post-
processing. From each stage, each sheet contained thousands of
data points, with each marker averaging 300 points. The simu-
lator requires only one point from each of the markers, so each
XYZ file was processed through Blender software using an orig-
inal Python script to isolate the center point of each marker and
export them as 3D points, relative to the world coordinate sys-
tem of the scanner base. A comparative image of the same sheet
is shown above in Fig. 9, demonstrating the reduction from the
point cloud marker clusters to single points.
On successful construction of the initial mesh, the bound-
ary conditions of the sheet are changed by releasing one of the
clamps. Note that the remaining clamps should not be moved to
maintain consistency in the entire process. The sheet state af-
ter changing the boundary conditions is defined as released state.
Fig. 8shows the difference between the initial state and released
state.
FIGURE 8: (A) Labeled sequence of stages depicting sheet position
movements during one trial
Since no mesh is required to generate from the released state,
the point cloud data of such a state is clustered to represent each
marker. The data set acquired in this process is further defined as
the observed data. Fig. 9shows the point cloud data clustering
process.
FIGURE 9: Point cloud clusters (left) vs single point vertices (right) on
one sheet
4.3 Model Parameter Estimation
4.3.1 Sheet Simulation System
The crucial element of the parameter estimation process is the
composite sheet simulation system. Fig. 10 gives the block di-
agram for the proposed simulation system. The simulator sys-
tem utilizes the initial mesh as geometric information input and
applies the model parameters to construct the composite sheet
model.
The model parameters are categorized into two types: 1.)
Material parameters and 2.) Integrator parameters. Material pa-
rameters consists of the surface density and the internal force
parameters, which are tensile stiffness, shear stiffness, and bend-
ing stiffness. On the other hand, integrator parameters include
damping stiffness and damping mass.
Once the sheet model is constructed [3], the numerical inte-
grator applies the boundary conditions and external forces, such
as gravity, to the sheet model and solves the deformation equa-
tion [31].
After the predicted mesh is generated, the prediction error
is obtained by comparing the predicted mesh and the observed
data. The prediction error, E, is a function of model parameters,
P, initial mesh, M, and observed data, O. Algorithm (1) is used
to calculate the prediction error function.
Algorithm 1 Prediction Error
1: Let d∈O
2: Mpre ←Simulate(P,M)
3: for all ddo
4: v←FindCl osest(d,Mpre )
5: dis p ← |v−d|
6: end for
7: Error ←Average(dis p) + 0.5∗Maximum(disp)
Let dbe a data point in the observed data, O. After getting
the predicted mesh, Mpre, by applying Pand Mto the simulator,
we query each dto find the closest vertex, v, on Mpre. The pre-
diction error is then defined as the average displacement between
vand dplus half of the maximum displacement across the entire
mesh. The prediction error is used for parameter optimization,
which will be discussed in Section 4.3.3.
FIGURE 10: Simulation Process Overview: The simulator used the
material parameters and mesh information to predict sheet behavior un-
der specified conditions.
4.3.2 Parameter Boundary Selection
The model parameter identification uses a nonlinear optimizer
to compute the optimal parameters for simulating the composite
prepreg. However, the optimizer is not required to compute all
model parameters. Some of these parameters can be measured
directly.
As mentioned in section-4.3.1, model parameters comprise
of material parameters and integrator parameters. Since inte-
grator parameters are not related to model construction, we set
damping stiffness and damping mass to 1.0 and 0.0, respectively.
The surface density can be measured directly by scaling the sheet
and dividing it by the surface area. Therefore, the remaining pa-
rameters, tensile stiffness, shear stiffness, and bending stiffness,
are the parameters that require optimization.
To ensure effective performance of the optimizer, appropri-
ate upper bound, lower bound, and initial values for the param-
eters are required. Fig. 11 shows the input output diagram for
parameter boundary selection. Tensile stiffness and shear stiff-
ness are sampled into three categories: 1.) 500, 2.) 5000, and 3.)
50,000. For bending stiffness, the parameter is sampled into 1.)
0.01, 2.) 0.001, and 3.) 0.0001.
After getting three samplers for each parameter, we shuffle
them and get 27 candidates to test for parameter feasibility. All
candidates that cause system failure are eliminated, and those
who have the smallest prediction error, or are comparable to the
smallest one, are highlighted. Candidate {5000,5000,0.001}has
the best overall performance among 27 candidates, and there-
fore, it is selected to be the initial values set for the parame-
ter optimization. Then, the initial values are used as the me-
dian for the parameter boundary. Thus, the upper boundary
for the {Tensile stiffness, Shear stiffness , Bending stiffness}is
{9000,9000,0.0011}. The lower boundary for the parameter set
is {1000,1000,0.0009}.
4.3.3 Optimization Algorithm
The optimization library used in this work is NLOPT [15], an
open-source library for nonlinear optimization. The selected al-
gorithm is ISRES, Improved Stochastic Ranking Evolution, a
global-gradient-free optimization algorithm [28]. Fig. 12 shows
the block diagram for the optimization process.
The parameter optimizer utilizes the training sets and initial
parameters as inputs, and calculates the prediction errors for each
sampler. The goal for the optimizer is to find the parameters that
minimize the sum of the errors from the training set.
{m1,...,m6} ∈ M,
{o1,...,o6} ∈ O
P
opt ←arg min
P∑k
i=1(Ei(P,mi,oi)) (11)
The optimization problem can be expressed as equation (11).
Let Mbe initial meshes in the training set, which contains mesh
data, mi.Ois the observed data set, which contains observed data
oi. Recall that the prediction error, Ei, is defined in algorithm (1).
Since 5 samples are used for model training, kis set to 5. The
optimizer tries to find the parameters that reduce both the average
displacement difference and the maximum difference between
the predicted mesh and the observed data for each training set.
This error value was set to (Average Error+ 2*Max error). Once
the error converges, the identified parameters, P
opt , are used to
simulate the composite sheet in real-time.
FIGURE 12: Parameter Optimization Process.
FIGURE 11: Input-Output Diagram for Parameter Boundary Selection.
5 Results
5.1 Experimental Specifics
Four materials are considered in this work: prepreg composite
sheets, common cotton cloth, felt, and canvas. The composite
sheet was supplied by Boeing Inc. and came as 3ft x 4ft sheets,
while all other sheets were purchased locally and were 2ft x 2ft
in dimension, shown in Fig.13. As discussed previously, the
prepreg composite sheet contains viscoelastic properties due to
the resin contents, while all other sheets contain no resin or vis-
cous content. The elastic materials all have a standard/uniform
weave. Details of the elastic materials are shown in Table. 3.
c
FIGURE 13: Sheets used for four material samples. Left: Elastic Fabric
Materials. Right: Viscoelastic Prepreg Material.
TABLE 3: Elastic Material Measurements
Measurement Cloth Canvas Felt
Side1 611 mm 613 mm 615 mm
Side 2 611 mm 613 mm 612 mm
Thickness 0.1 mm 0.65mm 1.8mm
Mass 42.1 g 168.9 g 66.9 g
Surface Density 0.1126 kg/m20.4509 kg/m20.0018 kg/m2
The composite prepreg material provided by Boeing came
with manufacturer density and thickness specifications (refer Ta-
ble. 4). The elastic materials came with no specifications and
thus required density calculations and thickness measurements,
shown below in Table. 3
TABLE 4: Viscoelastic Material Specifications
Measurement Composite Sheet
Side1 1.17 m
Side 2 0.975 m
Thickness 0.3 mm
Poisson’s ratio 0.3
Surface Density 0.3 kg/m2
The physical experiment process described previously was
repeated twice for each material type for a total of eight trials. In
the case of the composite sheet, two different sheets were used, as
it was determined the combination of experimental movements
and exposure to air may degrade the sheet, possibly providing
poor results. The elastic materials all used the same sheet twice.
Of the two trials for each material, the data collected from one
trial was used for training purposes, while the other was used for
testing purposes.
5.2 Sheet Parameter Estimation
Estimation of sheet parameters was accomplished through the
simulator, as previously discussed. The internal parameters were
considered to be in one state throughout the simulation proce-
dure, but research has shown that linear elastic woven fabrics
show a variance in many of their internal parameters due to the
anisotropic behaviors of the fabric [6]. As such, manual manipu-
lation of the parameters from the initial internal parameters was
needed to find an ideal starting range for the optimization. The
initial parameters were found experimentally and were then run
through the optimizer, providing lowest error results.
The following two tables highlight the training and testing
performance of the simulator in terms of the average error and
maximum error for the training and testing data sets. Table 5
gives the results for the training of fabric materials and Table 6
gives the results for the testing of fabric material.
TABLE 5: Fabric Material Training Data
Fabric Material Training
Material Avg Error [cm] Maximum Error[cm]
Felt 2.3 7.8
Cloth 1.1 5.5
Canvas 1.4 6.0
TABLE 6: Material Testing Data
Fabric Material Testing
Material Avg Error [cm] Maximum Error[cm]
Felt 2.6 8.2
Cloth 1.6 5.6
Canvas 1.5 6.2
Figs. 14,15, and 16 show the configuration of stage 5 for
the felt, cloth, and canvas sheets, respectively. In each figure,
the upper image is the observed position photograph, the middle
image is the mesh generated from the scanned point cloud data,
and the lower image is an image of the simulation mesh at the
end of the stage.
TABLE 7: Composite Training and Testing Data.
Composite Sheet Material
Sheet Avg Error [cm] Maximum Error [cm]
Training 1 1.6 10.2
Testing 1 1.4 13.9
Testing 2 2.2 13.6
FIGURE 14: Trial 1 Stage 5 Results of Felt Sheet: (Upper) Ob-
served Position Image, (Middle) Generated Mesh, and (Lower) Simu-
lated Mesh.
FIGURE 15: Trial 1 Stage 5 Results of Cloth Sheet: (Upper) Ob-
served Position Image, (Middle) Generated Mesh, and (Lower) Simu-
lated Mesh
The following figures depict the initial and final positions
for the composite sheet trials. Within the initial stage shown in
Fig. 17, consistency can be seen between the photo, observed
data, and simulation data.
Within the final stage, there was consistent error across all
material types, however this error was higher than all other stages
indicating that this position was the most difficult for the simula-
FIGURE 16: Trial 1 Stage 5 Results of Canvas Sheet: (Upper) Ob-
served Position Image, (Middle) Generated Mesh, and (Lower) Simu-
lated Mesh
FIGURE 17: Trial 1 Stage 1 Results of Composite Sheet: (Upper) Ob-
served Position Image, (Middle) Generated Mesh, and (Lower) Simu-
lated Mesh.
tor to emulate. Fig. 18 depicts this position and displays results
from the observed data and simulated data.
The error from the training data is shown in Table.7. The
optimal parameters were a bending stiffness of 4.77682e7 N/m2
and a shear stiffness of 2942.93 N/m. The optimization pro-
cess began with initial values based on the data sheet supplied by
Boeing below in Table. 8.
FIGURE 18: Trial 1 Stage 5 Results of Composite Sheet: (Upper) Ob-
served Position Image, (Middle) Generated Mesh, and (Lower) Simu-
lated Mesh.
TABLE 8: Initial Optimizer Parameters
Parameter Initial Value
Sheet thickness 0.0003 [m]
Poisson’s ratio 0.3
Sheet density 0.3 [kg/m2]
Shear Stiffness 3.3e2 [N/m]
Bending Stiffness 1.1e8 [N/m2]
FIGURE 19: Real-time sheet simulation result: The first row shows the predicted mesh from the simulator. The second row shows the actual behavior
of the composite sheet.
5.3 Sheet Simulation
The results obtained from the material parameter estimation are
then utilized to develop a simulation system that can predict the
behavior of the material under varied constraints. This section fo-
cuses on one of the potential applications of such a high-fidelity
simulator. In Composite Prepreg sheet layups, as discussed ear-
lier, it is important for the sheet to be held at appropriate loca-
tions such that it does not result in any potential defects during
the layup. A sheet simulation system that can predict the ap-
propriate locations for grasping the sheet can be instrumental in
process planning. The key elements of such a simulation system
would be (1) A Sheet Simulation Model generated using the esti-
mated material parameters, and (2) A Real Time Sheet Tracking
System that can generate a mesh of the sheet in real time at high
frame rates, and that can then be used for comparison with sim-
ulated result. To demonstrate the feasibility of such a system, an
experiment was designed using the carbon fiber reinforced epoxy
sheet provided by Boeing Inc.
The methodology for material parameter estimation pro-
posed earlier was used to generate appropriate model for the Boe-
ing Composite Sheet. The simulation system was built using the
VegaFEM library [30]. Fig. 20 shows the block diagram of the
proposed real-time sheet tracking and simulation process.
The dimensions of the composite sheet used in this exper-
iment were 4ft x 3ft, which required a multi-camera system to
track the entire sheet. The real-time sheet tracking system pro-
posed in this study consists of three Realsense D415 sensors. The
entire sheet is captured by fusing the RGB-D feed from all the
three cameras. Color-based filtering techniques are employed to
achieve filtering of the prepreg sheet from rest of the scene. Re-
sampling is performed on resulting points to obtain a uniform
distribution of the filtered points. The face normals are then
recomputed using these points. Surface reconstruction is per-
formed in scale space by implementing Advancing Front Surface
Reconstruction [9]. A scale-space describes the point set at a dy-
namic scale, and this additional dimension allows us to control
the degree of smoothness required for the reconstruction [10].
After post-processing, a surface mesh with around 6,000 trian-
gles was obtained.
FIGURE 20: Real-time sheet simulation process.
To test the system performance, we constrained the com-
posite sheet in four different states. Initially the sheet is con-
strained at four grasping locations. A human and two 7 DOF
robotic manipulators KUKA iiwa R7, are used to apply the fixed
constraints. The Real Time Sheet Tracking System captures the
mesh and grasping locations for the corresponding state. In the
next step, one of the grasping locations is released. The sheet is
allowed to settle to its minimum energy state and the real-time
sheet tracking system captures the sheet in the respective state.
This exercise is repeated for three other states by changing the
grasping locations and capturing the sheet behavior. This data
is then compared to the predictions generated by the simulation
system. Fig. 19 contrasts the four states in which the compos-
ite sheet was suspended and the corresponding simulator predic-
tions.
The proposed simulation and real-time sheet tracking system
can play a pivotal role in predicting optimal grasping locations
for the draping of composite sheet. These grasping locations can
then be used to deploy a human robot collaborative cell where
the robots can aid the human in the draping process by holding
the sheet appropriately [23]. Furthermore, the sheet tracking data
can act as a rectifying input for the grasping location in case of
sub-optimal simulator predictions. Such a conjunctive system
can aid in streamlining the prepreg layup process and achieve an
overall higher degree of automation.
6 Conclusion
In this work, we presented a methodology to predict material pa-
rameters to generate an energy-based model of various materials.
We showcased how these models aid in simulating the material
behavior. In addition, we discussed the implementation and de-
velopment of a high-speed thin-shell simulator based on the Ve-
gaFEM library. The procedure followed for data collection for
model training and testing is highlighted as well. The inputs to
parameter model are elaborated along with the boundary condi-
tions pertaining to fixed constraints and external forces. Valida-
tion of the developed model for various test cases is presented
and discussed in depth.
This work has provided detailed insights into the simulation
of multi-material composite material-based systems, particularly
prepreg sheets along with other non-viscoelastic materials. Em-
phasis has been laid on predicting behavior of prepreg compos-
ite sheets under different types of constraints and on simulating
how the sheet will behave under these conditions. The predic-
tion capabilities of our system opens new avenues in the domain
of automated composite layup processes, particularly in plan-
ning for the development of completely autonomous and human-
robot collaborative cells. This system can be employed to plan
and execute a defect-free layup using robotic assistants, by using
the simulator to generate optimal locations to grasp the prepreg
sheets during a layup process. In the future, we aim to explore the
capability of employing this simulator in trajectory and motion
planning of collaborative robots mainly used for manipulation of
prepreg sheets, thus enabling the automation of composite layup
tasks.
Acknowledgment: This work is supported in part by National
Science Foundation Grant #1925084. Opinions expressed are
those of the authors and do not necessarily reflect opinions of the
sponsors.
References
[1] M. Aono. A wrinkle propagation model for cloth. In CG
International’90, pages 95–115. Springer, 1990.
[2] M. Aono, D. E. Breen, and M. J. Wozny. Fitting a woven-
cloth model to a curved surface: mapping algorithms.
Computer-Aided Design, 26(4):278–292, 1994.
[3] D. Baraff and A. P. Witkin. Large steps in cloth simulation.
In Proc. of ACM SIGGRAPH 98, pages 43–54, 1998.
[4] D. E. Breen, D. H. House, and P. H. Getto. A physically-
based particle model of woven cloth. The Visual Computer,
8(5-6):264–277, 1992.
[5] D. E. Breen, D. H. House, and M. J. Wozny. A particle-
based model for simulating the draping behavior of woven
cloth. Textile Research Journal, 64(11):663–685, 1994.
[6] F. D. C. C. Barile, C. Casavola. A simple approach to non-
linear tensile stiffness for accurate cloth simulation. Com-
posites Part B: Engineering, 2019.
[7] Y. Chen, R. Joseph, A. Kanyuck, S. Khan, R. K. Malhan,
O. M. Manyar, Z. McNulty, B. Wang, J. Barbic, and S. K.
Gupta. A digital twin for automated layup of prepreg com-
posite sheets. Manufacturing Sciences and Engineering
Conference. Virtual, June 2021.
[8] J. R. Collier, B. J. Collier, G. O’Toole, and S. Sargand.
Drape prediction by means of finite-element analysis. Jour-
nal of the Textile Institute, 82(1):96–107, 1991.
[9] F. Da and D. Cohen-Steiner. Advancing Front Surface Re-
construction. CGAL Editorial Board, 5.0.2 edition, 2020.
[10] J. Digne, J. M. Morel, C. M. Souzani, and C. Lartigue.
Scale space meshing of raw data point sets. Computer
Graphics Forum, 30(6):1630–1642, 2011.
[11] M. Elkington, C. Ward, and A. Sarkytbayev. Automated
composite draping: a review. In SAMPE 2017. SAMPE
North America, May 2017.
[12] J. Fleischer, R. Teti, G. Lanza, P. Mativenga, H.-C.
M¨
ohring, and A. Caggiano. Composite materials parts
manufacturing. CIRP Annals - Manufacturing Technology,
2018.
[13] S. G. Hancock and K. D. Potter. The use of kinematic drape
modelling to inform the hand lay-up of complex composite
components using woven reinforcements. Composites Part
A: Applied Science and Manufacturing, 37(3):413 – 422,
2006.
[14] D. House and D. Breen. Cloth modeling and animation.
CRC Press, 2000.
[15] S. G. Johnson. The NLopt nonlinear-optimization package.
Available at http://github.com/stevengj/nlopt.
[16] S. Kawabatra. The standardization and analysis of hand
evaluation. The Textile Machinery Society Japan, 1980.
[17] C. Krogh, J. A. Glud, and J. Jakobsen. Modeling the robotic
manipulation of woven carbon fiber prepreg plies onto dou-
ble curved molds: A path-dependent problem. Journal of
Composite Materials, page 0021998318822722, 2018.
[18] R. K. Malhan, A. M. Kabir, B. Shah, T. Centea, and S. K.
Gupta. Automated prepreg sheet placement using collab-
orative robotics. North America Society for the Advance-
ment of Material and Process Engineering (SAMPE) Long
beach conference. CA, USA. 2018.
[19] R. K. Malhan, A. M. Kabir, B. Shah, T. Centea, and S. K.
Gupta. Hybrid cells for multi-layer prepreg composite sheet
layup. IEEE International Conference on Automation Sci-
ence and Engineering (CASE). Munich, Germany., 2018.
[20] R. K. Malhan, A. M. Kabir, B. Shah, T. Centea, and S. K.
Gupta. Determining feasible robot placements in robotic
cells for composite prepreg sheet layup. ASMEs 14th Man-
ufacturing Science and Engineering Conference. Erie, PA,
USA, 2019.
[21] R. K. Malhan, A. M. Kabir, B. C. Shah, and S. K. Gupta.
Identifying feasible workpiece placement with respect to
redundant manipulator for complex manufacturing tasks.
IEEE International Conference on Robotics and Automa-
tion (ICRA). Montreal, Canada., 2019.
[22] R. K. Malhan, A. V. Shembekar, A. M. Kabir, P. M. Bhatt,
B. Shah, S. Zanio, S. Nutt, and S. K. Gupta. Automated
planning for robotic layup of composite prepreg. Robotics
and Computer-Integrated Manufacturing, 67:102020.
[23] O. M. Manyar, J. A. Desai, N. Deogaonkar, R. J. Joseph,
R. K. Malhan, Z. McNulty, B. Wang, J. Barbiˇ
c, and S. K.
Gupta. A simulation-based grasp planner for enabling
robotic grasping during composite sheet layup. IEEE Inter-
national Conference on Robotics and Automation (ICRA).
Xi’an, China., 2021.
[24] S. Mazumdar, D. Pichler, M. Benevento, W. Seneviratine,
R. Liang, and E. Witten. American composites manufactur-
ing association 2020 state of the industry report. Available
at http://compositesmanufacturingmagazine.com/2020/01/
2020-state-of-the-industry-report/ (2020).
[25] H. N. Ng and R. L. Grimsdale. Computer graphics tech-
niques for modeling cloth. IEEE Computer Graphics and
Applications, 16(5):28–41, 1996.
[26] N. M.-T. P. Volino and F. Faure. Mechanical comparison of
new composite materials for aerospace applications. ACM
Transactions on Graphics, 2009.
[27] X. Provot et al. Deformation constraints in a mass-spring
model to describe rigid cloth behaviour. In Graphics in-
terface, pages 147–147. Canadian Information Processing
Society, 1995.
[28] T. P. Runarsson and X. Yao. Search biases in con-
strained evolutionary optimization. IEEE Trans. on Sys-
tems, Man, and Cybernetics Part C: Applications and Re-
views, 35(2):233–243, 2005.
[29] S. B. Sharma and M. P. F. Sutcliffe. Draping of woven fab-
rics: Progressive drape model. Plastics, Rubber and Com-
posites, 32(2):57–64, 2003.
[30] F. S. Sin, D. Schroeder, and J. Barbiˇ
c. Vega FEM library.
Available at http://www.jernejbarbic.com/vega (2012).
[31] F. S. Sin, D. Schroeder, and J. Barbiˇ
c. Vega: Nonlinear
FEM deformable object simulator. Computer Graphics,
32(1):38–50, 2013.
[32] R. Tamstorf and E. Grinspun. Discrete bending forces and
their jacobians. Graphical models, 75(6):362–370, 2013.
[33] P. Volino, N. Magnenat-Thalmann, and F. Faure. A sim-
ple approach to nonlinear tensile stiffness for accurate cloth
simulation. 2009.
[34] J. Wang, R. Paton, and J. Page. The draping of woven fabric
preforms and prepregs for production of polymer compos-
ite components. Composites Part A: Applied Science and
Manufacturing, 30(6):757–765, 1999.
List of Figures
1 Left: the simulated prepreg under external forces
and constraints. Right: the current robotic cell
with two Kuka iiwa R7 robots and one Kuka iiwa
R14 robot. ..................... 2
2 Undeformed(left) and Deformed(right) states of
a triangle in the mesh representing the sheet. The
warp and weft vectors Vand Uare used to com-
pute the tensile and shear strains. ......... 3
3 (Left) Two adjacent triangles in the mesh and the
bending angle θbetween them. (Right) Three
neighboring triangles for a triangle under consid-
eration are shown. ................. 4
4 Process Overview: The initial state of the sheet
is defined as the sheet configuration under ini-
tial boundary conditions. The releasing/released
sheet state is defined as the sheet behavior after
releasing one of the boundary conditions. Af-
ter conducting physical experiments, initial mesh
and observed data in form of a mesh of the sheet
are obtained from two sheet states, respectively.
The data is further fed to the optimizer to acquire
computed parameters. ............... 4
5 (A) Romer Absolute Arm, (B) Laser Scanner,
(C) Contact Probe and (D) Contact probe variety 5
6 Sheets were prepared by attaching 289 quarter-
inch markers to each sheet ............ 5
7 Clamps fixed the sheet configuration at four
points. This is defined as the initial state. ..... 5
8 (A) Labeled sequence of stages depicting sheet
position movements during one trial ....... 6
9 Point cloud clusters (left) vs single point vertices
(right) on one sheet ................ 6
10 Simulation Process Overview: The simulator
used the material parameters and mesh infor-
mation to predict sheet behavior under specified
conditions. ..................... 7
12 Parameter Optimization Process. ......... 7
11 Input-Output Diagram for Parameter Boundary
Selection. ..................... 8
13 Sheets used for four material samples. Left:
Elastic Fabric Materials. Right: Viscoelastic
Prepreg Material. ................. 8
14 Trial 1 Stage 5 Results of Felt Sheet: (Upper)
Observed Position Image, (Middle) Generated
Mesh, and (Lower) Simulated Mesh. ....... 9
15 Trial 1 Stage 5 Results of Cloth Sheet: (Upper)
Observed Position Image, (Middle) Generated
Mesh, and (Lower) Simulated Mesh ....... 9
16 Trial 1 Stage 5 Results of Canvas Sheet: (Up-
per) Observed Position Image, (Middle) Gener-
ated Mesh, and (Lower) Simulated Mesh ..... 10
17 Trial 1 Stage 1 Results of Composite Sheet: (Up-
per) Observed Position Image, (Middle) Gener-
ated Mesh, and (Lower) Simulated Mesh. . . . . 10
18 Trial 1 Stage 5 Results of Composite Sheet: (Up-
per) Observed Position Image, (Middle) Gener-
ated Mesh, and (Lower) Simulated Mesh. . . . . 10
19 Real-time sheet simulation result: The first row
shows the predicted mesh from the simulator.
The second row shows the actual behavior of the
composite sheet. .................. 11
20 Real-time sheet simulation process. ....... 11
List of Tables
1 Settings used on the Hexagon RS5 laser scanner . 5
2 Accuracy Specifications for Laser Scanner and
Contact Probe ................... 5
3 Elastic Material Measurements .......... 8
4 Viscoelastic Material Specifications ....... 8
5 Fabric Material Training Data .......... 9
6 Material Testing Data ............... 9
7 Composite Training and Testing Data. ...... 9
8 Initial Optimizer Parameters ........... 10
