A Digital Twin for Automated Layup of Prepreg Composite Sheets

Abstract

The composite sheet layup process involves stacking several layers of a viscoelastic prepreg sheet and curing the laminate to manufacture the component. Demands for automating functional tasks in the composite manufacturing processes have dramatically increased in the past decade. A simulation system representing a digital twin of the composite sheet can aid in the development of such an autonomous system for prepreg sheet layup. While Finite Element Analysis (FEA) is a popular approach for simulating flexible materials, material properties need to be encoded to produce high-fidelity mechanical simulations. We present a methodology to predict material parameters of a thin-shell FEA model based on real-world observations of the deformations of the object. We utilize the model to develop a digital twin of a composite sheet. The method is tested on viscoelastic composite prepreg sheets and fabric materials such as cotton cloth, felt and canvas. We discuss the implementation and development of a high-speed FEA simulator based on the VegaFEM library [29]. By using our method to identify sheet material parameters, the sheet simulation system is able to predict sheet behavior within 5 cm of average error and have proven its capability for 10 fps real-time sheet simulation.

Full text

Note: This is a Draft version of the paper accepted at ASME’s Manufacturing Science and Engineering Conference, June
2021, Virtual. Readers are encouraged to get an official version from ASME.
In order to cite this paper please use the following template:
Yi-Wei Chen, Rex Jomy Joseph, Alec Kanyuck, Shahwaz Khan, Rishi K. Malhan, Omey M. Manyar, Zachary McNulty,
Bohan Wang, Jernej Barbiˇc, Satyandra K. Gupta. A Digital Twin for Automated Layup of Prepreg Composite Sheets.
ASME Manufacturing Science and Engineering Conference, Virtual, June 2021
1

A Digital Twin for automated layup of Prepreg Composite
Sheets
Yi-Wei Chen1, Rex Jomy Joseph1, Alec Kanyuck1, Shahwaz Khan1, Rishi K. Malhan1, Omey M.
Manyar1, Zachary McNulty1,
Bohan Wang2, Jernej Barbiˇc2, and Satyandra K. Gupta1
1Center for Advanced Manufacturing, University of Southern California, Los Angeles, CA, USA.
2Department of Computer Science, University of Southern California, Los Angeles, CA, USA.
Abstract
The composite sheet layup process involves stacking several layers of a viscoelastic prepreg sheet and curing the laminate
to manufacture the component. Demands for automating functional tasks in the composite manufacturing processes have
dramatically increased in the past decade. A simulation system representing a digital twin of the composite sheet can
aid in the development of such an autonomous system for prepreg sheet layup. While Finite Element Analysis (FEA)
is a popular approach for simulating flexible materials, material properties need to be encoded to produce high-fidelity
mechanical simulations. We present a methodology to predict material parameters of a thin-shell FEA model based on
real-world observations of the deformations of the object. We utilize the model to develop a digital twin of a composite
sheet. The method is tested on viscoelastic composite prepreg sheets and fabric materials such as cotton cloth, felt and
canvas. We discuss the implementation and development of a high-speed FEA simulator based on the VegaFEM library [29].
By using our method to identify sheet material parameters, the sheet simulation system is able to predict sheet behavior
within 5 cm of average error and have proven its capability for 10 fps real-time sheet simulation.
1 Introduction
Composites manufacturing methods form an integral part of the aerospace sector. Recently the aerospace industry has
gravitated towards an increase in the use of composites. The use of composites is projected to increase with a CAGR of about
10% year on year [23]. This growth in the utilization of composites has stirred a demand for automating functional tasks in
the composite manufacturing processes. Out of the myriad of manufacturing methods, pre-impregnated with resin (prepreg)
composite sheet layup method has been of key interest for the aerospace industry. Prepreg sheets are advantageous due to
their controlled volume fraction, simple and inexpensive tooling, and ease of handling. These materials can be customized
in terms of the types of fibers, weave type, and number of plies, allowing for diversification of component production and
a broad range of applications [11,25]. Although the multi-component material systems delivered by prepreg sheets carry
improved functionality, their manufacturing method is still susceptible to defects and quality issues. These defects can be
classified as air gaps, wrinkles, bridging, etc. Currently these layups are predominantly executed by skilled operators who
place each ply (prepreg sheet) manually on the tooling and apply localized pressure using their hands or custom hand tools.
The high factor of human involvement in the layup processes introduces a potential variability and error in each subsequent
layup. With an increase in demand for composite parts, it becomes crucial to increase production rates and improve quality
to maintain process capability measures. This can be achieved through automation the composite manufacturing methods.
Presently automated techniques such as Automated Fiber Placement (AFP) and Automated Tape Layup (ATL) are employed
primarily for tooling which does not involve complex features. Automating a prepreg layup process introduces challenges
that can only be addressed if the prepreg sheet behavior can be predicted accurately beforehand. For automated sheet
layup, rectifying a defect in-process can be arduous as prepregs once draped are difficult to separate. Consequently, the
separation can result in additional defects such as bridging, fiber misalignment, and impairment of previously laid up base
sheets. As a result, executing a layup perfectly at once becomes a pivotal criterion for an automated prepreg layup process.
This necessitates a need for a precise and low-latency simulation system that can accurately predict prepreg sheet behavior
during the automated layup process. Such a simulation system acts as a digital twin of the prepreg, and can be utilized to
plan automation tasks.
In previous work accomplished at the Center for Advanced Manufacturing (CAM) at the University of Southern Cal-
ifornia (USC), Viterbi School of Engineering, the use of robots for performing automated prepreg sheet layup has been
demonstrated [17–21]. Such a highly autonomous robotic cell needs estimation of the sheet parameters in-situ for optimal
and efficient planning. Along with accuracy, a high data transfer rate plays a critical role. In this research, the aim is to
study the identification of different material parameters of a prepreg sheet, namely bending stiffness, tensile stiffness, shear
stiffness, damping stiffness, and surface density. These material parameters can help develop a FEM simulation model that
can estimate and accurately simulate prepreg sheet behavior.
2

For developing the thin-sheet FEA model, we used the VegaFEM library [29]. We present a modeling technique to
accurately simulate prepreg sheets under fixed constraints. An experimental methodology of recording sheet characteristics
that can aid in the determination of key material parameters is presented as well. This recorded data is used to train a model
that can evaluate the parameters with an acceptable accuracy using the Vega FEM library. The evaluated parameters are
subsequently used to construct a digital twin represented by a force, damping and mass matrix with the ability to emulate
prepreg sheet behavior under external fixed constraints. The study then focuses on model evaluation and testing for different
conditions. A detailed comparison of the parameter model predictions and experimental data are presented as well. We
test our methodology on other materials such as cotton cloth, felt and canvas. Additionally, we introduce a real-time sheet
tracking system with depth image sensing to track the sheet for in-process validation.
Figure 1: Left: the simulated prepreg under external forces and constraints. Right: the current robotic cell with two Kuka iiwa R7
robots and one Kuka iiwa R14 robot.
The proposed in-situ simulation system can aid in the development of an autonomous human-robot collaborative cell for
prepreg sheet layup. In order to test the modelling concept, we developed a robotic cell setup and the sheet simulation as
depicted in Fig. 1. Such a simulation system can particularly assist in the development of a grasp planning system that can
aid in the handling of the prepreg sheets by robots. The potential applications of this simulation system are also discussed
in detail.
2 Related Work
Mechanical simulation of composite prepreg sheets is extensively used in context of predicting the draping of prepreg over a
mold. The sheet is represented by a mesh and fiber alignment, shear, and bending of the sheet can be predicted. A simple
and computationally fast approach is to use kinematic simulations which only consider the mold geometry [2,33]. More
accurate models use elasticity theory to compute the strains within the fabric [1]. Advanced finite element analysis (FEA)
based models are used to capture the deformation mechanics of the cloth [7]. In [16], a robot was used to place the flexible
material on a doubly curved guiding tool. They used FEA based models to predict the conformity. While FEA models are
more accurate, they are often computationally expensive. In our work, we utilize a variant of FEA developed in the computer
graphics community, providing a good trade-off between simulation speed, stability and accuracy [31,32]. As a result, our
method is not just reasonably accurate, but also runs at interactive rates which permits us to perform more design iterations.
A survey work done in [24] presents an overview of all the techniques and models used to simulate fabrics. An overview
of recent advancements in automated composite draping was presented in [10]. Progressive drape model which are a hybrid
between kinematic and FEA simulations have also been proposed [12,28]. Such methods improve the accuracy compared to
purely kinematics model but also reduce the computational expense compared to FEA. Other commonly used models are
particle systems [4,5,13,26] which nonetheless suffer from an accuracy loss due to their inherent particle discretization.
The parameters of the model need to be tuned to match real-world thin sheet observations. For cloth, this can be done
using Kawabata plots [15]. Pre-preg composite sheets are sturdier than cloth, however, necessitating precise displacement
and force measuring equipment, rendering Kawabata plots less applicable. Instead we develop an approach which uses FEA
simulation and optical motion tracking to tune the material parameters for a given sheet, without explicitly measuring any
internal elastic forces.
3 Simulation Model
We use thin-shell FEA to simulate the behavior of viscoelastic prepreg materials. Our thin-shell mathematical simulation
model was described in computer graphics references [31,32], and we summarize it in this chapter for completeness. We note
that these models were designed as a tradeoff between computational efficiency and accuracy, and have not been previously
3

applied to real-world structures such as composite pre-preg. We stress that accuracy alone is not sufficient for a successful
material optimization system, as the speed by which design iterations can be explored also greatly affects one’s ability
to obtain satisfactory results. Our system runs interactively at 10 FPS, which enables rapid exploration of the material
parameter optimization space. We incorporate layup domain knowledge, realistic constraint-handling, and real-time tracking
of sheet deformation, and optimize the material properties to match real-world prepreg observations.
The prepreg is represented as a triangulated mesh with x,y,z displacements of the vertices treated as simulation degrees
of freedom of the model. The internal forces as a result of bending, shear, and stretching of the mesh govern the behavior
of the material. Therefore, these internal forces need to be computed based on laws of elasticity for mechanical simulations.
In our approach we also compute the bending and tensile-shear force Jacobians to improve the computational efficiency of
simulations.
3.1 Tensile and Shear Forces
Our algorithm takes the triangle elements of the mesh as an input. We parameterize the 3D surface of the material using
2D parameter space represented by parameters uand v. Corresponding 3D weft and warp direction vectors Uand Vof the
mesh can be computed in terms of these parameters. These vectors need not be orthonormal to each other after deformation.
Consider the undeformed and deformed states of a triangle element in the mesh to illustrate the concept shown in Fig. 2.
The 3D vertices of the triangle Pa, Pb, Pcare computed as a function of ua, va,ub, vb, and uc, vc.
Figure 2: Undeformed(left) and Deformed(right) states of a triangle in the mesh representing the sheet. The warp and weft vectors V
and Uare used to compute the tensile and shear strains.
The weft and warp vectors are represented by weighted sums of the three parametric vertices of the triangle. We can
formulate a linear system of six equations: X
i
ruiui= 1, X
i
ruivi= 0, X
i
rui = 0, X
i
rviui= 0, X
i
rvivi= 1, and
X
i
rvi = 0, where i∈ {a, b, c}. The weights rui and rvi can be precomputed using the equations: rua =d−1(vb−vc),
rva =d−1(uc−ub), rub =d−1(vc−va), rvb =d−1(ua−uc), ruc =d−1(va−vb), and rvc =d−1(ub−ua), where d=
ua(vb−vc) + ub(vc−va) + uc(va−vb). The system of six linear equations is solved to obtain the vectors Uand Vgiven
by the equation (1). Viscosity of the material is given by the evolution rates or rate change of these vectors given by the
equation (2). The vectors are then used to compute the Green-Lagrange strain tensor which consists of shear and tensile
strains. Rate of change of these strains are then derived. Equations (3,4,5, and 6) gives the representations.
U=X
i∈{a,b,c}
ruiPiV=X
i∈{a,b,c}
rviPi(1)
U0=X
i∈{a,b,c}
ruiP0
iV0=X
i∈{a,b,c}
rviP0
i(2)
uu =1
2(UTU−1) 0
uu =1
2(UTU0) (3)
vv =1
2(VTV−1) 0
uu =1
2(VTV0) (4)
uv =1
2(UTV−VTU) (5)
0
uv =1
2(UTV0+VTU0) (6)
Deriving the weft, warp, and shear components of total elastic energy of the triangle with respect to vertex position gives
us the force applied at the jth vertex of the triangle given by the equation (7).
Fj=−|d|
2(σuu(ruj U) + σvv(rv j V) + σuv(ruj V+rv j U)) (7)
4

Stress tensor provides the values of the stresses σij used in the equation (7). The relationship between stress and strain
tensor is given by σ=E +E00where Eand E’ are the elastic and viscosity stiffness matrices of the material and σand 
are the 3D stress and strain vector. We then compute the force Jacobian which is necessary for implementation and efficiency
of numerical techniques. The Jacobian for ith and jth vertex where both i, j ∈ {a, b, c}is computed using the equation (8).
If viscosity is also considered, then another contribution given by equation (9) has to be considered as well.
∂Fj
∂Pi=−|d|
2 X
m,n∈{uu,vv,uv}
∂σm
∂n ∂T
m
∂Pi
∂n
∂Pj!
+X
m,n∈{uu,vv,uv}
σm ∂
∂Pi
∂T
m
∂Pj!! (8)
∂Fj
∂P 0
i=−|d|
2 X
m,n∈{uu,vv,uv}
∂σm
∂0
n ∂T
m
∂Pi
∂n
∂Pj!! (9)
The stiffness component governs how stress-strain relationship affects the forces acting on the vertices of the triangle.
The new position of the vertices are then found by taking into account internal and external forces into account during
the simulation. The force acting due to bending stress is also superimposed with the tensile and shear forces to improve
simulation accuracy. We will now discuss the method to compute the bending force.
3.2 Bending Forces
The bending force is computed as a function of the hinge angle between two adjacent triangles in the mesh; we adopt the
mathematical model of [31]. Fig. 3shows the two normals n1and n2corresponding to each triangle and to the most three
neighboring triangles in the mesh. The angle between these normals or the bending angle is the hinge angle θ. Consider the
total bending energy Eb=X
i
ψ(θi) as a function of θsummed over all possible hinges iof the mesh. The function ψis
an application specific function of the bending angle θ. We can obtain the bending force by differentiating the energy with
respect to the vertex position xas F(x) = −X
i
∇ψand the hessian can be obtained as H(x) = X
i
ψ0Hess(θi)+ ψ00∇θiT∇θi,
where Hess(θi) is the second-order derivative of θiwith respect to x. In this work, the function ψ(θ) is given by the equation
ψ(θ) = k(2tan(θ
2)−2tan(¯
θ
2))2, θ ∈(−π, π),(10)
where kis a constant dependent on material properties and ¯
θis the angle at the rest configuration. Details on how to
compute the gradient and the hessian of the bending energy analytically are given in [31]. At each timestep, we update the
bending forces and the Jacobian of the forces based on the vertex positions.
Figure 3: (Left) Two adjacent triangles in the mesh and the bending angle θbetween them. (Right) Three neighboring triangles for a
triangle under consideration are shown.
4 Estimating Sheet Parameters
4.1 Overview
In this section, we discuss the methodology of estimating material parameters for composite prepregs, cotton cloth, felt and
canvas. The experimentation was accomplished in three stages: data acquisition, data and parameter training/testing, and
parameter optimization. Data acquisition was conducted within a physical environment through a guided manipulation of
the various material sheets and collecting point cloud data of the position of each sheet as it moved. The training data
optimization was conducted in a purely computational space, repetitively running the acquired data through an optimizing
5

simulation sequence to characterize optimal parameter outputs. The outputs were then run through the simulation testing
sequence to produce the final results. This can be seen in Fig. 4below. This method was completed for all four material
types separately. The parameter estimation approach will be discussed comprehensively in Section-4.3.
Figure 4: Process Overview: The initial state of the sheet is defined as the sheet configuration under initial boundary conditions. The
releasing/released sheet state is defined as the sheet behavior after releasing one of the boundary conditions. After conducting physical
experiments, initial mesh and observed data in form of a mesh of the sheet are obtained from two sheet states, respectively. The data
is further fed to the optimizer to acquire computed parameters.
4.2 Acquisition of Training and Testing Data
The data collection was conducted using a Hexagon RS5 laser scanner and contact probe attached to the Romer Absolute
Arm (87-Series). The class 2M laser scanner is hand-operated and generates point cloud data. All scanning equipment is
shown below in Fig 5. The sampling filter can be manually set to optimize the percentage of points recorded and exposure
time based on the light in the sampling environment and color of the component being scanned to reduce noise and capture
the target locations. The settings can be seen in Table 1below.
Figure 5: (A) Romer Absolute Arm, (B) Laser Scanner, (C) Contact Probe and (D) Contact probe variety
The contact probe was calibrated and operated using a TESA TKJ 3mm Ruby Ball Probe. The probe and laser were
subject to accuracy specifications, designated by the manufacturer in Table 2.
4.2.1 Sheet Preparation
To prepare each sheet for scanning, quarter-inch markers were placed at even intervals along the sheet in a 17x17 grid,
for a total of 289 markers. The markers used were white 3M double-sided foam tape squares, cut to the correct size. The
markers were raised from the surface, allowing for easier detection by the scanner. One of the sample materials used was
white in color and the markers had to be colored in black to be picked up by the scanner, otherwise the color difference was
considered optimal for scanning purposes with the settings previously mentioned. The four locations of the clamps obscured
the markers, and were recorded separately via the contact probe for use within the simulation as a fixed point.
6

Table 1: Settings used on the Hexagon RS5 laser scanner
Settings Value
Max Capture Rate 752000 points/sec
Percentage 25%
Exposure Time 200 µs
Point Spacing 0.052 mm
Line Width 130 mm
Sampling Rate 51 Hz
Table 2: Accuracy Specifications for Laser Scanner and Contact Probe
Equipment Accuracy
Laser Scanner 0.028mm
Contact Probe 0.046mm
Figure 6: Sheets were prepared by attaching 289 quarter-inch markers to each sheet
4.2.2 Observed Data Generation
The data was collected during two trials for each materials tested. Each edge of a sheet was grasped using four vertically-
fixed clamps, allowing the sheet to rest suspended between them, as shown in Fig 7. The clamping locations were chosen at
differing and unique distances from adjacent corners. The collection period for each trial featured a five-stage setup, with
each stage introducing a new modification to sheet positioning by moving the clamp with respect to the room, while retaining
the clamping position on the sheet. The translational movement of the clamp was chosen arbitrarily, but all movements had
a change in distance of no more than 350mm.
Figure 7: Clamps fixed the sheet configuration at four points. This is defined as the initial state.
The first stage required placing the sheet under the initial boundary conditions of the first stage, resulting in the sheet
suspended in a relaxed, horizontal position. The next three stages involved isolated movement of only one grasping location
7

to a new position, with a general increase in the z-axis, coinciding with x- and y-axis movement toward the center of the
sheet. The last stage for every trial was a release of the clamp, allowing the material to settle into a hanging position. Within
each stage, each clamp changed location only once and was allowed to settle into position, until no visible movement could
be detected. At the beginning of each stage, after clamp relocation had occurred, the new clamp locations were collected via
the contact probe and the sheet was scanned via the laser scanner. This procedure can be observed in Fig 8.
The point cloud data collected from each stage was processed using PC-DMIS and exported as an XYZ file for post-
processing. From each stage, each sheet contained thousands of data points, with each marker averaging 300 points. The
simulator requires only one point from each of the markers, so each XYZ file was processed through Blender software using
an original Python script to isolate the center point of each marker and export them as 3D points, relative to the world
coordinate system of the scanner base. A comparative image of the same sheet is shown above in Fig. 9, demonstrating the
reduction from the point cloud marker clusters to single points.
On successful construction of the initial mesh, the boundary conditions of the sheet are changed by releasing one of the
clamps. Note that the remaining clamps should not be moved to maintain consistency in the entire process. The sheet state
after changing the boundary conditions is defined as released state. Fig. 8shows the difference between the initial state and
released state.
Figure 8: (A) Labeled sequence of stages depicting sheet position movements during one trial
Since no mesh is required to generate from the released state, the point cloud data of such a state is clustered to represent
each marker. The data set acquired in this process is further defined as the observed data. Fig. 9shows the point cloud data
clustering process.
Figure 9: Point cloud clusters (left) vs single point vertices (right) on one sheet
4.3 Model Parameter Estimation
4.3.1 Sheet Simulation System
The crucial element of the parameter estimation process is the composite sheet simulation system. Fig. 10 gives the block
diagram for the proposed simulation system. The simulator system utilizes the initial mesh as geometric information input
and applies the model parameters to construct the composite sheet model.
The model parameters are categorized into two types: 1.) Material parameters and 2.) Integrator parameters. Material
parameters consists of the surface density and the internal force parameters, which are tensile stiffness, shear stiffness, and
bending stiffness. On the other hand, integrator parameters include damping stiffness and damping mass.
8

Once the sheet model is constructed [3], the numerical integrator applies the boundary conditions and external forces,
such as gravity, to the sheet model and solves the deformation equation [30].
After the predicted mesh is generated, the prediction error is obtained by comparing the predicted mesh and the observed
data. The prediction error, E, is a function of model parameters, P, initial mesh, M, and observed data, O. Algorithm (1)
is used to calculate the prediction error function.
Algorithm 1 Prediction Error
1: Let d∈O
2: Mpre ←Simulate(P , M)
3: for all ddo
4: v←F indClosest(d, Mpr e)
5: disp ← |v−d|
6: end for
7: Err or ←Average(disp)+0.5∗Maximum(disp)
Let dbe a data point in the observed data, O. After getting the predicted mesh, Mpre , by applying Pand Mto the
simulator, we query each dto find the closest vertex, v, on Mpr e. The prediction error is then defined as the average
displacement between vand dplus half of the maximum displacement across the entire mesh. The prediction error is used
for parameter optimization, which will be discussed in Section 4.3.3.
Figure 10: Simulation Process Overview: The simulator used the material parameters and mesh information to predict sheet behavior
under specified conditions.
4.3.2 Parameter Boundary Selection
The model parameter identification uses a nonlinear optimizer to compute the optimal parameters for simulating the
composite prepreg. However, the optimizer is not required to compute all model parameters. Some of these parameters can
be measured directly.
As mentioned in section-4.3.1, model parameters comprise of material parameters and integrator parameters. Since
integrator parameters are not related to model construction, we set damping stiffness and damping mass to 1.0 and 0.0,
respectively. The surface density can be measured directly by scaling the sheet and dividing it by the surface area. Therefore,
the remaining parameters, tensile stiffness, shear stiffness, and bending stiffness, are the parameters that require optimization.
To ensure effective performance of the optimizer, appropriate upper bound, lower bound, and initial values for the
parameters are required. Fig. 11 shows the input output diagram for parameter boundary selection. Tensile stiffness and
shear stiffness are sampled into three categories: 1.) 500, 2.) 5000, and 3.) 50,000. For bending stiffness, the parameter
is sampled into 1.) 0.01, 2.) 0.001, and 3.) 0.0001. After getting three samplers for each parameter, we shuffle them and
get 27 candidates to test for parameter feasibility. All candidates that cause system failure are eliminated, and those who
have the smallest prediction error, or are comparable to the smallest one, are highlighted. Candidate {5000,5000,0.001}has
the best overall performance among 27 candidates, and therefore, it is selected to be the initial values set for the parameter
optimization. Then, the initial values are used as the median for the parameter boundary. Thus, the upper boundary for
the {Tensile stiffness, Shear stiffness , Bending stiffness}is {9000,9000,0.0011}. The lower boundary for the parameter set
is {1000,1000,0.0009}.
4.3.3 Optimization Algorithm
The optimization library used in this work is NLOPT [14], an open-source library for nonlinear optimization. The selected
algorithm is ISRES, Improved Stochastic Ranking Evolution, a global-gradient-free optimization algorithm [27]. Fig. 12 shows
the block diagram for the optimization process.
9

Figure 11: Input-Output Diagram for Parameter Boundary Selection.
The parameter optimizer utilizes the training sets and initial parameters as inputs, and calculates the prediction errors
for each sampler. The goal for the optimizer is to find the parameters that minimize the sum of the errors from the training
set.
{m1, . . . , m6} ∈ M,
{o1, . . . , o6} ∈ O
Popt ←arg min
PPk
i=1(Ei(P, mi, oi)) (11)
The optimization problem can be expressed as equation (11). Let Mbe initial meshes in the training set, which contains
mesh data, mi.Ois the observed data set, which contains observed data oi. Recall that the prediction error, Ei, is defined
in algorithm (1). Since 5 samples are used for model training, kis set to 5. The optimizer tries to find the parameters that
reduce both the average displacement difference and the maximum difference between the predicted mesh and the observed
data for each training set. This error value was set to (Average Error+ 2*Max error). Once the error converges, the identified
parameters, Popt, are used to simulate the composite sheet in real-time.
Figure 12: Parameter Optimization Process.
5 Results
5.1 Experimental Specifics
Four materials are considered in this work: prepreg composite sheets, common cotton cloth, felt, and canvas. The composite
sheet was supplied by Boeing Inc. and came as 3ft x 4ft sheets, while all other sheets were purchased locally and were 2ft x 2ft
in dimension, shown in Fig.13. As discussed previously, the prepreg composite sheet contains viscoelastic properties due to
the resin contents, while all other sheets contain no resin or viscous content. The elastic materials all have a standard/uniform
weave. Details of the elastic materials are shown in Table. 3.
10

c
Figure 13: Sheets used for four material samples. Left: Elastic Fabric Materials. Right: Viscoelastic Prepreg Material.
Table 3: Elastic Material Measurements
Measurement Cloth Canvas Felt
Side1 611 mm 613 mm 615 mm
Side 2 611 mm 613 mm 612 mm
Thickness 0.1 mm 0.65mm 1.8mm
Mass 42.1 g 168.9 g 66.9 g
Surface Density 0.1126 kg/m20.4509 kg/m20.0018 kg/m2
The composite prepreg material provided by Boeing came with manufacturer density and thickness specifications (refer
Table. 4). The elastic materials came with no specifications and thus required density calculations and thickness measure-
ments, shown below in Table. 3
Table 4: Viscoelastic Material Specifications
Measurement Composite Sheet
Side1 1.17 m
Side 2 0.975 m
Thickness 0.3 mm
Poisson’s ratio 0.3
Surface Density 0.3 kg/m2
The physical experiment process described previously was repeated twice for each material type for a total of eight trials.
In the case of the composite sheet, two different sheets were used, as it was determined the combination of experimental
movements and exposure to air may degrade the sheet, possibly providing poor results. The elastic materials all used the
same sheet twice. Of the two trials for each material, the data collected from one trial was used for training purposes, while
the other was used for testing purposes.
5.2 Sheet Parameter Estimation
Estimation of sheet parameters was accomplished through the simulator, as previously discussed. The internal parameters
were considered to be in one state throughout the simulation procedure, but research has shown that linear elastic woven
fabrics show a variance in many of their internal parameters due to the anisotropic behaviors of the fabric [6]. As such,
manual manipulation of the parameters from the initial internal parameters was needed to find an ideal starting range for the
optimization. The initial parameters were found experimentally and were then run through the optimizer, providing lowest
error results.
The following two tables highlight the training and testing performance of the simulator in terms of the average error
and maximum error for the training and testing data sets. Table 5gives the results for the training of fabric materials and
Table 6gives the results for the testing of fabric material.
Table 5: Fabric Material Training Data
Fabric Material Training
Material Avg Error [cm] Maximum Error[cm]
Felt 2.3 7.8
Cloth 1.1 5.5
Canvas 1.4 6.0
11

Table 6: Material Testing Data
Fabric Material Testing
Material Avg Error [cm] Maximum Error[cm]
Felt 2.6 8.2
Cloth 1.6 5.6
Canvas 1.5 6.2
Figs. 14,15, and 16 show the configuration of stage 5 for the felt, cloth, and canvas sheets, respectively. In each figure,
the upper image is the observed position photograph, the middle image is the mesh generated from the scanned point cloud
data, and the lower image is an image of the simulation mesh at the end of the stage.
Figure 14: Trial 1 Stage 5 Results of Felt Sheet: (Upper) Observed Position Image, (Middle) Generated Mesh, and (Lower) Simulated
Mesh.
Figure 15: Trial 1 Stage 5 Results of Cloth Sheet: (Upper) Observed Position Image, (Middle) Generated Mesh, and (Lower) Simulated
Mesh
The following figures depict the initial and final positions for the composite sheet trials. Within the initial stage shown
in Fig. 17, consistency can be seen between the photo, observed data, and simulation data.
Within the final stage, there was consistent error across all material types, however this error was consistently higher
than all other stages indicating that this position was the most difficult for the simulator to emulate. Fig. 20 depicts this
12

Figure 16: Trial 1 Stage 5 Results of Canvas Sheet: (Upper) Observed Position Image, (Middle) Generated Mesh, and (Lower) Simulated
Mesh
Table 7: Composite Training and Testing Data. ∗The high value of error 24.9 in this case is due to an anomaly in data collection at
one particular location (refer Fig. 19). The next highest value is 14.8 . We believe that value 14.8 is a reasonable estimate for error in
this case.
Composite Sheet Material
Sheet Avg Error [cm] Maximum Error [cm]
Training 1 1.9 14.5
Training 2 1.4 10.9
Testing 1 3.2 14.6
Testing 2 2.5 14.8(24.9)∗
Figure 17: Trial 1 Stage 1 Results of Composite Sheet: (Upper) Observed Position Image, (Middle) Generated Mesh, and (Lower)
Simulated Mesh.
position and displays results from the observed data and simulated data. Inconsistencies within the data were seen in the
fourth stage of Trial 4, shown in Fig. 18. Comparatively, Fig. 19 shows all four trials at the fourth position, and the line of
13

tension within the sheet is visible in the lower right-hand corner of each image. As can be observed, all Trials show similar
positioning, with Trials 1, 2, and 3 show similarity in sheet behavior, but Trial 4 shows an anomalous configuration of the
sheet material on the right hand side of the sheet.
Figure 18: Trial 4 Stage 4 Results of Composite Sheet: (Upper) Generated Image, (Middle) Observed Position Image, (Lower) Simulated
Mesh
Figure 19: Comparison of the 4th configuration of the composite sheet for Trials 1-4. For T4 we can see the sheet configuration is
inconsistent with that of the others for same fixed locations, hence giving a higher value of error.
The error from the training data is shown in Table.7. The optimal parameters were a bending stiffness of 7.96031e7 N/m2
and a shear stiffness of 1353.1 N/m. The optimization process began with initial values based on the data sheet supplied by
Boeing below in Table. 8.
5.3 Sheet Simulation
The results obtained from the material parameter estimation are then utilized to develop a simulation system that can predict
the behavior of the material under varied constraints. This section focuses on one of the potential applications of such a
high-fidelity simulator. In Composite Prepreg sheet layups, as discussed earlier, it is important for the sheet to be held at
appropriate locations such that it does not result in any potential defects during the layup. A sheet simulation system that
can predict the appropriate locations for grasping the sheet can be instrumental in process planning. The key elements of
such a simulation system would be (1) A Sheet Simulation Model generated using the estimated material parameters, and
14

Table 8: Initial Optimizer Parameters
Parameter Initial Value
Sheet thickness 0.0003 [m]
Poisson’s ratio 0.3
Sheet density 0.3 [kg/m2]
Shear Stiffness 3.3e2 [N/m]
Bending Stiffness 1.1e8 [N/m2]
Figure 20: Trial 1 Stage 5 Results of Composite Sheet: (Upper) Observed Position Image, (Middle) Generated Mesh, and (Lower)
Simulated Mesh.
(2) A Real Time Sheet Tracking System that can generate a mesh of the sheet in real time at high frame rates, and that
can then be used for comparison with simulated result. To demonstrate the feasibility of such a system, an experiment was
designed using the carbon fiber reinforced epoxy sheet provided by Boeing Inc. The methodology for material parameter
estimation proposed earlier was used to generate appropriate model for the Boeing Composite Sheet. The simulation system
was built using the VegaFEM library [29]. Fig. 22 shows the block diagram of the proposed real-time sheet tracking and
simulation process.
The dimensions of the composite sheet used in this experiment were 4ft x 3ft, which required a multi-camera system to
track the entire sheet. The real-time sheet tracking system proposed in this study consists of three Realsense D415 sensors.
The entire sheet is captured by fusing the RGB-D feed from all the three cameras. Color-based filtering techniques are
employed to achieve filtering of the prepreg sheet from rest of the scene. Re-sampling is performed on resulting points to
obtain a uniform distribution of the filtered points. The face normals are then recomputed using these points. Surface
reconstruction is performed in scale space by implementing Advancing Front Surface Reconstruction [8]. A scale-space
describes the point set at a dynamic scale, and this additional dimension allows us to control the degree of smoothness
required for the reconstruction [9]. After post-processing, a surface mesh with around 6,000 triangles was obtained. To test
the system performance, we constrained the composite sheet in four different states. Initially the sheet is constrained at four
grasping locations. A human and two 7 DOF robotic manipulators KUKA iiwa R7, are used to apply the fixed constraints.
The Real Time Sheet Tracking System captures the mesh and grasping locations for the corresponding state. In the next
step, one of the grasping locations is released. The sheet is allowed to settle to its minimum energy state and the real-time
sheet tracking system captures the sheet in the respective state. This exercise is repeated for three other states by changing
the grasping locations and capturing the sheet behavior. This data is then compared to the predictions generated by the
simulation system. Fig. 21 contrasts the four states in which the composite sheet was suspended and the corresponding
simulator predictions.
The proposed simulation and real-time sheet tracking system can play a pivotal role in predicting optimal grasping
locations for the draping of composite sheet. These grasping locations can then be used to deploy a human robot collaborative
cell where the robots can aid the human in the draping process by holding the sheet appropriately [22]. Furthermore, the
sheet tracking data can act as a rectifying input for the grasping location in case of sub-optimal simulator predictions. Such
a conjunctive system can aid in streamlining the prepreg layup process and achieve an overall higher degree of automation.
15

Figure 21: Real-time sheet simulation result: The first row shows the predicted mesh from the simulator. The second row shows the
actual behavior of the composite sheet.
Figure 22: Real-time sheet simulation process.
6 Conclusion
In this work, we presented a methodology to predict material parameters to generate an energy-based model of various
materials. We showcased how these models aid in simulating the material behavior. In addition, we discussed the implemen-
tation and development of a high-speed thin-shell simulator based on the VegaFEM library. The procedure followed for data
collection for model training and testing is highlighted as well. The inputs to parameter model are elaborated along with the
boundary conditions pertaining to fixed constraints and external forces. Validation of the developed model for various test
cases is presented and discussed in depth.
This work has provided detailed insights into the simulation of multi-material composite material-based systems, partic-
ularly prepreg sheets along with other non-viscoelastic materials. Emphasis has been laid on predicting behavior of prepreg
composite sheets under different types of constraints and on simulating how the sheet will behave under these conditions.
The prediction capabilities of our system opens new avenues in the domain of automated composite layup processes, par-
ticularly in planning for the development of completely autonomous and human-robot collaborative cells. This system can
be employed to plan and execute a defect-free layup using robotic assistants, by using the simulator to generate optimal
locations to grasp the prepreg sheets during a layup process. In the future, we aim to explore the capability of employing
this simulator in trajectory and motion planning of collaborative robots mainly used for manipulation of prepreg sheets, thus
enabling the automation of composite layup tasks.
Acknowledgment: This work is supported in part by National Science Foundation Grant #1925084. Opinions expressed
are those of the authors and do not necessarily reflect opinions of the sponsors.
References
[1] M. Aono. A wrinkle propagation model for cloth. In CG International’90, pages 95–115. Springer, 1990.
[2] M. Aono, D. E. Breen, and M. J. Wozny. Fitting a woven-cloth model to a curved surface: mapping algorithms.
Computer-Aided Design, 26(4):278–292, 1994.
[3] D. Baraff and A. P. Witkin. Large steps in cloth simulation. In Proc. of ACM SIGGRAPH 98, pages 43–54, 1998.
[4] D. E. Breen, D. H. House, and P. H. Getto. A physically-based particle model of woven cloth. The Visual Computer,
8(5-6):264–277, 1992.
[5] D. E. Breen, D. H. House, and M. J. Wozny. A particle-based model for simulating the draping behavior of woven cloth.
Textile Research Journal, 64(11):663–685, 1994.
16

[6] F. D. C. C. Barile, C. Casavola. A simple approach to nonlinear tensile stiffness for accurate cloth simulation. Composites
Part B: Engineering, 2019.
[7] J. R. Collier, B. J. Collier, G. O’Toole, and S. Sargand. Drape prediction by means of finite-element analysis. Journal
of the Textile Institute, 82(1):96–107, 1991.
[8] F. Da and D. Cohen-Steiner. Advancing Front Surface Reconstruction. CGAL Editorial Board, 5.0.2 edition, 2020.
[9] J. Digne, J. M. Morel, C. M. Souzani, and C. Lartigue. Scale space meshing of raw data point sets. Computer Graphics
Forum, 30(6):1630–1642, 2011.
[10] M. Elkington, C. Ward, and A. Sarkytbayev. Automated composite draping: a review. In SAMPE 2017. SAMPE North
America, May 2017.
[11] J. Fleischer, R. Teti, G. Lanza, P. Mativenga, H.-C. M¨ohring, and A. Caggiano. Composite materials parts manufactur-
ing. CIRP Annals - Manufacturing Technology, 2018.
[12] S. G. Hancock and K. D. Potter. The use of kinematic drape modelling to inform the hand lay-up of complex composite
components using woven reinforcements. Composites Part A: Applied Science and Manufacturing, 37(3):413 – 422, 2006.
[13] D. House and D. Breen. Cloth modeling and animation. CRC Press, 2000.
[14] S. G. Johnson. The NLopt nonlinear-optimization package. Available at http://github.com/stevengj/nlopt.
[15] S. Kawabatra. The standardization and analysis of hand evaluation. The Textile Machinery Society Japan, 1980.
[16] C. Krogh, J. A. Glud, and J. Jakobsen. Modeling the robotic manipulation of woven carbon fiber prepreg plies onto
double curved molds: A path-dependent problem. Journal of Composite Materials, page 0021998318822722, 2018.
[17] R. K. Malhan, A. M. Kabir, B. Shah, T. Centea, and S. K. Gupta. Automated prepreg sheet placement using collaborative
robotics. North America Society for the Advancement of Material and Process Engineering (SAMPE) Long beach
conference. CA, USA. 2018.
[18] R. K. Malhan, A. M. Kabir, B. Shah, T. Centea, and S. K. Gupta. Hybrid cells for multi-layer prepreg composite sheet
layup. IEEE International Conference on Automation Science and Engineering (CASE). Munich, Germany., 2018.
[19] R. K. Malhan, A. M. Kabir, B. Shah, T. Centea, and S. K. Gupta. Determining feasible robot placements in robotic
cells for composite prepreg sheet layup. ASMEs 14th Manufacturing Science and Engineering Conference. Erie, PA,
USA, 2019.
[20] R. K. Malhan, A. M. Kabir, B. C. Shah, and S. K. Gupta. Identifying feasible workpiece placement with respect to
redundant manipulator for complex manufacturing tasks. IEEE International Conference on Robotics and Automation
(ICRA). Montreal, Canada., 2019.
[21] R. K. Malhan, A. V. Shembekar, A. M. Kabir, P. M. Bhatt, B. Shah, S. Zanio, S. Nutt, and S. K. Gupta. Automated
planning for robotic layup of composite prepreg. Robotics and Computer-Integrated Manufacturing, 67:102020.
[22] O. M. Manyar, J. A. Desai, N. Deogaonkar, R. J. Joseph, R. K. Malhan, Z. McNulty, B. Wang, J. Barbiˇc, and S. K.
Gupta. A simulation-based grasp planner for enabling robotic grasping during composite sheet layup. IEEE International
Conference on Robotics and Automation (ICRA). Xi’an, China., 2021.
[23] S. Mazumdar, D. Pichler, M. Benevento, W. Seneviratine, R. Liang, and E. Witten. American composites manufacturing
association 2020 state of the industry report. Available at http://compositesmanufacturingmagazine.com/2020/01/
2020-state-of-the-industry-report/ (2020).
[24] H. N. Ng and R. L. Grimsdale. Computer graphics techniques for modeling cloth. IEEE Computer Graphics and
Applications, 16(5):28–41, 1996.
[25] N. M.-T. P. Volino and F. Faure. Mechanical comparison of new composite materials for aerospace applications. ACM
Transactions on Graphics, 2009.
[26] X. Provot et al. Deformation constraints in a mass-spring model to describe rigid cloth behaviour. In Graphics interface,
pages 147–147. Canadian Information Processing Society, 1995.
[27] T. P. Runarsson and X. Yao. Search biases in constrained evolutionary optimization. IEEE Trans. on Systems, Man,
and Cybernetics Part C: Applications and Reviews, 35(2):233–243, 2005.
17

[28] S. B. Sharma and M. P. F. Sutcliffe. Draping of woven fabrics: Progressive drape model. Plastics, Rubber and Composites,
32(2):57–64, 2003.
[29] F. S. Sin, D. Schroeder, and J. Barbiˇc. Vega FEM library. Available at http://www.jernejbarbic.com/vega (2012).
[30] F. S. Sin, D. Schroeder, and J. Barbiˇc. Vega: Nonlinear FEM deformable object simulator. Computer Graphics,
32(1):38–50, 2013.
[31] R. Tamstorf and E. Grinspun. Discrete bending forces and their jacobians. Graphical models, 75(6):362–370, 2013.
[32] P. Volino, N. Magnenat-Thalmann, and F. Faure. A simple approach to nonlinear tensile stiffness for accurate cloth
simulation. 2009.
[33] J. Wang, R. Paton, and J. Page. The draping of woven fabric preforms and prepregs for production of polymer composite
components. Composites Part A: Applied Science and Manufacturing, 30(6):757–765, 1999.
18