Content uploaded by Simon Stieber
Author content
All content in this area was uploaded by Simon Stieber on Dec 13, 2021
Content may be subject to copyright.
PermeabilityNets: Comparing Neural Network
Architectures on a Sequence-to-Instance Task in
CFRP Manufacturing
Simon Stieber∗, Niklas Schr¨
oter∗, Ewald Fauster †, Alexander Schiendorfer∗‡, Wolfgang Reif∗
∗Institute for Software & Systems Engineering, University of Augsburg, Augsburg, Germany
Email: {stieber, schroeter, reif}@isse.de
†Processing of Composites Group, Montanuniversit¨
at Leoben, Austria
Email: Ewald.Fauster@unileoben.ac.at
‡Digital Production Group, Technische Hochschule Ingolstadt, Germany
Email: Alexander.Schiendorfer@thi.de
Abstract—Carbon fiber reinforced polymers (CFRP) offer
highly desirable properties such as weight-specific strength and
stiffness. Liquid composite moulding (LCM) processes are promi-
nent, economically efficient, out-of-autoclave manufacturing tech-
niques and, in particular, resin transfer moulding (RTM), allows
for a high level of automation. There, fibrous preforms are
impregnated by a viscous polymer matrix in a closed mould.
Impregnation quality is of crucial importance for the final part
quality and is dominated by preform permeability. We propose
to learn a map of permeability deviations based on a sequence of
camera images acquired in flow experiments. Several ML models
are investigated for this task, among which ConvLSTM networks
achieve an accuracy of up to 96.56%, showing better performance
than the Transformer or pure CNNs. Finally, we demonstrate that
models, trained purely on simulated data, achieve qualitatively
good results on real data.
Index Terms—Sequence-to-Instance Learning, Architecture
comparison, CFRP, LCM, Industry 4.0, Digital Twin
I. RTM AND PE RM EA BI LI TY MEASUREMENTS
Carbon fiber reinforced polymers (CFRP) are superior to
other engineering materials in terms of weight-specific me-
chanical properties such as strength and stiffness. Replacing
conventional steel or aluminum with CFRP helps reduce fuel
consumption and CO2 emissions. These composite materials
are made from a polymer matrix that is reinforced with fibers
made from carbon. To produce CFRP parts industrially, liquid
composite moulding (LCM) processes are one of the most
prominent, economically efficient out-of-autoclave manufac-
turing techniques. More specifically, resin transfer molding
(RTM, [1]) is a commonly applied manufacturing process for
medium volumes (1,000s to 10,000s of parts per year): A
liquid thermoset polymer (called a resin) is injected under
pressure into a mold cavity that contains the reinforcement
material, i.e., a textile of carbon fibers (called the preform).
During injection, this results in a “flow front” that sepa-
rates impregnated material from dry material (cf. Fig. 2).
This research is funded by the Bavarian Ministry of Economic Affairs,
Regional Development and Energy in the project CosiMo.
Fig. 1. Overview of the sequence-to-instance learning task: a sequence of flow
front images is mapped to an image that contains the permeability values of
the material, i.e,. the permeability map.
The temporal progress of the flow front is predominantly
influenced by the permeability characteristics of the preform,
which describes the ability of a porous medium to transmit
fluids.In CFRP, the preform permeability is dominated by
(i) fiber volume content (FVC) and (ii) preform architecture.
Local changes of preform permeability can occur at curved
sections of the component or at locations of wall thickness
changes. In addition, they may be caused by imperfect material
properties (e.g., missing or misaligned fiber bundles) or by
manual handling of the fibrous structure The variation of
permeability can reach up to 20% [2]. Local variations in
permeability affect the temporal progress of the flow front
and can thus cause suboptimal impregnation quality or even
dry spots, deteriorating the mechanical performance. These
variations are typically unknown for an individual preform
prior to or during an industrial RTM process – a map of FVC
or permeability values would be highly desirable to diagnose
issues early on. Equipping the mold with sensors facilitates
in-situ monitoring of the injection process [3].
Aside from industrial RTM instances, permeability char-
acteristics of preform materials can be experimentally de-
termined by means of permeability characterization cells, as
shown in Fig. 2. For the study at hand, an 2-D in-plane, optical
permeability characterization cell (called a permeameter) was
1
Fig. 2. Permeameter experiments: Left: In-plane permeameter system with
an optically transparent upper mould half. Middle: Beginning of injection trial
in permeameter, a dark spot is visible between the frame. Right: the injection
is in progress, with a visible deviation of the flow front caused by the patch
with changed permeability, which is visible on both images.
used, which follows the radial flow technique combined with
optical flow front tracking. Using an optical permeameter
offers the advantage that almost the entire flow front can be
tracked in the form of planar images instead of sensory time
series at certain locations only.
Therefore, we propose to use machine learning (ML) models
that predict changes in the permeability, given an observed
sequence of flow front images during an injection process in
a permeameter, i.e., a sequence-to-instance task (cf. Fig. 1).
The training data is obtained from PAM RTM, an industrially
applied FEM simulation tool for RTM processes (see Sec-
tion II). This task constitutes a part of a larger pipeline that
eventually leads to a permeability map prediction from in-
mold sensory data. For that, an intermediary step that maps
sensors to flow front images would be necessary: e.g. using
transposed convolutions [3] or other generative models. To
sum up, our contributions are: We compare several ML models
for this sequence-to-instance task in an engineering domain.
We apply these networks, trained on simulated data only, also
on real permeameter data and demonstrate that they are able
to reliably predict permeability deviations. This approach can
be seen as a part of a digital twin of the process [4]. If the
component does not show excessive variations in permeability
and, consequently, in FVC, it can be considered acceptable.
A. Related work
The online measurement of the permeability or its deviation
has received attention in the composites processing literature.
Barkoula et al. [5] estimate global and local permeability
during the injection with pressure and flow front sensors by
calculating it with Darcy’s law [6]:
v=−1
ηK∇p, (1)
with volume-average flow velocity v, fluid viscosity η, perme-
ability tensor Kand pressure drop ∇p. Darcy’s law is central to
liquid composite molding simulation. For 2-dimensional flow
(as addressed in the work at hand), the planar, anisotropic
permeability tensor is set up as follows:
K2D,aniso =kxkxy
kxy ky(2)
Fig. 3. Left: Measures of the final setup for the injection trials. Middle:
projected kx, Right: projected ky(from simulation).
The permeability xand ydirection is shown by kxand ky,
whereas kxy describes the dependency of the flow in one main
direction on a pressure gradient in the other main direction.
Gonzales et al. [7] used CNNs to detect changes in the flow
front from pressure sensors to be able to detect changes in
permeability. They use the data of whole recorded runs, but
can only detect single, rectangular changes in permeability.
Our approach is more versatile by not only detecting fixed
features of the patch (such as length, width or center point)
but constructing the entire permeability map of the preform.
Besides online permeability estimation, other papers combine
RTM analysis and ML: Stieber et al. [3] present a learning-
based dry spot classifier, working only on simulated data.
II. DATA REG IM E
The available data either stem from FEM-based RTM-
simulations or the optical permeameters depicted in Fig. 2.
a) Simulated data: To obtain the simulated data , we
followed a setup comparable to [3]: On a 2D plate of homoge-
neous material, we randomly inserted small patches of changed
permeability and FVC into the textile of a simulated RTM
process within an automated pipeline (cf. “label” in Fig. 1).
These modified permeability maps provoke altered flow fronts
and need to be re-discovered by the ML models. As input,
the models get a sequence of flow front images, as shown in
Fig. 1. We only predict the kxpermeability (as a first feasibility
check of this approach), for kythe models would needed to be
retrained with an additional output, kxy is 0in the simulation.
A total of 10,990 runs was produced.
b) Real data from permeameter: To generate the real
data, we introduced one patch of lower permeability at the
same spot to each of six runs in the permeameter (cf. Fig. 2).
To ensure that we use a patch of sufficient size and FVC to
actually provoke a dry spot, we simulated different patch con-
figurations, Fig. 3 shows the final measures. The experiments
were run with a glass fiber woven fabric, type Hexion 1202
of Hexcel. Table I lists the most relevant properties of the
preform in both the neat and the patch region. The reinforcing
material used for the experiments in this paper shows principal
flow directions well aligned with the directions of the woven
fiber bundles, i.e.: kxy ≈0. Moreover, standard plant oil was
used as a test fluid for the experiments. It exhibits a viscosity
2
TABLE I
PRE FOR M SE TU P FOR R EA L WO RLD T RI AL S
Region # of layers FVC kx[mm2]ky[mm2]
Preform mat. 11 41.7 % 72.47e−6 17.21e−6
Patch region 16 60.7 % 1.19e−6 15.32e−6
of 65 mPas at room temperature and was colored with a red
color pigment to enhance image contrast of saturated preform
regions against unsaturated preform regions.
To use the experimental data, especially passing it through a
model trained on simulation-only data, the acquired sequence
images had to be preprocessed. We transformed the perme-
ameter images to align them with the images created by the
simulation: We removed the empty area around the textile,
removed the stiffing frame and approximated the parts of the
image that were covered by the stiffening frame. This is done
by fitting an elliptical geometry model to selected sets of
flow front data [8]. After these processing steps, the approach
is identical to the simulated data; the image sequences are
sampled over the filling percentage of the visible flow front,
while adding padding to create sequences of the same length.
We conclude the section on data by noting that creating
real-world experiments is costly in a lab or in an industrial
environment and thus limited data are available. Further,
some fixed preprocessing necessary is required to align real
images with simulated ones, e.g., due to the stabilising metal
frame. While there is no measured permeability map of the
real injection experiments, we can rely on the simulation
permeability maps that were implemented using well-defined
glass fiber woven fabrics for our evaluations in Section IV-A2.
III. APP ROAC H - MO DEL COM PARISON
Several models are suitable to address our sequence-to-
instance task. The first model we employed is a basic con-
volutional neural network (CNN) with four conv2D layers.
From a subsampling step, we get 100 single-channel images of
the time steps of the injection process. Since 2D convolutions
work over any number of channels, we use the 100 time
steps as independent channels for the first convolutional layer.
The second approach emphasizes the temporal aspect by
using a Convolutional Long Short Term Memory (ConvLSTM)
architecture [9]. In contrast to regular LSTMs, ConvLSTMs
work over sequences of two dimensional matrices, instead
of one dimensional vectors, which makes them suitable for
our task. The last proposed model relies on the transformer
mechanism [10] which works on one-dimensional embed-
dings. Hence, the input image sequence needs to be converted
into an embedding sequence. To do so, a fully convolutional
encoder creates feature vectors for the individual images and
is trained end-to-end in the sequence-to-instance pipeline. The
transformer output creates the permeability map using 2D
transposed convolutions. For further details, see the code1.
1We published the code, data and checkpoints: https://github.com/
isse-augsburg/PermeabilityNets
TABLE II
RES ULTS OV ERVI EW - MOD EL COMPARISON. TRA IN.T IM E:O NE EP OC H
Model IOU Acc. Train.
time
Inf. time Real data
perf.
CNN 0.6277 94.74 52 s 24 s ??
ConvLSTM 0.7496 96.56 27:30 m 40 s ???
Transformer 0.6373 95.08 1:45 m 27 s ?
IV. EVALUATIO N
To evaluate our models, we had to define a set of met-
rics aside from the “expert eye” that solely investigates the
output images. Defining a metric faithful to the observed
performance (cf. Figure 4) turned out to be more difficult
than expected since, e.g., pixelwise accuracy tends to focus
too much on “blurry” predictions. Exact accuracy further
proved to be uninformative due to large portions of the
permeability maps having roughly similar base permeabilities.
We therefore allowed ε-tolerances to still classify individual
pixels as correct. On normalised images (between 0 and 1), we
empirically determined εto be 0.03 by manually inspecting
prediction-label pairs. Values greater than 3% decrease the
sensibility of the metric, while values smaller than 3% are
too sensitive to create comparable results. In addition, we
employed the intersection over union (IOU). This metric
focuses on the extent of the detected variation in permeability
and ignores the absolute value of the variation. Table II shows
that the CNN and the Transformer network are the fastest
models for training and inference. Certainly, the importance
of speed is debatable, given the excellent performance of the
slowest network, the ConvLSTM, which is further discussed
in Section IV-A. The most interesting finding here is that
in our case study, the Transformer shows promising results,
given their short training time – albeit not outperforming
the more conventional ConvLSTM. Regarding the evaluation
on real data, Section IV-A2 gives a detailed insight, where
the performance of the Transformer is much worse than on
simulation-only data.
A. Results
1) Sim-only: Table II presents an overview on how each
of the models performed. The ConvLSTM outperforms all
other models in both IOU and accuracy. But it also takes the
longest to train. The CNN model offers a reasonable tradeoff
between training speed, ease of development, and quality for
our problem. The Transformer gives a two-sided impression:
the metrics on simulated data are close to the best models and
it has very short training times, but the performance on real
data is poor, as described in Section IV-A2.
Figure 4 displays the qualitative performances of the differ-
ent models on selected exemplary runs2. Here, the background
in green shows the same base permeability, whereas higher
permeability is depicted in yellow, and lower permeability is
2More examples at: https://figshare.com/s/1d0ac937e739620fadb9
3
Label CNN ConvLSTM Transformer
Sim
Real
Fig. 4. Example outputs of different models on simulated and real data. Real
data “labels” are taken from simulation.
shown in dark blue. The winner is again the ConvLSTM, with
Transformer as the runner-up. In run Sim, the ConvLSTM
shows the ability to detect higher permeabilities. Only the
Transformer is able detect the area of higher permeability as
well, but in a more blurry fashion. To sum up, the ConvNet
networks performs worse than ConvLSTM and Transformer,
which matches the numbers from Table II.
2) Real data as test set - qualitative comparison of out-
comes: We do not claim to measure absolute values of the per-
meability during the process, but only relative changes to the
permeability. Thus, we used our models, trained on simulated
data, on the same task with real data from the permeameter:
showing relative changes to the permeability. Figure 4 shows
the result for one run for the different models. Since we do not
have the measured permeability map in real life to serve as a
label, we have to use the permeability map from simulation,
presented in the setup in Fig. 3, for qualitative comparison.
When inspecting Fig. 4, several aspects stand out directly.
The qualitative performance is rated with stars in Table II. (1)
The ConvLSTM outperforms every other approach by far, for
every run. The patches of lower permeability have relatively
clear borders and show the same, darker color, referring to
lower permeability, as expected. Other than that, the rest of
the textile is fairly homogeneous, with some slight artifacts,
resembling the flow front (3/3 stars). (2) For the CNN, these
artifacts are much stronger. When taking a closer look at
these related approaches, we can observe stronger deviations
for the CNN model, especially for run Real, while the other
two show acceptable or even good results for both patch and
base permeability. The CNN receives 2/3 stars. (3) Lastly, the
Transformer performs the worst, by far. The outputs on the
different runs look very much alike, with patches of lower
permeability at all places, but the correct one. There are
even slightly lighter spots at the locations, above the center,
where darker patches (of lower permeability) should be for
all runs. This unsatisfying performance on real data of the
pretrained Transformer is an interesting result. Despite their
recent popularity, for our case study, ConvLSTMs are the most
accurate model and standard CNNs are also more desirable.
Given the Transformer’s performance on simulated data (there,
second best), these results were rather disappointing, resulting
in the lowest rating of 1 star.
V. DISCUSSION AND FUTURE WORK
As already mentioned in Section I, in-situ sensor data would
be necessary to apply the results of this paper to an industrial
setting. Another step would be using more real data.With that,
we could try real transfer learning from simulation to reality,
not just testing on real data in order to capture real-life effects
that are not properly captured by simulations mainly based
on Darcy’s law. We could try to learn these effects from real
data and become more accurate than the simulation. Examples
of these effects are channels in the textile that lead to race
tracking, the faster advancement of resin at certain places, and
general divergence regarding timing.
VI. CONCLUSION
We presented a comparison of three network architectures
on a sequence-to-instance task in an engineering domain. This
approach is geared towards a digital twin of the injection
process for CFRP. Models were trained to predict permeability
deviation maps from flow front image sequences. The models
trained on simulated data performed well on real data stem-
ming from a permeameter. Depending on the available training
time, CNNs and ConvLSTMs showed the most promising
results both on simulated and real data.
REFERENCES
[1] D. A. Babb, W. F. Richey, K. Clement, E. R. Peterson, A. P. Kennedy,
Z. Jezic, L. D. Bratton, E. Lan, and D. J. Perettie, “Resin transfer
molding process for composites,” 1998.
[2] K. I. Tifkitsis and A. A. Skordos, “Real time uncertainty estimation in
filling stage of resin transfer molding process,” Polymer Composites,
vol. 41, no. 12, pp. 5387–5402, 2020.
[3] S. Stieber, N. Schr¨
oter, A. Schiendorfer, A. Hoffmann, and W. Reif,
“FlowFrontNet: Improving Carbon Composite Manufacturing with
CNNs,” in ECML PKDD, 2021, pp. 411–426.
[4] S. Stieber, A. Hoffmann, A. Schiendorfer, W. Reif, M. Beyrle, J. Faber,
M. Richter, and M. Sause, “Towards Real-time Process Monitoring and
Machine Learning for Manufacturing Composite Structures,” ETFA -
IEEE Emerging Technology and Factory Automation, pp. 2–5, 2020.
[5] B.-J. Wei, Y.-S. Chang, Y. Yao, and J. Fang, “Online estimation and
monitoring of local permeability in resin transfer molding,” Polymer
Composites, vol. 37, no. 4, pp. 1249–1258, 4 2016.
[6] H. P. G. Darcy, Les Fontaines publiques de la ville de Dijon. V.
Dalamont, 1856.
[7] C. Gonz´
alez and J. Fern´
andez-Le´
on, “A Machine Learning Model
to Detect Flow Disturbances during Manufacturing of Composites by
Liquid Moulding,” Journal of Composites Science, vol. 4, no. 2, p. 71,
2020.
[8] E. Fauster, D. C. Berg, D. Abliz, H. Gr¨
ossing, D. Meiners, G. Ziegmann,
and R. Schledjewski, “Image processing and data evaluation algorithms
for reproducible optical in-plane permeability characterization by radial
flow experiments,” Journal of Composite Materials, vol. 53, no. 1, pp.
45–63, 2019.
[9] X. Shi, Z. Chen, H. Wang, D. Y. Yeung, W. K. Wong, and W. C.
Woo, “Convolutional LSTM network: A machine learning approach for
precipitation nowcasting,” in Advances in Neural Information Processing
Systems, vol. 2015-Janua, 2015, pp. 802–810.
[10] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez,
L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in
Neural Information Processing Systems, vol. 2017-Decem, 6 2017, pp.
5999–6009.
4