Prediction of Failure Categories in Plastic Extrusion Process with Deep Learning

Full text

Research Article
Journal of Intelligent Systems: Theory and Applications 5(1) 2022: 27-34
DOI: 10.38016/jista.878854
_________________________________
* Corresponding Author. Recieved : 12 Feb 2021
E-mail: fatma.demircan.keskin@ege.edu.tr Revision : 16 Sep 2021
Accepted : 27 Oct 2021
**This paper is an extended version of the paper published in the Proceedings Book of 11th International Statistics Congress, on 4-8
October 2019.
Prediction of Failure Categories in Plastic Extrusion Process
with Deep Learning
Fatma Demircan Keskin1* , Ural Gökay Çiçekli2, İsmail Doğukan İçli3
1,2 Ege University/Department of Business Administration, Izmir, Turkey
3 Ege University/Graduate Faculty of Social Sciences, Izmir, Turkey
fatma.demircan.keskin@ege.edu.tr, gokay.cicekli@ege.edu.tr, dogukan.icli@gmail.com
Abstract
Today’s manufacturing vision necessitates extracting insights from the data collected in real-time from manufacturing processes.
Predicting failures with the predictive analysis of the collected process data and preventing these failures by taking necessary actions
before they occur is a key factor in ensuring quality at the desired level, increasing productivity, and reducing costs in production
systems. In the literature on predictive analysis of process data, machine learning and deep learning methods have attracted
considerable attention, especially in recent years. This study has addressed a multi-class failure classification problem in the plastic
extrusion process with a real case study. Classification models have been developed based on Long Short-term Memory (LSTM) as a
deep learning method and Multilayer Perceptron (MLP) and Logistic Regression (LR) as machine learning methods to predict the
failure categories. In the case study, real data taken from the extrusion process of one of the leading insulation companies operated in
Izmir has been used. The final dataset includes actual measurements of seven parameters related to temperature and pressure and
failure categories as the target variable. Three failure categories have been identified to define Category 0 (No failure), Category 1
(Filter change), and Category 2 (Feeding failures) states, and coded as 0,1 and 2 in the models, respectively. LSTM, MLP, and LR’s
performance to predict the failure categories have been evaluated and compared based on accuracy, precision, recall, and F1 Score
measures. LSTM is the highest performing among the three methods, with 100% prediction accuracy for each failure category. On
the other hand, LR and MLP have achieved considerable and close results except for Category 1.
Keywords: Deep learning, failure prediction, machine learning, plastic extrusion process.
Plastik Ekstrüzyon Sürecinde Derin Öğrenme İle Hata Kategorilerinin Tahmini
Öz
Günümüz üretim anlayışı, imalat süreçlerinden gerçek zamanlı olarak toplanan süreç verisinden kestirim yapabilmeyi
gerektirmektedir. Toplanan süreç verilerinin kestirimci analizi ile hataların tahmin edilmesi ve gerekli aksiyonların alınmasıyla
hataların ortaya çıkmadan önlenmesi, üretim sistemlerinde kalitenin istenilen seviyede sağlanması, verimliliğin artırılması ve
maliyetlerin azaltılmasında kilit bir faktördür. Makine öğrenmesi ve derin öğrenme yöntemleri, süreç verilerinin kestirimci
analizinde, özellikle son dönemlerde büyük ilgi görmektedir. Bu çalışmada plastik ekstrüzyon sürecinde çok sınıflı hata sınıflandırma
problemi bir gerçek hayat örneğiyle ele alınmıştır. Problemin çözümü için derin öğrenme yöntemlerinden Uzun-Kısa Süreli Bellek
(LSTM) ve makine öğrenmesi yöntemlerinden Çok Katmanlı Algılayıcı (MLP) ve Lojistik Regresyon (LR) kullanılmıştır.
Çalışmanın uygulama kısmında, İzmir'de faaliyet gösteren Türkiye'nin önde gelen yalıtım firmalarından birinin plastik ekstrüzyon
sürecinden alınan gerçek veriler kullanılmıştır. Nihai veri seti, süreçten alınan sıcaklık ve basınçla ilişkili yedi parametrenin gerçek
ölçümlerini ve hedef değişken olarak hata kategorilerini içermektedir. Modellerde Kategori 0 (Hata yok), Kategori 1 (Filtre değişimi)
ve Kategori 2 (Besleme hataları) durumlarını tanımlamak için üç hata kategorisi belirlenmiş ve sırasıyla 0,1 ve 2 olarak kodlanmıştır.
LSTM, MLP ve LR'nin hata kategorilerini tahmin etme performansı, tahmin doğruluğu, kesinlik, duyarlılık ve F1 skoru metriklerine
göre değerlendirilmiş ve karşılaştırılmıştır. LSTM, her hata kategorisi için %100 tahmin doğruluğu ile en yüksek performansa sahip
olmuştur. LR ve MLP, Kategori 1 dışındaki hata kategorileri tahminlerinde başarılı ve birbirine yakın sonuçlar elde etmiştir.
Anahtar Kelimeler: Derin öğrenme, hata tahmini, makine öğrenmesi, plastik ekstrüzyon süreci

Journal of Intelligent Systems: Theory and Applications 5(1) (2022) 27-34 28
1. Introduction
Rapid developments in digital technologies have
had transformative effects on manufacturing systems
and turned them into smart systems. The current
industrial vision, Industry 4.0, has structured an
interconnected manufacturing environment and forced
companies to reconsider their processes. One of the
critical cornerstones of Industry 4.0 and smart
manufacturing is collecting real-time data from the
plant via sensors and networks and providing value by
conducting a data-driven predictive analysis.
Failures may occur in the manufacturing
environment due to many causes. Therefore, in smart
manufacturing systems, it is critical to monitor
manufacturing processes in real-time, predict failures,
and take appropriate actions to prevent them from
happening to ensure product quality (Tao et al., 2018).
Fault detection and prediction problems in the
manufacturing environment have been extensively
addressed through machine learning methods (Konar
and Chattopadhyay, 2011; Jing and Hou, 2015) and, in
particular, deep learning methods with increasing
interest recently (Jing et al., 2017; Shao et al., 2017;
Zhang et al., 2017a).
Neural networks (Hou, Liu, and Lin, 2003;
Quintana et al., 2011) and LR (De Menezes et al.,
2017) are among the most widely applied supervised
machine learning methods for failure classification and
prediction problems using process data.
LR is a supervised machine learning method with a
wide range of application areas for prediction problems
containing a categorical dependent variable and a set of
independent variables (Caesarendra et al., 2010).
When the dependent variable has multi-class, like the
problem addressed in this study, multi-class LR needs
to be employed. The conditional probability P(Y = y |
X = x) in multi-class LR is calculated by using
Equation (1) (Le Thi et al., 2020):
󰇛    󰇜  󰇛

󰇜
󰇛

󰇜


(1)
where 󰇝󰇛 󰇜      󰇞 is a training set that
includes observation vectors   and labels 
󰇝   󰇞, Q denotes the number of classes, W is the
dxQ matrix and   󰇛   󰇜  . It is aimed to
find a (W,b) pair that maximizes the total probability of
the correct class y to which  belongs. The negative
log-likelihood function needs to be minimized to obtain
(W,b) estimation (Le Thi et al., 2020).
MLP, one of the most employed neural network
techniques, especially for the problems related to
production control (Cadavid et al., 2020), contains
input and output layers of units and hidden unit/units’
layers between them (Fallah, Mitnitski and Rockwood,
2011). In MLP, the units are organized in a feed-
forward layered topology (Venkatesan and Anitha,
2006). MLP uses various nonlinear functions to
convert n inputs to l outputs. In Equation (2), the
activation function used to determine the network
output is given (Yilmaz and Kaynar, 2011):
 󰇛 󰇜 (2)
where f denotes the activation function,  is hth
hidden layer node’s activation and  is the hth
hidden layer node and oth output layer interconnection.
Deep learning, which has significant successful
applications in many different areas such as text
detection and classification, speech and image
recognition, provides advanced analytical opportunities
for analyzing big data obtained from manufacturing
processes (Wang et al., 2018). There have been
different deep learning approaches, including
Convolutional Neural Networks (CNN), Recurrent
Neural Networks (RNN), Auto-encoders, Deep Belief
Network, Deep Boltzmann Machines, and each of them
may have some sub-variants (Zhao et al., 2019). LSTM
is an architecture of RNN which uses past sequences to
forecast future data (Moghar and Hamiche, 2020). In
LSTM architecture, information flows, including
determining which information to remain and how long
it persists, are regulated via input, forget, and output
gates (Bandara, Bergmeir, and Smyl, 2020). The input,
output, and forget gates have different abilities and
tasks in the architecture. The input gate can choose
information necessary to be stored in the internal state,
the output gate has the capability of deciding the output
information, and the forget gate can throw away the
useless information (Zhang et al., 2017b). LSTM has
widely preferred for the predictive analysis of
sequential data and has a wide range of application
areas, including failure prediction, remaining useful
life prediction, voice recognition, time series analysis,
document classification (Nabipour et al., 2020). LSTM
stands out for its ability to recognize long-term
dependencies and patterns in sequential data and
provide more successful results of anomaly and failure
detections than standard RNN in this data type (Greff
et al., 2016; Meyes et al., 2019).
The mathematical expression of LSTM output of
the jth cell () at time t is given in Equation (3)
(Hochreiter and Schmidhuber, 1997; Smagulova and
James, 2019):
󰇛󰇜 󰇛󰇜󰇛󰇛󰇜󰇜 (3)
where 󰇛󰇜 is an internal state:
󰇛󰇜 󰇛󰇜󰇛  󰇜 󰇛󰇜󰇛󰇛󰇜󰇜 (4)
where  is an output of forget gate:
󰇛󰇜 
󰇛󰇜 (5)

Journal of Intelligent Systems: Theory and Applications 5(1) (2022) 27-34 29
The output values of the output gate (󰇜 and
input gate (󰇜 are given in Equations (6)-(7)
(Hochreiter and Schmidhuber, 1997; Smagulova and
James, 2019):
󰇛󰇜 
󰇛󰇜 (6)
󰇛󰇜 
󰇛󰇜 (7)
Net inputs of a cell are expressed in Equations (8)-
(10) (Hochreiter and Schmidhuber, 1997; Smagulova
and James, 2019): (u: denotes units)
󰇛󰇜󰇛  󰇜
 (8)
󰇛󰇜󰇛  󰇜
 (9)
󰇛󰇜󰇛  󰇜
 (10)
This study aims to address the multi-class failure
classification problem in the plastic extrusion process
using the actual sequential process data of an insulation
company by applying LSTM, one of the deep learning
methods widely known for its successful performance
in prediction problems for the sequentially formed
datasets, and machine learning methods of MLP and
LR, to evaluate and compare the class prediction
performance of these approaches.
Even though there have been studies handling the
determination of process parameters problem in the
plastic extrusion process with machine learning
methods (Huang and Liao, 2002; Al Rozuq and Al
Robaidi, 2013; Cirak and Kozan, 2009), none studies
reached addressing any problems in this process with
deep learning. Therefore, this study aims to contribute
to the related literature by addressing the failure
classification problem in the plastic extrusion process
and applying deep learning.
In the next section of this study, some of the
relevant related works addressing similar problems by
applying LSTM, MLP, and LR are introduced.
Afterward, the problem is explained in detail.
Following this section, application findings and their
analysis are presented. Finally, the results are evaluated
and discussed in the conclusion section.
2. Related Works
Failure classification and prediction problems have
been attracted considerable attention in previous
studies. In those studies, wide range of machine
learning based methods, including Artificial Neural
Networks (ANN) (Dreiseitl and Ohno-Machado, 2002;
Gyimothy, Ferenc and Siket, 2005; Singh, Kaur and
Malhotra, 2009), CNN (Janssens et al., 2016; Tan and
Pan, 2019), Decision Tree (DT) (Dreiseitl and Ohno-
Machado, 2002; Gyimothy, Ferenc and Siket, 2005;
Singh, Kaur and Malhotra, 2009), Deep CNN
(Razaviarab, Sharifi and Banadaki, 2019), k-nearest-
neighbour (Dreiseitl and Ohno-Machado, 2002), LR
(Dreiseitl and Ohno-Machado, 2002; Gyimothy, Ferenc
and Siket, 2005; Singh, Kaur and Malhotra, 2009;
Malhotra and Singh, 2011), LSTM (Malhotra et al.,
2015; Zhang et al., 2017b; Morariu et al., 2018; Tan
and Pan, 2019; Ye et al., 2019), MLP (Liukkonen et
al., 2009; Kutyłowska, 2015; Hore et al., 2016; Orrù et
al., 2020), Random Forest (Tan and Pan, 2019) and
Support Vector Machine (SVM) (Dreiseitl and Ohno-
Machado, 2002; Singh, Kaur and Malhotra, 2009;
Zhang et al., 2017a; Oh et al., 2019) have been applied.
This section presents indicators and findings related to
LSTM, MLP, and LR models for the failure prediction
problem used in previous studies.
Zhang et al. (2017b) employed the LSTM-RNN
method to predict the battery’s remaining useful life
with deep learning capability. They compared the
LSTM and SVM methods and noted that the LSTM-
RNN method predictions are more accurate than SVM.
Tan and Pan (2019) proposed a model to predict faults
of wireless networks based on LSTM and CNN. This
study compared CNN, CNN-LSTM, and Random
Forest models’ performances and showed that their
CNN-LSTM hybrid prediction model had better
performance than the other applied models. Malhotra et
al. (2015) studied the fault prediction problem by
applying the stacked LSTM. They used data sets,
including power demand, multi-sensor motor, space
shuttle. Their results indicated that normal time-series
behavior could be modeled with the stacked LSTM.
Morariu et al. (2018) used the LSTM approach to
estimate energy consumption patterns in the production
cycle accurately. They proposed a structure that
processes the information flow in high-capacity
production systems using map reduction algorithms
and focuses on energy consumption with big data
concepts collected in various layers. Ye et al. (2019)
proposed the LSTM-RNN structure by making
parameter estimates for a reasonable estimate of river
water quality.
Hore et al. (2016) used the MLP-FFN classifier to
predict failures of reinforced concrete buildings. They
identified the possibility of failure of the handled
buildings in the future. The experimental results
obtained in this study indicated that the proposed
model provides satisfactory performance. Kutyłowska
(2015) developed MLP networks to model the damage
frequency in the water supply systems. She noted that
the plumbing could use the created model to determine
the frequency of breakdowns and plan the replacement
of broken pipes. Liukkonen et al. (2009) performed a
wave soldering event study to predict product failures
using the MLP neural network model. They focused on
root causes in response to the number of failures they
detected in their work. As the MLP algorithm’s input,
they accepted the types of failure as the output of the
process parameters. Finally, Orrù et al. (2020) applied
MLP and SVM for the fault prediction problem using

Journal of Intelligent Systems: Theory and Applications 5(1) (2022) 27-34 30
real-time collected sensor data from a refinery’s
production line.
Malhotra and Singh (2011) used the LR and seven
other machine learning methods to predict faulty
classes with object-oriented metrics in software testing.
Singh, Kaur, and Malhotra (2009) compared LR,
Artificial Neural Network (ANN), SVM, and DT
methods for the fault proneness of object-oriented
system classes by using Receiver Operating
Characteristic analysis. Gyimothy, Ferenc, and Siket
(2005) employed LR, neural network, and DT for fault
prediction. The results showed that the logistic
regression analysis was significant. Finally, Dreiseitl
and Ohno-Machadob (2002) compared LR and ANN
methods with other classification algorithms, such as
SVM, k-nearest neighbors, and DT.
3. Problem and Data Description
Conducting predictive analysis based on process
data is one of the prerequisites of today’s
manufacturing understanding. In this study, the
classification problem of multi failure types occurring
during the plastic extrusion process of an insulation
company has been addressed. Plastic extrusion is a
continuous process in which a solid plastic material is
converted into a molten fluid; the flowable melt moves
into the die and takes the desired shape. The
temperature and pressure rollers fed from the top and
bottom layers produce a double waterproofing sheet.
The line is fed from top layers via extruders A and B,
and from bottom layers, via extruder C. There are
lower, central, and upper calenders at the end of the
die. Finally, the calendered product is cooled and
wound in rolls. Extruder C is used to reprocess
granulated plastic waste. For this reason, a filter system
is used in Extruder C.
The initial dataset received from the company
covers the real measurements taken every 5 minutes
sequentially and the failure categories at measurement
times. In this process, a number of failure types,
including edge tearing, die cleaning, die changing,
filter changing, failures of material feeding can occur.
However, only filter changes and material feeding
failures to the line during the data collection period
have occurred. So the failure categories have been
labeled as “No failure,” “Filter change,” and “Feeding
failures” in the initial dataset. These categories are
coded to be used in the models as follows:
• No failure (0)
• Filter change (1)
• Feeding failures (2)
Some of the parameters’ values do not change
during the analysis period. Therefore, these parameters
were excluded from the analysis. Also, there are some
parameters with some missing values. After cleaning
the dataset, the final dataset includes 7171 observations
regarding seven parameters and failure categories as
the target variable. The variables, their descriptions,
and ranges are given in Table 1.
Table 1. Description of the variables in the dataset
No
Variable Name
Description
Range
1
Pane1-Temperature
Central Roll (°C)
Temperature of the
central roll
[9.500,57.089]
2
Pane1-Temperature
Lower Roll (°C)
Temperature of the
lower roll
[9.619,68.725]
3
Pane1-Temperature
Upper Roll (°C)
Temperature of the
upper roll
[10.363,58.945]
4
Pane1-Melt
Temp. A (°C)
Melt temperature
of
extruder A
[8.766,196.181]
5
Pane1-Melt ,
Temp. B (°C)
Melt temperature
of
extruder B
[9.530,195.107]
6
Pane1-Melt
Temp. C (°C)
Melt temperature
of
extruder C
[11.527,197.047]
7
Ext. C. melt
pressure_difference
(°C)
Difference of the
two consecutive
melt pressures in
extruder C
[-157.937,223.702]
4. Application and Findings
This study has addressed the multi-class failure
classification problem using actual measurement data
taken from the plastic extrusion process of an
insulation company with LSTM, MLP, and LR. The
performance of LSTM depends on the values of its
hyperparameters. Since there is no exact way of
choosing which hyperparameter values work best, one
of the most frequently followed methods is to use some
combinations of parameters and test these
combinations’ performances with several experiments
(Greff et al., 2016).
In this study, the analysis-ready data set was
randomly divided into training, validation, and testing
sets with the size of 70%, 20%, and 10% of the whole
data set, respectively. Therefore, the fault categories’
observations into the training, validation and testing
sets are as equal as possible. In Table 2, features of the
training, validation, and testing sets are presented.
Table 2. Features of the training, validation, and testing sets
Number (percentage) of
failure categories in the training
set
No failure: 4675
(94.79%)
Filter change: 6
(0.12%)
Feeding failure:
251 (5.09%)
Training set size
4932
Number (percentage) of
failure categories in the
validation set
No failure: 1441
(95.43%)
Filter change: 3
(0.20%)
Feeding failure: 66
(4.37%)
Validation set size
1510
Number (percentage) of
failure categories in the testing
No failure: 695
(95.34%)

Journal of Intelligent Systems: Theory and Applications 5(1) (2022) 27-34 31
set
Filter change: 2
(0.27%)
Feeding failure: 32
(4.39%)
Testing set size
729
Before implementing the analyzed methods, all
inputs were normalized. In normalization, firstly, min-
max and z-score normalization techniques are among
the most widely used normalization techniques. The
classification performance of MLP and LSTM have
indicated that these models have better performance
with the data normalized by the z-score technique.
Therefore, the z-score normalization technique has
been selected, and the results obtained with the z-score
normalized data set have been presented in the rest of
the study. This study followed the methodology of
training models, running the trained models with
multiple parameter settings several times by using the
validation set and finally evaluating the models’
performances in the testing set. Depending on the
dataset’s highly unbalanced structure, the performance
of the methods is evaluated by employing evaluation
metrics of precision, recall, and F1 score for each
category. In addition to these metrics, the overall
accuracy of the methods’ predictions is also computed
and compared. All employed metrics’ formulas are
given in Equations (11)-(14) (Orrù et al., 2020):
  
 (11)
  
 (12)
  
 (13)
    󰇛󰇜
󰇛󰇜 (14)
(TP: True positives, TN: True negatives, FP: False
positives, and FN: False negatives)
In LSTM and MLP models, batch size, epoch,
learning rate, dropout rate, and optimizer type
combinations seen in Table 3 are run ten times in the
validation set, and the combination that gives the best
result among these combinations is employed in the
testing set. All models are coded in R.
Table 3. Parameters in the experiments
Parameters
Value
Output Units
3
Batch Size
4,8,16,32,64
Epoch
10,20,50,100
Optimizer
RMSprop, adam
Learning Rate
0.001,0.0001
Dropout Rate
0.02, 0.2
Confusion matrices of the models are presented in
Figures 1-3. Overall accuracies of the applied methods
are given in the lower right corner of the matrices.
Moreover, at the rightest column and in the bottom
row, recall and precision values of the methods on
each failure category are presented, respectively.
Figure 1. Confusion matrix of LR
Figure 2. Confusion matrix of MLP
Figure 3. Confusion matrix of LSTM
All applied methods have reached a high overall
accuracy. LSTM is the best with 100% accuracy, while
LR is the last with 97.94% accuracy. Performances of
the applied methods in terms of precision, recall, and
F1 Score evaluation metrics are presented in Table 4.
The results have indicated that LSTM has 100%
performance for each category for each evaluation
metric. The most important point revealing the success
of LSTM is that it accurately predicts the class of
Category 1 (Filter change), which occurs only twice in
the whole test set. LR and MLP have been inadequate
to predict the class of these two observations in
Category 1. LR has predicted Category 1 as Category 0
(No failure), while MLP has predicted Category 2

Journal of Intelligent Systems: Theory and Applications 5(1) (2022) 27-34 32
(Feeding failure). The precision of LP and MLP for
Category 1 is obtained as undefined from 0 divided by
0. Therefore, F1 Scores cannot be calculated.
Both LR and MLP have reached high precision,
recall, and F1 Score over Category 0. However, MLP
has better precision and F1 Score for Category 0 than
LR, while LR has a higher recall value over Category 0
than MLP. Therefore, in parallel with this study’s aim,
to test and compare LR and MLP performances, it is
more appropriate to emphasize Category 2 (Feeding
failure) rather than Category 0.
LR has achieved a higher precision value over
Category 2 than MLP. This result has revealed that the
portion of the classes that LR predicts as Category 2
actually to be Category 2 is higher than MLP. On the
other hand, MLP has yielded more successful results
than LR in recall and F1 Score for Category 2. It
implies that the ratio of actual “Feeding failure”
detected correctly by MLP is higher than LR.
A 5-fold cross-validation procedure has been
carried out to assess the validity of the models. As a
result, the overall accuracy of LR, MLP, and LSTM
methods have been obtained as 97.89%, 98.76%, and
99.26%, respectively.
In addition to overall accuracy, precision, recall,
and F1 score for each category for each method have
been calculated. The results presented in Table 5
indicate that LSTM has the best performance for all
metrics. For example, while LR and MLP could not
detect any observations of Category 1, LSTM has
correctly predicted 66.67% of Category 1 observations.
The results of 5-fold cross-validation have also
confirmed that LSTM has the most successful
performance in all failure categories for the problem
examined in this study.
Table 4 Prediction performances of the methods on the testing set
LR
MLP
LSTM
Category
Precision
Recall
F1 Score
Precision
Recall
F1 Score
Precision
Recall
F1 Score
0
98.30%
99.57%
98.93%
99.14%
99.28%
99.21%
100.00%
100.00%
100.00%
1
-
0.00%
-
-
0.00%
-
100.00%
100.00%
100.00%
2
88.00%
68.75%
77.19%
78.79%
81.25%
80.00%
100.00%
100.00%
100.00%
Table 5. Prediction performances of the methods with 5-fold cross-validation
LR
MLP
LSTM
Category
Precision
Recall
F1 Score
Precision
Recall
F1 Score
Precision
Recall
F1 Score
0
98.55%
99.25%
98.90%
99.16%
99.60%
99.38%
99.52%
99.77%
99.64%
1
-
0.00%
-
-
0.00%
-
66.67%
77.78%
68.89%
2
82.87%
72.69%
77.25%
91.31%
84.35%
86.87%
94.86%
90.24%
92.30%
5. Conclusions
This study addresses the problem of failure
classification, which takes an important role in our
age’s manufacturing vision, based on the actual process
data. The problem’s application is conducted using the
actual process data obtained from the plastic extrusion
process of an insulation company. The study aims to
contribute to the literature by addressing the failure
classification problem in the plastic extrusion process
and applying a deep learning method, LSTM, to the
problem. In addition to LSTM, machine learning
methods of MLP and LR are also applied, and
performances of the models are compared based on
accuracy, precision, recall, and F1 Score measures.
The models’ class prediction accuracy has been
obtained within a high range of 97.94% (LR) and
100.00% (LSTM). LSTM has classified all failure
categories correctly. LR and MLP have reached a
considerable and close performance in classifying
Category 0 and Category 2, but they have been
insufficient to predict the class of Category 1.
LSTM, as a deep learning method, has performed
better than the considered machine learning methods
and had 100% accuracy even though the problem
dataset contains an extremely low number of Failure-1
observations. Further studies might test the models
with larger datasets, including sufficient failure
observations and more process parameters.
References
Al Rozuq, R. A. M. I., Al Robaidi, A. M. I. N. 2013.
Application of neural network ANN to predict XLPE
cable in extrusion processes. Journal of Materials
Sciences and Applications, 2013.
Bandara, K., Bergmeir, C., Smyl, S. 2020. Forecasting across
time series databases using recurrent neural networks on
groups of similar series: A clustering approach. Expert
Systems with Applications, 140, 112896.
Cadavid, J. P. U., Lamouri, S., Grabot, B., Pellerin, R.,
Fortin, A. 2020. Machine learning applied in production
planning and control: a state-of-the-art in the era of
industry 4.0. Journal of Intelligent Manufacturing, 1-28.
Caesarendra, W., Widodo, A., Yang, B. S. 2010. Application
of relevance vector machine and logistic regression for

Journal of Intelligent Systems: Theory and Applications 5(1) (2022) 27-34 33
machine degradation assessment. Mechanical Systems
and Signal Processing, 244, 1161-1171.
Cirak, B., Kozan, R. 2009. Prediction of the coating thickness
of wire coating extrusion processes using artificial neural
network ANN. Modern Applied Science, 37, 52-66.
De Menezes, F. S., Liska, G. R., Cirillo, M. A., Vivanco, M.
J. 2017. Data classification with binary response through
the Boosting algorithm and logistic regression. Expert
Systems with Applications, 69, 62-73.
Dreiseitl, S., Ohno-Machado, L. 2002. Logistic regression
and artificial neural network classification models: a
methodology review. Journal of biomedical
informatics, 355-6, 352-359.
Fallah, N., Mitnitski, A., Rockwood, K. 2011. Applying
neural network Poisson regression to predict cognitive
score changes. Journal of Applied Statistics, 389, 2051-
2062.
Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R.,
Schmidhuber, J. 2016. LSTM: A search space
odyssey. IEEE transactions on neural networks and
learning systems, 2810, 2222-2232.
Gyimothy, T., Ferenc, R., Siket, I. 2005. Empirical validation
of object-oriented metrics on open source software for
fault prediction. IEEE Transactions on Software
engineering, 3110, 897-910.
Hochreiter, S., Schmidhuber, J. 1997. Long short-term
memory. Neural computation, 9(8), 1735-1780.
Hore, S., Chatterjee, S., Sarkar, S., Dey, N., Ashour, A. S.,
Balas-Timar, D., Balas, V. E. 2016. Neural-based
prediction of structural failure of multistoried RC
buildings. Structural Engineering and Mechanics, 583,
459-473.
Hou, T. H. T., Liu, W. L., Lin, L. 2003. Intelligent remote
monitoring and diagnosis of manufacturing processes
using an integrated approach of neural networks and
rough sets. Journal of Intelligent Manufacturing, 142,
239-253.
Huang, H. X., Liao, C. M. 2002. Prediction of parison swell
in plastics extrusion blow molding using a neural network
method. Polymer testing, 217, 745-749.
Janssens, O., Slavkovikj, V., Vervisch, B., Stockman, K.,
Loccufier, M., Verstockt, S., de Walle, R.V. Van
Hoecke, S. 2016. Convolutional neural network based
fault detection for rotating machinery. Journal of Sound
and Vibration, 377, 331-345.
Jing, C., Hou, J. 2015. SVM and PCA based fault
classification approaches for complicated industrial
process. Neurocomputing, 167, 636-642.
Jing, L., Zhao, M., Li, P., Xu, X. 2017. A convolutional
neural network based feature learning and fault diagnosis
method for the condition monitoring of
gearbox. Measurement, 111, 1-10.
Konar, P., Chattopadhyay, P. 2011. Bearing fault detection of
induction motor using wavelet and Support Vector
Machines SVMs. Applied Soft Computing, 116, 4203-
4211.
Kutyłowska, M. 2015. Neural network approach for failure
rate prediction. Engineering Failure Analysis, 47, 41-48.
Le Thi, H. A., Le, H. M., Phan, D. N., & Tran, B. 2020.
Stochastic DCA for minimizing a large sum of DC
functions with application to multi-class logistic
regression. Neural Networks, 132, 220-231.
Liukkonen, M., Hiltunen, T., Havia, E., Leinonen, H.,
Hiltunen, Y. 2009. Modeling of soldering quality by
using artificial neural networks. IEEE Transactions on
electronics packaging manufacturing, 322, 89-96.
Malhotra, P., Vig, L., Shroff, G., Agarwal, P. 2015., Long
short term memory networks for anomaly detection in
time series, Proceedings of European Symposium on
Artificial Neural Networks, Computational Intelligence
and Machine Learning, Presses universitaires de Louvain.
22-24 April 2015, pp. 89-94.
Malhotra, R., Singh, Y. 2011. On the applicability of machine
learning techniques for object oriented software fault
prediction. Software Engineering: An International
Journal, 11, 24-37.
Meyes, R., Donauer, J., Schmeing, A., Meisen, T. 2019. A
Recurrent Neural Network Architecture for Failure
Prediction in Deep Drawing Sensory Time Series
Data. Procedia Manufacturing, 34, 789-797.
Moghar, A., Hamiche, M. 2020. Stock market prediction
using LSTM recurrent neural network. Procedia
Computer Science, 170, 1168-1173.
Morariu, C., Răileanu, S., Borangiu, T., Anton, F. 2018, June.
A distributed approach for machine learning in large
scale manufacturing systems. In International Workshop
on Service Orientation in Holonic and Multi-Agent
Manufacturing pp. 41-52. Springer, Cham.
Nabipour, M., Nayyeri, P., Jabani, H., Shahab, S., Mosavi,
A. 2020. Predicting stock market trends using machine
learning and deep learning algorithms via continuous and
binary data; a comparative analysis. IEEE Access, 8,
150199-150212.
Oh, Y., Ransikarbum, K., Busogi, M., Kwon, D., Kim, N.
2019. Adaptive SVM-based real-time quality assessment
for primer-sealer dispensing process of sunroof assembly
line. Reliability Engineering System Safety, 184, 202-
212.
Orrù, P. F., Zoccheddu, A., Sassu, L., Mattia, C., Cozza, R.,
Arena, S. 2020. Machine learning approach using MLP
and SVM algorithms for the fault prediction of a
centrifugal pump in the oil and gas
industry. Sustainability, 12(11), 4776.
Quintana, G., Garcia-Romeu, M. L., Ciurana, J. 2011.
Surface roughness monitoring application based on
artificial neural networks for ball-end milling
operations. Journal of Intelligent Manufacturing, 224,
607-617.
Razaviarab, N., Sharifi, S., Banadaki, Y. M. 2019. Smart
additive manufacturing empowered by a closed-loop
machine learning algorithm, In Nano-, Bio-, Info-Tech
Sensors and 3D Systems III, International Society for
Optics and Photonics, Vol. 10969 2009, p. 109690H.
Shao, S. Y., Sun, W. J., Yan, R. Q., Wang, P., Gao, R. X.
2017. A deep learning approach for fault diagnosis of
induction motors in manufacturing. Chinese Journal of
Mechanical Engineering, 306, 1347-1356.
Singh, Y., Kaur, A., Malhotra, R. 2009. Comparative analysis
of regression and machine learning methods for
predicting fault proneness models. International journal
of computer applications in technology, 352-4, 183-193.

Journal of Intelligent Systems: Theory and Applications 5(1) (2022) 27-34 34
Smagulova, K., & James, A. P. 2019. A survey on LSTM
memristive neural network architectures and
applications. The European Physical Journal Special
Topics, 228(10), 2313-2324.
Tan, Z., Pan, P. 2019. Network Fault Prediction Based on
CNN-LSTM Hybrid Neural Network. In 2019
International Conference on Communications,
Information System and Computer Engineering
CISCE pp. 486-490. IEEE.
Tao, F., Qi, Q., Liu, A., Kusiak, A. 2018. Data-driven smart
manufacturing. Journal of Manufacturing Systems, 48,
157-169.
Venkatesan, P., & Anitha, S. 2006. Application of a radial
basis function neural network for diagnosis of diabetes
mellitus. Current Science, 91(9), 1195-1199.
Wang, J., Ma, Y., Zhang, L., Gao, R. X., Wu, D. 2018. Deep
learning for smart manufacturing: Methods and
applications. Journal of Manufacturing Systems, 48, 144-
156.
Ye, Q., Yang, X., Chen, C., Wang, J. 2019. River Water
Quality Parameters Prediction Method Based on LSTM-
RNN Model. In 2019 Chinese Control And Decision
Conference CCDC pp. 3024-3028. IEEE.
Yilmaz, I., Kaynar, O. 2011. Multiple regression, ANN
(RBF, MLP) and ANFIS models for prediction of swell
potential of clayey soils. Expert systems with
applications, 38(5), 5958-5966.
Zhang, S., Wang, Y., Liu, M., Bao, Z. 2017a. Data-based line
trip fault prediction in power systems using LSTM
networks and SVM. IEEE Access, 6, 7675-7686.
Zhang, Y., Xiong, R., He, H., Liu, Z. 2017b, July. A LSTM-
RNN method for the lithuim-ion battery remaining useful
life prediction. In 2017 Prognostics and System Health
Management Conference PHM-Harbin pp. 1-4. IEEE.
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., Gao, R. X.
2019. Deep learning and its applications to machine
health monitoring. Mechanical Systems and Signal
Processing, 115, 213-237.