ArticlePDF Available

Integrating Physics and Data Driven Cyber-Physical System for Condition Monitoring of Critical Transmission Components in Smart Production Line


Abstract and Figures

In response to the lack of a unified cyber–physical system framework, which combined the Internet of Things, industrial big data, and deep learning algorithms for the condition monitoring of critical transmission components in a smart production line. In this study, based on the conceptualization of the layers, a novel five-layer cyber–physical systems framework for smart production lines is proposed. This architecture integrates physics and is data-driven. The smart connection layer collects and transmits data, the physical equation modeling layer converts low-value raw data into high-value feature information via signal processing, the machine learning modeling layer realizes condition prediction through a deep learning algorithm, and scientific decision-making and predictive maintenance are completed through a cognition layer and a configuration layer. Case studies on three critical transmission components—spindles, bearings, and gears—are carried out to validate the effectiveness of the proposed framework and hybrid model for condition monitoring. The prediction results of the three datasets show that the system is successful in distinguishing condition, while the short time Fourier transform signal processing and deep residual network deep learning algorithm is superior to that of other models. The proposed framework and approach are scalable and generalizable and lay the foundation for the extension of the model.
Content may be subject to copyright.
Integrating Physics and Data Driven Cyber-Physical System for
Condition Monitoring of Critical Transmission Components in
Smart Production Line
Lin Song 1, Liping Wang 1,2, Jun Wu 2,*, Jianhong Liang 2and Zhigui Liu 1
Citation: Song, L.; Wang, L.; Wu, J.;
Liang, J.; Liu, Z. Integrating Physics
and Data Driven Cyber-Physical
System for Condition Monitoring of
Critical Transmission Components in
Smart Production Line. Appl. Sci.
2021,11, 8967.
Academic Editor: José Machado
Received: 14 August 2021
Accepted: 21 September 2021
Published: 26 September 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
1College of Information Engineering, Southwest University of Science and Technology,
Mianyang 621000, China; (L.S.); (L.W.); (Z.L.)
2State Key Laboratory of Tribology and Institute of Manufacturing Engineering,
Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China;
*Correspondence:; Tel.: +86-010-62772633
In response to the lack of a unified cyber–physical system framework, which combined the
Internet of Things, industrial big data, and deep learning algorithms for the condition monitoring of
critical transmission components in a smart production line. In this study, based on the conceptu-
alization of the layers, a novel five-layer cyber–physical systems framework for smart production
lines is proposed. This architecture integrates physics and is data-driven. The smart connection
layer collects and transmits data, the physical equation modeling layer converts low-value raw
data into high-value feature information via signal processing, the machine learning modeling layer
realizes condition prediction through a deep learning algorithm, and scientific decision-making and
predictive maintenance are completed through a cognition layer and a configuration layer. Case
studies on three critical transmission components—spindles, bearings, and gears—are carried out to
validate the effectiveness of the proposed framework and hybrid model for condition monitoring.
The prediction results of the three datasets show that the system is successful in distinguishing
condition, while the short time Fourier transform signal processing and deep residual network deep
learning algorithm is superior to that of other models. The proposed framework and approach are
scalable and generalizable and lay the foundation for the extension of the model.
cyber-physical system; critical transmission components; smart production line; condition
monitoring; machine learning
1. Introduction
Cyber–physical systems (CPSs) are an indispensable part of intelligent manufactur-
ing (IM) and Industry 4.0, which is gradually transforming the landscape of the global
manufacturing industry. CPSs are multidisciplinary systems that combine computation,
communication, and control technologies to conduct real-time measurements, data trans-
mission, monitoring, decision making, feedback control, and other functions on widely
distributed embedded computing systems [
]. CPSs are considered to be a major fea-
ture of the new industrial revolution [
]. The deployment of a CPS system in Industry
4.0 has attracted the interest of many scholars and researchers [
]. Lee et al. [
] believe
that the use of CPSs will move the manufacturing industry towards Industry 4.0; they
proposed a unified 5C architecture as a guideline for deploying CPS. This guideline shows
how to use the feedback from the initial data collection through to analytics in the final
decision-making stage. Zhang et al. [
] presented a four-level unified architecture for
a cyber–physical production system based on a digital twin. Liu et al. [
] presented a
three-level CPS framework for workshops and introduced a guideline for the connection
of physical components; obtaining and preprocessing data; visualizing results; and the
Appl. Sci. 2021,11, 8967.
Appl. Sci. 2021,11, 8967 2 of 20
final knowledge acquisition. The various kinds of divisions of the CPS layer framework
can be divided into five levels [
], four levels [
], and three levels [
], as
shown in Table 1. These layers are divided into simple CPS workflows and basic concepts,
the key CPS technologies and modeling methods used in the manufacturing field are not
considered, and deep learning-based algorithms are not integrated in the framework.
Table 1. The division of CPS layers.
Number of CPS Layer
Framework Layers
five levels [2,3,5,812]: 5C framework
[13]: machine, data analysis, optimization, design layer
four levels [6]: physical, network, virtual, application
[14]: connection, features, mining and modeling,
three levels [7]: physical connection, middleware, computation
[15]: physical resources, local server, cloud server
Unclear delineated [1621]
Sophisticated machine learning (ML) based on industrial big data is a key component
of CPSs that will enable us to reach a new phase of IM [
]. ML can extract the features
of data collected by various sensors in a physical system and ultimately contribute to
the integration of the physical system and cyber space. ML has been applied in practical
industry and achieved impressive performance [
]. Shin et al. [
] proposed deploying
a shop-floor CPS framework where a support vector machine was used as an ML tech-
nology for predicting the occurrence of anomalies. They designed and simulated this
CPS framework based on ML for Industry 4.0. Ahmed et al. [
] proposed a real-time
data-driven CPS with classification and regression ML algorithms implemented in the
system. Wu et al. [
] proposed a novel ML algorithm for imbalanced data in a Prognostics
and Health Management (PHM) CPS.
Table 2
shows the application area and algorithm
implementation of CPS. It can be seen from Tables 1and 2that the methods implemented
by deep learning (DL) do not have a clear CPS architecture but simply involve the CPS
field. Meanwhile, these methods only realize CPS applications based on data modeling,
without considering the influence of physical modeling, and lack a framework to monitor
the condition of critical transmission components (CTCs) in smart production lines (SPLs)
from the perspective of CPS hybrid modeling.
Table 2. The application area and algorithm implementation of CPS.
CPS Application Area CPS Algorithm Implementation
Workshop management system: [7,8] Shallow ML: [2,3,810]
IM: [5,1113]
Biomedicine: [9,18]
Welding: [2,3] DL: [16,1821]
PHM: [6,10,15,16,21]
Production optimization: [17]
Intelligent video monitoring system: [19] No algorithm implementation:
Transportation: [14] [57,1115,17]
Object Detection: [20]
With the development of IM and network collaborative manufacturing, data-driven
SPL now plays an important role in ensuring the stable production of workshops, enter-
prises, and industrial chains [
]. The CTCs of SPL, such as motor spindles, bearings,
and gears, operate under variable conditions for a long time. The breakdown of mechanical
equipment in production lines is caused by motors, bearings, and gears, which account
Appl. Sci. 2021,11, 8967 3 of 20
for failure proportions of 27%, 41%, and 20%, respectively [
]. Xiao et al. [
] identified
motors to be a vital component of the production line and showed that reliability analyses
can ensure the reliable operation of the overall security of the SPL, enabling it to avoid
economic losses and catastrophic accidents. Bampoula et al. [
] proposed a way to change
from preventive maintenance to predictive in cyber–physical production systems and in-
troduced a new DL model for estimating the remaining useful life (RUL) of the monitored
equipment. A new automatic RUL prediction method for continuous production line equip-
ment based on ML is proposed in [
]. Ayvaz et al. [
] developed a data-driven PHM for
potential failures in production lines. Kang et al. [
] performed a systematic literature
review of ML applications in production lines, where quality optimization, scheduling
optimization, yield improvement, and product failure detection are the main research
fields. DL technology has obtained significant achievements in SPL applications. There is
great potential to realize the condition monitoring of CTC by integrating DL technology
into a CPS architecture.
Within the background of network collaborative manufacturing, the integration of
cyber and physical resources of SPL is further deepened and the condition monitoring of
CTC has become a hot area in academia and industry. Some research gaps could be identi-
fied after reviewing the literature: (1) The research focuses on quality control, production
scheduling, yield improvement, etc. The research on the condition monitoring of CTC
in SPL is limited, especially that based on the data of motor spindles in the production
line of machine tools. (2) There is a lack of a unified CPS framework and algorithm imple-
mentation that combines emerging technologies such as Internet of Things, big data, DL,
etc., for CTC condition monitoring and reliability analysis in the manufacturing process of
SPL. (3) The existing framework does not consider the key technical characteristics of CPS,
especially those of deploying a CPS framework from the perspective of hybrid modeling.
In this paper, we proposed a theoretical CPS framework for CTC condition monitoring
and reliability analysis, which is distinguished from existing solutions. Hybrid modeling,
detailed algorithm implementation, and integrating DL rather than shallow ML are the
main characteristics of the CPS framework. The accurate mapping of the vibration signal,
which is easy to collect, and the CTC condition, which is difficult to obtain, was established.
The effectiveness and predictive ability of the framework were validated with the use of
multiple DL algorithms and signal processing steps through experiments on three datasets.
The framework and approach proposed in this paper can be generalized to other SPL big
data scenarios.
The structure of this paper is organized as follows. The deployment and implementa-
tion of the CPS framework and the description of the layers are presented in Section 2. The
proposed hybrid model is elaborated on in Section 3. In Section 4, three experiments are
carried out with different experimental results and the findings are provided to demon-
strate the effectiveness of the developed CPS framework. Some observations and future
directions are summarized in Section 5. Finally, conclusions are given in Section 6.
2. CPS Architecture for CTC Condition Monitoring
CPS is a computer-based technical system that has a clear architecture and sequential
workflow manner that closely connects various complex processes and information from
physical reality with that from cyber space, providing computation, communication, and
control. Inspired by the layer architecture design of CPS, industrial big data, Internet of
Things, and DL, this paper proposes a novel five-layer CPS framework which consists of
the following layers: a smart connection layer, physics equation-based modeling (PEM)
layer, ML-based modeling (MLM) layer, cognition layer, and configuration layer. The
detailed architecture is shown in Figure 1, which presents the integration of physical
components (e.g., motors, spindles, bearings, gears, and sensors) and cyber components
(e.g., communication, computing, control).
Appl. Sci. 2021,11, 8967 4 of 20
Smart Connection Layer
Acquired Data
Predictive Model
State Analysis
MLM LayerCognition Layer
Configuration Layer
Model Library
Update Transfer
Rotation Error Compensation
Replace The Bearing
Gear Maintenance
Data Normalization
Data fusion
Feature Extraction
PEM Layer
Layer Process
Layer Convolution
Full Connection
Hidden Layer
( , ) ( ) ( ) it
STFT t f h t e d
 
1,2 ( ) 2 ,
1,2 1( ) 2 ,
( ) ( )
( ) ( )
j n k l k j n
j n k l k j n
C k h C l
WPT C k g C l
+ +
Figure 1. CPS framework for CTC condition monitoring and reliability analysis.
Smart Connection Layer: data acquisition is the main function of this layer. Sensors
are the sensing elements of physical systems. Through reasonable multi-sensor position
arrangement and type selection [
], variable accurate and reliable signals, such as vibra-
tion, current, force, and rotation error signals, can be collected. The collected datasets are
transmitted to a large-capacity data storage device and saved in the cloud. Two important
problems must be considered at this layer. Firstly, the type and specification of the sensor
ensures the validity of the data, meaning whether the data are reliable. Secondly, the
location of the sensor is related to the accuracy of the DL model, which relates to whether
the data can be used to make accurate predictions.
PEM Layer: CPS modeling based on physical equations is the core technology used in
this layer. The raw data are transferred to this layer through fieldbus and/or industrial
Ethernet. Raw data are heterogeneous, imperfect, and large-scale [
] and cannot be directly
used in the next layer. Data preprocessing, including data fusion, data normalization, and
feature extraction, are the important considerations at this level. Through this layer, low-
value data can be transformed into high-value information. The detailed physical modeling
workflow will be discussed in Section 3.2.
MLM Layer: The meaningful information obtained from the PEM Layer is transferred
to this level to build a CPS data-driven model, and a workflow from the physical compo-
nents to the cyber components is established. This layer needs to provide support for the
development of DL algorithms, the prediction model construction, and transfer learning
and application implementation in a cyber virtual environment. Historical vibration data
Appl. Sci. 2021,11, 8967 5 of 20
will be used to develop and train the ML predictive model off-line, while real-time data
are fed into the pretraining model to achieve online prediction. In practical applications,
the problem of new datasets or tasks will be encountered; transfer learning can effectively
solve this problem of models requiring training from scratch. The essence of this method is
to solve new but similar tasks by applying knowledge that has been learned previously.
Cognition Layer: The real-time condition monitoring data of CTC and the prediction
results from the MLM Layer can be displayed to experts through visualization techniques
such as histograms and confusion matrices. In addition, the meaningful prediction results
can be further analyzed and mined to evaluate the real-time equipment state. This layer
informs us of the optimal decisions to be taken.
Configuration Layer: The core purpose of this layer is to transform the decision
information of the cognition layer into maintenance activities and realize supervisory
control from physical components to cyber components and back to physical components.
The required PHM actions, such as rotation error compensation, replace the bearing, and
gear maintenance, can be taken. The evaluation results of the CTC condition monitoring
and reliability analysis enable a transition from preventive maintenance activities into
predictive ones to be made.
3. Hybrid Modeling: Integrating Physics Preprocessing with Data-Driven Model
3.1. Physical and Data Driven Hybrid Model of CPS
There are two main modeling methods for CPS: modeling based on physical equations
(PE) and modeling based on data. PE modeling uses underlying physics relationships
to derive its mathematical representation; the main advantages of the PE model are its
interpretability and scalability. Interpretability means that the PE model can better un-
derstand the modeling process from input to output, which is a white-box model. The
causal relationship between parameters and variables is clear. Scalability is reflected in the
complex CPS, which is composed of many physical subsystems. Any system of simple
to moderate complexity can be established through the use of this method [
]. How-
ever, considering the complexity, uncertainty, and time-varying characteristics of CTC
working conditions, the physical model is usually simplified [
] to a rough model that
is an incomplete representation of the physical process of the real system. This process
is time-consuming, expensive, and requires a high level domain expertise for industrial
The ML method based on data eliminates the limitation of relying on system dynamics
knowledge and completely depends on data, meaning that it is suitable for modeling
different types of systems. However, as a complex mechanical system, the vibration signal
of CTC usually contains a large amount of redundant information, and it is difficult to
extract valuable state features without physical preprocessing. The use of appropriate data
preprocessing will enable us to improve the prediction accuracy [
]. Therefore, the misuse
of data and the DL model may lead to large errors or even obtain results that violate the
laws of physics.
As shown in Figure 2, in order to fully utilize the advantages of PE and DL, a new
hybrid CPS model based on physical preprocessing and data is proposed for CTC condition
monitoring and reliability analysis. CPS modeling based on PE consists of three steps: raw
vibration signal data fusion, data normalization, and feature extraction based on signal
processing. Data fusion uses multi-sensor information, data normalization eliminates the
influence of data distribution, and feature extraction can obtain valuable features from
the original data. Input data
are transformed into
after physical preprocessing. The
high-value features of the previous step
are divided into training sets and test sets and
then sent into the DL model via CPS data modeling. Through model training and testing,
the optimal DL model is used for the online deployment of SPL. The model training loss
Appl. Sci. 2021,11, 8967 6 of 20
is calculated using the cross entropy loss function. The cross entropy loss function is
expressed as:
is the number of the classes,
denotes predictive probability of the input data
Xbelonging to ith class, pi(x)denotes real probability.
Figure 2. A detailed flow chart of CPS hybrid modeling.
3.2. CPS Modeling Based on PE
Multi-sensor data fusion, as shown in Figure 2, can merge data to make them more in-
formative. The data collected by CPSs are multi-source and heterogeneous. Normalization
is therefore necessary in order to eliminate the limitations of data units and make them
dimensionless, which is convenient for the comparison and weighting of different units or
scales. Meanwhile, data normalization can accelerate the convergence of DL models [
As shown in Figure 2, the data distribution range of sensor 1 and sensor 2 is made more
consistent through the use of data normalization. The normalization method is expressed
Max Min (2)
represents the input sample,
is the maximum value of the sample data,
is the minimum value of the sample data, and Xis the normalized output.
In order to extract the meaningful state features from the data and reduce the difficulty
of DL training, it is necessary to perform signal processing, which is realized by the PE
modeling of the raw data. This processing method can enable us to meet the requirements of
2D convolutional neural networks (CNNs) for two-dimensional matrix inputs. Short-time
Fourier transform (STFT) and wavelet packet decomposition (WPD) are able to extract the
time–frequency characteristics of data. In addition, this paper also evaluates the accuracy
of data matrix transformation (DMT) without signal processing.
STFT: As shown in Figure 3a, through the spectrum function of MATLAB, a section
of the time sequence signal can be directly transformed into a time–frequency heat map
through STFT. The horizontal axis represents time and the vertical axis represents frequency,
while different colors represent different values. The theory of STFT is to multiply the
Appl. Sci. 2021,11, 8967 7 of 20
original signal by the window function and carry out the segment-by-segment Fourier
transform of the original signal by moving the window function. The window functions
commonly used by STFT include rectangular, Hanning, and Gaussian. The equation of
STFT is expressed as:
STFT(t,ω) = Z+
denote the time,
denotes the frequency,
is the time-domain data,
is a window function whose center is at time
, and STFT
is the time–
frequency matrix after STFT. Supposing that the length of a signal sample is 1024, through a
Hanning window STFT with the length of 64, the output data are a 33
33 time–frequency
50 100 150 200 250
Indices of terminal nodes
(b) (c)
Figure 3. Signal preprocessing based on PE. (a) STFT. (b) WPD. (c) DMT.
WPD: As illustrated by Figure 3b, WPD can decompose each vibration signal sample
into a sub-band containing a series of wavelet coefficients through the wpdec function of
MATLAB. WPD’s heat map horizontal axis represents time, the vertical axis represents
different sub-bands, and different colors represent different values. WPD is a widely used
time–frequency analysis method. The equation of WPD is expressed as:
Cj+1,2n(k) =
Cj+1,2n+1(k) =
represent the number of WPD layers and sub-bands, respectively;
denotes the wavelet coefficient sequence of the
node in the
is the original
signal; and
denote the high-pass filter and low-pass filter, respectively. The equation
between hand gis given as:
g(k) = (1)kh(k)(5)
In this paper, DB25 is used as the wavelet basis function and a signal length of 1024
is transformed into a 64
64 wavelet coefficient matrix through 6-level WPD. A brief
illustration of WPD is shown in Figure 4. The wavelet coefficients of different subbands
can be obtained by multi-level WPD.
Appl. Sci. 2021,11, 8967 8 of 20
,2 1
,2 2
,2 3
Figure 4. A brief illustration of WPD.
DMT: The raw data are reshaped from a one-dimensional time sequence signal to a
two-dimensional signal without signal processing. A simple illustration of DMT is shown
in Figure 5, numbers 0 to 1023 denote time sequence signal data point, and each blue square
represents the value of the matrix. Supposing that the length of a signal sample is 1024,
through DMT, the output data are made into a 32 × 32 matrix.
131 992 1023
991 992 1023
Amplitude A/(m s )
Data point
Figure 5. A brief illustration of DMT.
3.3. CPS Modeling Based on DL
As the most important branch of ML, DL has been widely used in computer vision [
and natural language processing [
]. We tested the performance of four categories of
representative models that are based on CNN, including 5-layer CNN, LeNet, multi-scale
CNN (MSCNN), and residual networks (ResNet).
CNN has great advantages in processing matrix data. Sparse connections and weight
sharing are its main characteristics. As shown in Figures 6and 7, CNNs generally include
five parts: an input layer, convolution layer, pooling layer, full connection layer, and output
layer. BN is batch normalization, ReLU is rectifier linear unit activation function, Conv is
convolution layer, GAP is global average pooling layer, MP is max pooling layer, AMP
denotes adaptive max pooling layer, green cubes with different shades of color represent
convolution operations with different convolution kernel sizes. LeNet is a CNN struc-
ture with a small number of channels which was first applied for image
recognition [43]
Wu et al. [44]
applied LeNet for bearing fault diagnosis. A 5-layer CNN is a medium-sized
CNN structure with a final output channel size of 128 [
]. The size of the convolution
kernel is vital in CNN, as different convolution kernel sizes can extract features of different
scales. The design idea of MSCNN comes from the novel network structure inception [
which can extract multi-scale features through parallel convolution branches. The MSCNN
contains three scales, and each scale consists of different kernel sizes, meaning that various
global features and local features of the time–frequency matrix can be extracted. Finally,
Appl. Sci. 2021,11, 8967 9 of 20
the extracted features are concatenated together for classification. The combination of
local and global feature extraction used for MSCNN has been used in gearbox condition
monitoring [
] and bearing fault diagnosis [
]. In CNN, as the number of network layers
increases, the phenomena of gradient disappearance and gradient explosion will occur,
which makes it difficult to train the network. Deep ResNet, as shown in Figure 7, is an
improved variant of CNN that uses identity shortcuts to ease the difficulty of training [
ResNet include a series of residual building units (RBU); an RBU can be composed of BN,
ReLU, and Conv. The output of each RBU is expressed as:
Y=F(X) + X(6)
denotes the input of RBU,
denotes the output of convolution operation,
and Yrepresents the output of RBU.
Figure 6. A brief architecture of LeNet, CNN, and MSCNN.
Input Layer
BN, ReLU, Conv
BN, ReLU, Conv
BN, ReLU, Conv
BN, ReLU, Conv
Full Connection
Output Layer
Figure 7. A brief architecture of ResNet.
Considering the WPD input as an example, the detailed structure and feature size of
each layer are shown in Table 3, where 3
3, 5
5, and 7
7 denote Conv+BN+ReLU with
kernel sizes of 3, 5, and 7; and
2 means that the same RBU block is appended two times in
sequence. The output sizes of each layer, such as 8, 62, 62 are the number of channels, the
height, and the width. In the spindle datasets, the first MP was removed.
Appl. Sci. 2021,11, 8967 10 of 20
Table 3. Feature size output by each layer of the DL algorithm.
CNN LeNet MSCNN ResNet10 ResNet18
8, 62, 62 5×5
6, 60, 60 3×3
16, 62, 62 7×7
64, 32, 32 7×7
64, 32, 32
16, 60, 60 MP
6, 60, 60 3×3
32, 60, 60 MP
64, 16, 16 MP
64, 16, 16
32, 58, 58 5×5
64, 26, 26 MP
32, 30, 30 RBU, 3 ×3
64, 16, 16 ×1RBU, 3 ×3
64, 16, 16 ×2
32, 29, 293×3, 5 ×5, 7 ×7
192, 30, 30  RBU, 3 ×3
128, 16, 16 ×1RBU, 3 ×3
128, 16, 16 ×2
64, 27, 273×3, 5 ×5, 7 ×7
768, 30, 30  RBU, 3 ×3
256, 16, 16 ×1RBU, 3 ×3
256, 16, 16 ×2
128, 25, 25 RBU, 3 ×3
512, 16, 16 ×1RBU, 3 ×3
512, 16, 16 ×2
128, 6, 6 AMP
128, 5, 5 AMP
768, 4, 4 GAP
512, 1, 1 GAP
512, 1, 1
4. CPS Framework Implementation and Experimentation
Spindle motors, bearings, and gears are three CTCs of SPL equipment. Component
failure will seriously affect the normal operation of the SPL and cause huge economic
losses [
]. Therefore, in this paper, the prediction of spindle motor rotation error
and the fault diagnosis of bearings and gears based on two public datasets are used to verify
the ability of the proposed CPS framework to tackle the condition monitoring problems of
4.1. Case 1: The Prediction of Spindle Motor Rotation Error
As complex systems integrating mechanical, electrical, hydraulic, and pneumatic
aspects, spindle motors are the CTC of SPL. The rotation error of a spindle is closely
related to the geometric error, surface quality, and roughness of the workpiece, which
is an important index used to reflect the reliability of machine tools [
]. Therefore,
the accurate prediction of spindle rotation error is of great significance for improving
machining precision and efficiency. The existing rotation error measurement methods are
usually implemented with the aid of a standard ball in the case of idle conditions [
which makes it difficult to reflect the real error in the actual machining process. The
derivation of the method used for predicting the rotation error based on the physical
dynamics model is complicated and inaccurate [
]. It is difficult to achieve real-time
condition monitoring and real-time rotation error compensation. The spindle rotation error
can be affected by the speed [
] and wear state [
]. Fortunately, the spindle vibration
data contain information related to the speed [
] and wear state [
]. Therefore, it is
possible to predict the rotation error through vibration data.
As shown in Figure 8, the spindle reliability experiment platform of Tsinghua Uni-
versity was established based on CPS. A loading experiment was carried out through the
spindle load spectrum [
] to ensure that the spindle load was close to the actual machining.
The spindle rotation error was collected by the spindle check machine capability tester
every 10 h, and the wear test was carried out during the rest time.
Appl. Sci. 2021,11, 8967 11 of 20
Spindle Drive
NI PXIe-1082
Data Acquisition
And Control Unit
Air Pump
Eddy Current
Sensors Vibration
Rotary Signal Processing
Analysis and Display
Cylinder Speed Sensor
Figure 8. The spindle experimental platform.
Sensors: The vibration sensors were installed on the base, bearing, and spindle to
collect vibration signals. The force sensor collected the load force to realize the closed-loop
control of the load spectrum; the speed sensor measured the spindle speed to realize closed-
loop control; and the eddy current sensor was used to detect the spindle displacement. The
rotation error could be obtained by a spindle check machine capability tester. A detailed
description of the smart connection layer components is provided in Table 4, including the
key components of the smart connection layer: specification, communication mode, and
function. These constitute the smart connection layer of CPS.
Table 4. Detailed description of the Smart Connection Layer components.
Key Components of Specification Communication Mode
Smart Connection Layer and Function
PXIe-1082 communicates with spindle
Spindle and driver through RS485-USB interface to
spindle drive control spindle torque, acceleration and
PCB 256A14
Connect with PXIe-1082 through Integrated
Vibration sensor Electronic Piezo-Electric interface to collect
vibration signals
Force sensor DYMH-104 The 4-line difference signal is fed back to
the analog input channel of PXIe-1082
Spindle check machine
capability tester Lion
The output digital signal of the tester is
directly connected with the NI PXIe-1082
through the USB interface, three eddy
current sensors and one speed sensor are
integrated in the tester
Data acquisition NI PXIe-1082 Data acquisition and spindle control
and control unit
Pneumatic AirTac-100 PXIe-1082 controls the pneumatic loading
loading unit unit through analog output channel
Appl. Sci. 2021,11, 8967 12 of 20
PXIe-1082: A data acquisition and control unit is the core component of the system
and is mainly used to realize the loading control of air pump, multi-sensor data acquisition,
data transmission to the analysis platform, spindle drive, and feedback control. The
configuration layer of the CPS is realized in this part.
Data analysis platform: This part is based on MATLAB and Python. The acquisition
and control unit sends the data to the analysis platform to support the PEM layer, MLM
layer, and cognition layer of CPS, which is the core part of the data processing and rotation
error prediction.
After the data acquisition, it is necessary to preprocess the vibration signal and the
corresponding rotation error. The spindle speed ranges from 1000 r/min to 4000 r/min; the
experiment shows that the variation range of rotation error is 5–14.5
m. After discretiza-
tion, the rotation error data are rounded to the nearest 0.5. Therefore, there are 20 types
of rotation error and the number class is set to 20. Two datasets of each class for a total of
40 datasets
were selected for the experiment; each dataset contained 200,000
3 (
3 means
3 sensors) raw data points. The former 70% of data points were selected as the training
data, while the last 30% were used as testing data. In this paper, the sample length was
1024 data points and the shift length [
] was 320 data points. Hence, the total number of
samples in each class was 870 and the total number of training samples was 20
870. Data
augmentation was not used in the testing set, and the total number of testing samples was
20 × 116.
4.2. Case 2: The Fault Diagnosis of Bearing
The bearing datasets contained 12 vibration sub-datasets, and the number class was
12. Vibration signals were collected at the three speeds of 600 r/min, 800 r/min, and
1000 r/min
. Each working condition contained one normal state and three fault modes,
which include rolling elements, outer rings, and inner rings. Detailed descriptions of
the bearing datasets are shown in Table 5. The detailed description of the experimental
platform can be found in reference [
]. After data preprocessing, the total number of
training samples was 7038 and the total number of testing samples was 1764.
Table 5. Detailed description of the bearing datasets.
Fault Mode Speed (r/min)
rolling element 600, 800, 1000
outer ring 600, 800, 1000
inner ring 600, 800, 1000
normal state 600, 800, 1000
4.3. Case 3: The Fault Diagnosis of Gear
The gear datasets contained 20 vibration sub datasets and the number class was
20. Vibration signals were collected in two kinds of working conditions. Each working
condition contained two normal states and eight fault modes. Detailed information on the
gear datasets is displayed in Table 6. A detailed description of the experimental platform
can be found in the reference [
]. In particular, for this gear drivetrain diagnostics
simulator datasets, Gaussian noise with a signal-to-noise ratio of 1 dB is added to the
vibration signal [
]. After data preprocessing, the total number of training samples was
16,380 and the total number of testing samples was 4100.
Appl. Sci. 2021,11, 8967 13 of 20
Table 6. Detailed description of the gear datasets.
Fault Mode Working Condition
normal gear 20 Hz—0 V, 30 Hz—2 V
normal bearing 20 Hz—0 V, 30 Hz—2 V
chipped tooth 20 Hz—0 V, 30 Hz—2 V
missing tooth 20 Hz—0 V, 30 Hz—2 V
root 20 Hz—0 V, 30 Hz—2 V
surface 20 Hz—0 V, 30 Hz—2 V
inner ring 20 Hz—0 V, 30 Hz—2 V
outer ring 20 Hz—0 V, 30 Hz—2 V
inner+outer ring 20 Hz—0 V, 30 Hz—2 V
rolling element 20 Hz—0 V, 30 Hz—2 V
4.4. Experiment and Result Analysis
This part focuses on the performance of different DL methods rather than the setting
of the hyperparameters. In the experiment, different methods in the same datasets used
the same hyperparameters. The learning rate was 0.001 for spindle datasets. The learning
rate was 0.001 from 0 to 29 epochs, 0.0001 from 30 to 59 epochs, and 0.00001 in the last
40 epochs
for the bearing and gear datasets. The mini-batch size was 64 and the max
number of epochs was 100. In the process of model training, Adam was used as the
optimizer. Momentum is an important parameter of Adam and can accelerate the training
process; it was set to 0.9 following the setup seen in [65].
The experiments were conducted on a system with i7 CPU @3.80 GHz and NVIDIA
GeForce RTX 2060 SUPER. Python was used as a programming environment with pytorch.
In the experiment, the maximum accuracy across all the epochs was chosen as the testing
accuracy. In this experiment, we evaluated the accuracy when using different signal
processing procedures and DL algorithms. In order to reduce the impact of the randomness,
five trials were carried out for each experiment. The average performance of these methods,
including LeNet, CNN, MSCNN, ResNet10, ResNet18, long short-time memory network
(LSTM), and bidirectional LSTM (BiLSTM) [
], are shown in Tables 79and Figures 911,
in which the horizontal axis represents the combination of different DL algorithms and
signal processing, the vertical axis represents the average accuracy, the histogram and black
numbers represent the average accuracy value and the red box chart shows the dispersion
at five times. STFT and ResNet obtained the best accuracy values of 92.08%, 95.12%, and
94.56%, respectively, in all three datasets.
Table 7. Experimental results of the spindle datasets.
Signal Average Testing Accuracy ± Standard Deviation (%)
Processing MSCNN CNN LeNet ResNet10 ResNet18 LSTM BiLSTM
DMT 82.96 ±1.39 74.78 ±2.10 54.95 ±2.21 58.87 ±4.41 60.43 ±4.31 61.66 ±1.55 62.20 ±0.80
STFT 90.65 ±0.26 85.90 ±0.98 86.88 ±0.21 90.69 ±0.39 92.08 ±0.41 88.90 ±0.64 89.77 ±0.28
WPD 89.88 ±0.47 88.21 ±0.54 83.73 ±0.55 86.59 ±0.23 86.28 ±0.43 84.41 ±0.45 84.85 ±1.04
Table 8. Experimental results of the bearing datasets.
Signal Average Testing Accuracy ± Standard Deviation (%)
Processing MSCNN CNN LeNet ResNet10 ResNet18 LSTM BiLSTM
DMT 89.05 ±0.80 82.69 ±1.34 59.15 ±1.48 86.63 ±0.61 86.03 ±0.66 80.41 ±0.68 80.40 ±0.90
STFT 93.48 ±0.27 90.41 ±1.14 83.87 ±1.37 95.02 ±0.22 95.12 ±0.20 89.90 ±0.79 91.01 ±0.56
WPD 91.94 ±0.53 88.53 ±0.91 70.58 ±1.64 91.15 ±0.40 91.10 ±0.46 83.80 ±0.73 84.79 ±0.63
Appl. Sci. 2021,11, 8967 14 of 20
Table 9. Experimental results of the gear datasets.
Signal Average Testing Accuracy ± Standard Deviation (%)
Processing MSCNN CNN LeNet ResNet10 ResNet18 LSTM BiLSTM
DMT 94.06 ±0.46 88.45 ±0.19 64.80 ±4.42 93.66 ±0.37 93.08 ±0.51 77.95 ±1.20 77.24 ±0.25
STFT 94.41 ±0.23 91.77 ±0.82 88.24 ±0.35 94.56 ±0.27 93.98 ±0.15 92.24 ±0.37 92.55 ±0.42
WPD 83.97 ±1.88 86.27 ±1.54 74.18 ±2.18 90.50 ±0.61 89.53 ±0.43 83.69 ±0.80 84.13 ±0.40
The average performance in spindle datasets
58.87 60.43
82.96 84.85 85.90 86.28 86.59 86.88 88.21 89.77 90.69 92.08
83.73 84.41
89.88 90.65
DL+Signal Processing
Average Accuracy(%)
Figure 9. The results of the spindle datasets.
The average performance in bearing datasets
80.40 80.41
82.69 83.80 83.87 84.79 86.03 86.63 88.53 89.05 89.90 90.41 91.01 91.10 91.15 91.94 93.48 95.02 95.12
DL+Signal Processing
Average Accuracy(%)
Figure 10. The results of the bearing datasets.
The average performance in gear datasets
74.18 77.24 77.95
83.69 83.97 84.13 86.27 88.24 88.45 89.53 90.50 91.77 92.24 92.55 93.08 93.66 93.98 94.06 94.41 94.56
DL+Signal Processing
Average Accuracy(%)
Figure 11. The results of the gear datasets.
Appl. Sci. 2021,11, 8967 15 of 20
As can be seen from Tables 79and Figures 911, STFT always obtains the best
accuracy in three datasets, the overall average accuracy is STFT > WPD > DMT. STFT and
WPD can be used to gain the time–frequency characteristics of the data, and their accuracy
is significantly higher than that of DMT. This result proves the necessity of the proposed
PEM layer. The reason why the accuracy of STFT is higher than that of WPD is that the
matter of how to choose the optimal wavelet basis function and wavelet decomposition
level is challenging. For example, in the experiment, it was found that the accuracy of
the CNN in spindle datasets was 72.24% for five-level wavelet decomposition under a
DB1 wavelet basis function and 88.21% for six-level wavelet decomposition under a DB25
wavelet basis function, which represents an improvement of 15.97%.
At the same time, we can see that the ResNet can always obtain the best accuracy.
The order for the accuracy of the proposed hybrid modeling in three datasets with STFT
is ResNet > MSCNN > BiLSTM > CNN > LeNet. Compared with other methods, the
accuracy of LeNet is the lowest. As the depth of LeNet is the shallowest and the number
of convolution channels here is the lowest, the features contained in the signal cannot be
extracted completely. CNN has more channels than LeNet and can extract more feature
information, so the accuracy of CNN is greater than LeNet. The BiLSTM designed in
reference [
] can not only extract spatial feature information through convolution, but
also extract temporal information. The accuracy of BiLSTM is greater than CNN. BiLSTM
can extract bidirectional temporal features, and its accuracy is greater than LSTM. MSCNN
can extract multi-scale features, which significantly improves the feature extraction ability,
and the performance of MSCNN is better than that of BiLSTM, CNN and LeNet. With the
network depth increases, ResNet can overcome the phenomena of gradient disappearance
and explosion, which makes the network easy to train; thus, ResNet has the best accuracy.
In the spindle and bearing datasets, ResNet18 performs better than ResNet10. In gear
datasets, ResNet10 performs better than ResNet18. This may be due to the slight overfitting
of the gear datasets.
Through the above analysis, the method proposed in this paper successfully monitors
the condition of CTC in SPL. The experimental results of STFT, WPD, DMT and different
DL algorithms show the necessity of CPS hybrid modeling and the feasibility of algorithm
To better understand the classification effect of these models for each label,
Figure 12
shows the confusion matrices of LeNet, CNN, LSTM, BiLSTM, MSCNN, and ResNet18 in
bearing datasets with STFT signal processing. In Figure 12, it is evident that almost all the
categories in the ResNet18 model are easier to diagnose than those of other models, except
for label 4, where ResNet18 is slightly less easy to diagnose than MSCNN.
(a) (b)
Figure 12. Cont.
Appl. Sci. 2021,11, 8967 16 of 20
(c) (d)
(e) (f)
Figure 12. Confusion matrix of bearing datasets. (a) LeNet. (b) CNN. (c) LSTM. (d) BiLSTM. (e) MSCNN. (f) ResNet18.
5. Discussions and Future Works
(1) Modeling Based on PE
Vibration data have the characteristics of high dimensionality, nonlinearity, and di-
versity. Signal processing is the core of the PEM layer. Other signal processing methods,
including empirical mode decomposition, local mean decomposition, and Hilbert–Huang
transform, can extract discriminative features from different domains, and those feature
extraction methods are worth studying. In the PEM layer, the question of how to extract
valuable features from noisy data and extract a sufficient number of features from imbal-
anced data are two problems that need to be considered. CTC often operates under harsh
working environments, and the vibration signal collected often contains irregular noise.
Feature information can easily be annihilated by strong background noise. At the same
time, it was found that the distribution between each category of samples was imbalanced
in our experiment. In the context of CPS hybrid modeling, there is still a great amount of
progress to be made in the feature extraction of noisy and imbalanced data.
(2) Modeling Based on data
Whether the data-driven CPS framework can be successfully deployed in the SPL de-
pends on the quality of the datasets used. The more labeled multi-sensor data are available,
the better the accuracy will be. In fact, the use of data with less labels limits the application
of DL in the CPS. The other problem is represented by limited data. A major challenge in
this CPS framework is in obtaining a sufficient amount of training data, which is a time-
consuming, laborious, and costly process. The use of semi-supervised or unsupervised DL
for less-labeled data and few-shot learning with a meta-learning paradigm for limited data
are techniques that have been widely used in the field of computer vision. These methods
are worth using in attempts to solve the above problems.
Appl. Sci. 2021,11, 8967 17 of 20
(3) Integrating digital twin with CPS
CPS improves the communication between physical and cyber space. In a digital
twin, a high-fidelity digital copy is built through big data and physical models in physical
systems to provide services for CPS monitoring, analysis, decision making, and feedback to
physical entities. The question of how to integrate digital twins with CPS for the condition
monitoring of CTC in SPL would be an interesting research direction to explore in the
(4) Infusing 5G into CPS
Various industrial Ethernet and fieldbus technologies are still the main data transmis-
sion technologies used in CPSs. Some wireless transmission technologies, such as Bluetooth
and ZigBe, etc., have also been applied to CPS. 5G promotes the development of wireless
communication technology. Its main features are enhanced mobile broadband, low latency,
high reliability, and mass machine communication. 5G can effectively meet the needs of
CPS for large-scale data acquisition and sensing, precise control, remote processing. The
question of how to deploy 5G in all layers of CPS, especially in the smart connection layer,
should be addressed in future research.
6. Conclusions
In this paper, CPS hybrid modeling was investigated and a novel five-layer architec-
ture which included a PEM layer and an MLM layer was proposed for CTC condition
monitoring in SPL. The CPS framework integrated various DL-based algorithms. Firstly,
the multi-sensor data of the SPL monitoring CTC were collected via a smart connection
layer; then, a large amount of low-value data was transformed into high-value feature
information through the PEM layer. After this, the feature information was used to train
and evaluate the DL model through the MLM layer; visualization techniques helped us to
make more effective and scientific decisions via the cognition layer. Finally, the decision
information was transformed into predictive maintenance activities through a configu-
ration layer. Therefore, the CPS framework was able to realize a closed-loop workflow
and reduce the potential for failures that could impact SPL operations. The effectiveness
and predictive power of the developed CPS framework, hybrid modeling, and algorithm
implementation were verified by the DL and signal processing methods. The proposed
framework and approach could be generalized in order to predict the RUL, product quality,
etc. from scratch with the integration of additional sensors and a training DL model, which
would lay the foundation for framework extensions.
Author Contributions:
Conceptualization, L.S. and J.W.; methodology, L.S. and J.W.; software, L.S.;
validation, L.W., J.W. and Z.L.; formal analysis, L.S.; investigation, L.S. and J.W.; resources, L.W.,
J.W. and Z.L.; data curation, L.S., J.L.; writing—original draft preparation, L.S.; writing—review and
editing, J.W.; visualization, L.S.; supervision, L.W. and Z.L.; project administration, J.W.; funding
acquisition, J.W. All authors have read and agreed to the published version of the manuscript.
This research was funded by The National Key Research and Development Program of
China grant number 2019YFB1706703.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement:
All data generated or analyzed during this study are included in
this article.
Conflicts of Interest: The authors declare no conflict of interest.
Appl. Sci. 2021,11, 8967 18 of 20
The following abbreviations are used in this manuscript:
CPS cyber physical system
IM intelligent manufacturing
ML machine learning
RUL remaining useful life
PHM prognostics and health management
SPL smart production line
DL deep learning
CTC critical transmission components
PEM physics equation-based modeling
MLM machine learning-based modeling
PE physical equations
CNN convolutional neural network
STFT short time Fourier transform
WPD wavelet packet decomposition
DMT data matrix transformation
MSCNN multi-scale CNN
ResNet residual network
RBU residual building units
Liu, Y.; Peng, Y.; Wang, B.; Yao, S.; Liu, Z. Review on cyber-physical systems. IEEE/CAA J. Autom. Sin.
,4, 27–40. [CrossRef]
Ahmed, F.; Jannat, N.E.; Gavidel, S.Z.; Rickli, J.; Kim, K.Y. A Conceptual Framework for Cyber-physical System in Connected
RSW Weldability Certification. Procedia Manuf. 2019,38, 431–438. [CrossRef]
Ahmed, F.; Jannat, N.E.; Schmidt, D.; Kim, K.Y. Data-driven cyber-physical system framework for connected resistance spot
welding weldability certification. Robot. Comput.-Integr. Manuf. 2021,67, 102036. [CrossRef]
Chang, L.P.; Kuo, T.W.; Gill, C.; Nakazawa, J. Introduction to the special issue on real-time, embedded and cyber-physical systems.
ACM Trans. Embed. Comput. Syst. 2014,13, 1–2. [CrossRef]
5. Lee, J.; Bagheri, B.; Kao, H.A. A cyber-physical systems architecture for industry 4.0-based manufacturing systems. Manuf. Lett.
2015,3, 18–23. [CrossRef]
Zhang, H.; Yan, Q.; Wen, Z. Information modeling for cyber-physical production system based on digital twin and automationml.
Int. J. Adv. Manuf. Technol. 2020,107, 2. [CrossRef]
Chao, L.; Jiang, P. A cyber-physical system architecture in shop floor for intelligent manufacturing. Procedia CIRP
Hyun-Jun, S.; Kyoung-Woo, C.; Chang-Heon, O. Svm-based dynamic reconfiguration cps for manufacturing system in industry
4.0. Wirel. Commun. Mob. Comput. 2018,2018, 13.
Feng, J.; Zhu, F.; Li, P.; Davari, H.; Lee, J. Development of An Integrated Framework for Cyber Physical System (CPS)-Enabled
Rehabilitation System. Int. J. Prognost. Health Manag. 2021,12. [CrossRef]
10. Lee, J.; Jin, C.; Bagheri, B. Cyber physical systems for predictive production systems. Prod. Eng. 2017,11, 155–165. [CrossRef]
Bagheri, B.; Yang, S.; Kao, H.A.; Lee, J. Cyber-physical systems architecture for self-aware machines in industry 4.0 environment.
IFAC-PapersOnLine 2015,48, 1622–1627. [CrossRef]
Lee, J.; Ardakani, H.D.; Yang, S.; Bagheri, B. Industrial big data analytics and cyber-physical systems for future maintenance and
service innovation. Procedia Cirp 2015,38, 3–7. [CrossRef]
Lee, J.; Azamfar, M.; Singh, J.; Siahpour, S. Integration of digital twin and deep learning in cyber-physical systems: Towards
smart manufacturing. IET Collab. Intell. Manuf. 2020,2, 34–36. [CrossRef]
Liu, Z.; Zhang, Z.; Xu, G.; Jin, W.; Lee, J. Design of Cyber-Physical Systems Architecture for Prognostics and Health Management
of High-speed Railway Transportation Systems. Int. J. Prognost. Health Manag. 2018,9. [CrossRef]
Herwan, J.; Kano, S.; Oleg, R.; Sawada, H.; Kasashima, N. Cyber-physical system architecture for machining production line.
IEEE Ind. Cyber-Phys. Syst. 2018,1, 387–391.
Bampoula, X.; Siaterlis, G.; Nikolakis, N.; Alexopoulos, K. A deep learning model for predictive maintenance in cyber-physical
production systems using lstm autoencoders. Sensors 2021,21, 972. [CrossRef] [PubMed]
Lim, K.; Zheng, P.; Chen, C.; Huang, L. A digital twin-enhanced system for engineering product family design and optimization.
J. Manuf. Syst. 2020,57, 82–93. [CrossRef]
Pandey, A.; Sequeria, R.; Kumar, P.; Kumar, S. A multistage deep residual network for biomedical cyber-physical systems. IEEE
Syst. J. 2019,14, 1953–1962. [CrossRef]
Wasim, M.; Ahmed, I.; Ahmad, J.; Hassan, M. A Novel Deep Learning Based Automated Academic Activities Recognition in
Cyber-Physical Systems. IEEE Access 2021,9, 63718–63728. [CrossRef]
Appl. Sci. 2021,11, 8967 19 of 20
Zhou, X.; Xu, X.; Liang, W.; Zeng, Z.; Shimizu, S.; Yang, L.; Jin, Q. Intelligent small object detection based on digital twinning for
smart manufacturing in industrial CPS. IEEE Trans. Ind. Inf. 2021,99, 1.
Wu, Z.; Guo, Y.; Lin, W.; Yu, S.; Ji, Y. A weighted deep representation learning model for imbalanced fault diagnosis in
cyber-physical systems. Sensors 2018,18, 1096. [CrossRef]
Fei, X.; Shah, N.; Verba, N.; Chao, K.M.; Sanchez-Anguix, V.; Lewandowski, J. CPS data streams analytics based on machine
learning for Cloud and Fog Computing: A survey. Future Gener. Comput. Syst. 2018,90, 435–450. [CrossRef]
Dreossi, T.; Donzé, A.; Seshia, S.A. Compositional falsification of cyber-physical systems with machine learning components. J.
Autom. Reason. 2019,63, 1031–1053. [CrossRef]
Subramaniyan, M.; Skoogh, A.; Salomonsson, H.; Bangalore, P.; Bokrantz, J. A data-driven algorithm to predict throughput
bottlenecks in a production system based on active periods of the machines. Comput. Ind. Eng. 2018,125, 533–544. [CrossRef]
Chen, B.; Liu, Y.; Zhang, C.; Wang, Z. Time series data for equipment reliability analysis with deep learning. IEEE Access
105484–105493. [CrossRef]
Ebrahimipour, V.; Najjarbashi, A.; Sheikhalishahi, M. Multi-objective modeling for preventive maintenance scheduling in a
multiple production line. J. Intell. Manuf. 2015,26, 111–122. [CrossRef]
Zhang, Y.; Cheng, Y.; Wang, X.; Zhong, R.; Zhang, Y.; Tao, F. Data-driven smart production line and its common factors. Int. J.
Adv. Manuf. Technol. 2019,103, 1211–1223. [CrossRef]
Henriquez, P.; Alonso, J.B.; Ferrer, M.A.; Travieso, C.M. Review of automatic fault diagnosis systems using audio and vibration
signals. IEEE Trans. Syst. Man Cybern. Syst. 2014,44, 642–652. [CrossRef]
Xiao, D.; Huang, Y.; Zhang, X.; Shi, H.; Liu, C.; Li, Y. Fault diagnosis of asynchronous motors based on LSTM neural network.
PHM-Chongqing 2018,2018, 540–545.
Kang, Z.; Catal, C.; Tekinerdogan, B. Remaining useful life (RUL) prediction of equipment in production lines using artificial
neural networks. Sensors 2021,21, 932. [CrossRef]
Ayvaz, S.; Alpay, K. Predictive maintenance system for production lines in manufacturing: A machine learning approach using
IoT data in real-time. Expert Syst. Appl. 2021,173, 114598. [CrossRef]
32. Kang, Z.; Catal, C.; Tekinerdogan, B. Machine learning applications in production lines: A systematic literature review. Comput.
Ind. Eng. 2020,149, 106773. [CrossRef]
Li, T.; Zhao, Z.; Sun, C.; Yan, R.; Chen, X. Adaptive channel weighted CNN with multisensor fusion for condition monitoring of
helicopter transmission system. IEEE Sens. J. 2020,20, 8364–8373. [CrossRef]
34. Meng, T.; Jing, X.; Yan, Z.; Pedrycz, W. A survey on machine learning for data fusion. Inf. Fusion 2020,57, 115–129. [CrossRef]
Rai, R.; Sahu, C.K. Driven by data or derived through physics? A review of hybrid physics guided machine learning techniques
with cyber-physical system (CPS) focus. IEEE Access 2020,8, 71050. [CrossRef]
Kang, T.; Cao, H.R. Dynamic prediction method for machine tool spindle rotational accuracy under cutting condition. J. Mech.
Eng. 2020,56, 9.
Tang, S.; Yuan, S.; Zhu, Y. Data preprocessing techniques in convolutional neural network based on fault diagnosis towards
rotating machinery. IEEE Access 2020,8, 149487. [CrossRef]
Zhao, Z.; Li, T.; Wu, J.; Sun, C.; Chen, X. Deep learning algorithms for rotating machinery intelligent diagnosis: An open source
benchmark study. ISA Trans. 2020,107, 224–255. [CrossRef] [PubMed]
Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf.
Process. Syst. 2012,25, 1097–1105. [CrossRef]
Farabet, C.; Couprie, C.; Najman, L.; LeCun, Y. Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach.
Intell. 2012,35, 1915–1929. [CrossRef]
41. Hirschberg, J.; Manning, C.D. Advances in natural language processing. Science 2015,349, 261–266. [CrossRef]
Sun, S.; Luo, C.; Chen, J. A review of natural language processing techniques for opinion mining systems. Inf. Fusion
10–25. [CrossRef]
43. Lecun, Y.; Bottou, L. Gradient-based learning applied to document recognition. Roceed. IEEE 1998,86, 2278–2324. [CrossRef]
Wu, C.; Yang, S.; Huang, H.; Gu, X.; Sui, Y. An improved fault diagnosis method of rolling bearings based on LeNet-5. J. Vibrat.
Shock 2021,12, 55–61.
Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the Inception Architecture for Computer Vision. In
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016;
Volume 2016, pp. 2818–2826.
Jiang, G.; He, H.; Yan, J.; Xie, P. Multiscale convolutional neural networks for fault diagnosis of wind turbine gearbox. IEEE Trans.
Ind. Electron. 2018,66, 3196–3207. [CrossRef]
Chen, J.; Huang, R.; Zhao, K.; Wang, W.; Liu, L.; Li, W. Multiscale Convolutional Neural Network with Feature Alignment for
Bearing Fault Diagnosis. IEEE Trans. Instrum. Meas. 2021,70, 1–10. [CrossRef]
Zhao, M.; Kang, M.; Tang, B.; Pecht, M. Multiple wavelet coefficients fusion in deep residual networks for fault diagnosis. IEEE
Trans. Ind. Electron. 2018,66, 4696–4706. [CrossRef]
Cubillo, A.; Perinpanayagam, S.; Esperon-Miguez, M. A review of physics-based models in prognostics: Application to gears and
bearings of rotating machinery. Adv. Mech. Eng. 2016,8, 1–21. [CrossRef]
Appl. Sci. 2021,11, 8967 20 of 20
Liang, C.; Chen, C. Generalized Composite Multiscale Diversity Entropy and Its Application for Fault Diagnosis of Rolling
Bearing in Automotive Production Line. IEEE Access 2021,9, 84545–84558. [CrossRef]
Abe, K.; Matsui, G.; Tadakuma, K.; Yamano, M.; Tadakuma, R. Development of the omnidirectional transporting table based on
omnidirectional driving gear. Adv. Robot. 2020,34, 358–374. [CrossRef]
Zuo, Y.; Wang, H.; Wu, G.; Gu, Y.; Qiao, W. Research on remote state monitoring and intelligent maintenance system of CNC
machine tools. J. Eng. 2019,23, 8671–8675. [CrossRef]
53. Lee, J.D.; Tsai-Lin, C.W.; Lee, Y.C.; Liu, M.C.; Chen, L.Y. Fully automatic CNC machining production system. MATEC Web Conf.
2017,108, 04002. [CrossRef]
Wang, L.P.; Zhang, B.B.; Wu, J. Rotation accuracy evaluation of electric spindle based on least square method. Manuf. Technol.
Mach. Tool 2018,2, 54–60.
Wang, L.P.; Zhao, Q.Z.; Zhang, B.B. Accuracy of an electric spindle. Accur. Electr. Spind. Qinghua Daxue Xuebao/J. Tsinghua Univ.
2018,58, 746–751.
Anandan, K.P.; Ozdoganlar, O.B. A multi-orientation error separation technique for spindle metrology of miniature ultra-high-
speed spindles. Precis. Eng. 2016,43, 119–131. [CrossRef]
Xi, S.; Cao, H.; Chen, X. Dynamic modeling of spindle bearing system and vibration response investigation. Mech. Syst. Signal
Process. 2019,114, 486–511. [CrossRef]
Urbanek, J.; Barszcz, T.; Str ˛aczkiewicz, M.; Jablonski, A. Normalization of vibration signals generated under highly varying speed
and load with application to signal separation. Mech. Syst. Signal Process. 2017,82, 13–31. [CrossRef]
Jia, Y.; Li, G.; Dong, X.; He, K. A novel denoising method for vibration signal of hob spindle based on EEMD and grey theory.
Measurement 2021,169, 108490. [CrossRef]
Chen, D.W.; Wu, J.; Zhang, B.B.; Wang, L.P.; Liang, J.H. Load spectrum compilation for machining center spindles based on
S-shaped specimens. J. Tsinghua Univ. 2018,58, 1107–1114.
Chen, Z.; Wu, M.; Zhao, R.; Guretno, F.; Yan, R.; Li, X. Machine remaining useful life prediction via an attention-based deep
learning approach. IEEE Trans. Ind. Electron. 2020,68, 2521–2531. [CrossRef]
Li, K.; Ping, X.; Wang, H.; Chen, P.; Cao, Y. Sequential fuzzy diagnosis method for motor roller bearing in variable operating
conditions based on vibration analysis. Sensors 2013,13, 8013–8041. [CrossRef] [PubMed]
Shao, S.; McAleer, S.; Yan, R.; Baldi, P. Highly accurate machine fault diagnosis using deep transfer learning. IEEE Trans. Ind. Inf.
2018,15, 2446–2455. [CrossRef]
Han, Y.; Tang, B.; Deng, L. An enhanced convolutional neural network with enlarged receptive fields for fault diagnosis of
planetary gearboxes. Comput. Ind. 2019,107, 50–58. [CrossRef]
He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; Volume 2016, pp. 770–778.
... In this way, the aim is to simulate behaviors that could occur in a real system to which there is not access or which has not yet failures that can be used to train the models. In this line, there are many works that collect vibration signals from different experimental rotors (Cakir et al., 2021;Li et al., 2019;Liang et al., 2020;Liao et al., 2016;Pang et al., 2020;Qian et al., 2019;Satishkumar & Sugumaran, 2017;Song et al., 2021;Una et al., 2017;Wang et al., 2018;Wang, Zhang, et al., 2020;Yang, Lei, et al., 2019;Zhang, Li, Wang, et al., 2019), fans (Sampaio et al., 2019;Xu et al., 2021;Zenisek, Holzinger, & Affenzeller, 2019), centrifugal pumps (Hu et al., 2020), air compressors (Cupek et al., 2018) or industrial robots (Panicucci et al., 2020). Some works (Venkataswamy et al., 2020;Zenisek, Holzinger, & Affenzeller, 2019;Zenisek, Kronberger, et al., 2019) generate synthetic data from a mathematical approach that models the behavior of their problem, using for that specialized software in simulation. ...
... Specifically, several works use the Fast Fourier Transformation (FFT) in order to reduce the feature space (Aqueveque et al., 2021;Casoli et al., 2019;Sampaio et al., 2019;Shamayleh et al., 2020;Zschech et al., 2019). The study of Song et al. (2021) employed the Short-time Fourier Transform, which focuses on determining the sinusoidal frequency and phase content of local sections of the signal as it changes over time. Another analysis uses the Wavelet Transform which is based on applying operations that affect the time, but not the shape, so that the transformed signal contains both frequency and time information. ...
... This work also compares its proposal with both traditional ML (kNN, SVM), and DL models (Deep Belief Network [DBN], Stacked Auto-Encoder [SAE] and simple CNN). Another framework focused on failures detection from vibration data is found in (Song et al., 2021). Several DL architectures are compared achieving the best performance with the CNN-based ResNet. ...
Full-text available
Predictive maintenance is a field of study whose main objective is to optimize the timing and type of maintenance to perform on various industrial systems. This aim involves maximizing the availability time of the monitored system and minimizing the number of resources used in maintenance. Predictive maintenance is currently undergoing a revolution thanks to advances in industrial systems monitoring within the Industry 4.0 paradigm. Likewise, advances in artificial intelligence and data mining allow the processing of a great amount of data to provide more accurate and advanced predictive models. In this context, many actors have become interested in predictive maintenance research, becoming one of the most active areas of research in computing, where academia and industry converge. The objective of this paper is to conduct a systematic literature review that provides an overview of the current state of research concerning predictive maintenance from a data mining perspective. The review presents a first taxonomy that implies different phases considered in any data mining process to solve a predictive maintenance problem, relating the predictive maintenance tasks with the main data mining tasks to solve them. Finally, the paper presents significant challenges and future research directions in terms of the potential of data mining applied to predictive maintenance. This article is categorized under: Application Areas > Industry Specific Applications Technologies > Internet of Things
... New industrial paradigms, including network collaborative manufacturing, industrial big data, and cyber-physical systems, have promoted the transformation of traditional manufacturing to intelligent manufacturing [1]. High accuracy, high reliability, and high efficiency are the main development directions of rotary machinery health monitoring against the background of intelligent manufacturing [2]. ...
Full-text available
The remaining useful life (RUL) prediction of rolling bearings based on vibration signals has attracted widespread attention. It is not satisfactory to adopt information theory (such as information entropy) to realize RUL prediction for complex vibration signals. Recent research has used more deep learning methods based on the automatic extraction of feature information to replace traditional methods (such as information theory or signal processing) to obtain higher prediction accuracy. Convolutional neural networks (CNNs) based on multi-scale information extraction have demonstrated promising effectiveness. However, the existing multi-scale methods significantly increase the number of model parameters and lack efficient learning mechanisms to distinguish the importance of different scale information. To deal with the issue, the authors of this paper developed a novel feature reuse multi-scale attention residual network (FRMARNet) for the RUL prediction of rolling bearings. Firstly, a cross-channel maximum pooling layer was designed to automatically select the more important information. Secondly, a lightweight feature reuse multi-scale attention unit was developed to extract the multi-scale degradation information in the vibration signals and recalibrate the multi-scale information. Then, end-to-end mapping between the vibration signal and the RUL was established. Finally, extensive experiments were used to demonstrate that the proposed FRMARNet model can improve prediction accuracy while reducing the number of model parameters, and it outperformed other state-of-the-art methods.
... The prescription should be put to the cotton crop right away, as it is non-destructive and generates a lot of money. The different applications where smart cameras discover their versatile characteristic as well as developed a platform for calling as well as posting articles relating to our issue, according to the source cited in [16]. According to the source [17], a range of cameras are accessible for awareness and purchase for farm monitoring based on outdoor use, night vision, waterproofing, wifi, etc. ...
Systems that can infer crop state from low-cost sensing devices are required for agricultural applications such as yield prediction, precision agriculture, and autonomous harvesting. This research is focusing on agriculture monitoring system of real-time video frame processing, extraction and classification with deep learning techniques. Here the input video data has been collected from agriculture surveillance camera and transmitted through IoT module. Then the feature was extracted using probability-based lasso network regression (Pr_La-Net_Reg). Then the extracted feature has been classified using Dynamic radial functional neural network (Dy_Rad_FuNN) based Deep learning architecture. For MATLAB R2018a, an HP envy machine with Intel core i5, RAM 4 GB DDR2-RAM, Digital Camera 16 Mega Pixel, and disk space 8 GB was used to validate the system. Three benchmark datasets for video frame- based evaluation are used to conduct the suggested approach's qualitative and quantitative experimental studies.
Full-text available
A Cyber-Physical System (CPS)-enabled rehabilitation system framework for enhanced recovery rate in gait training systems is presented in this paper. Recent advancements in sensing and data analytics have paved the way for the transformation of healthcare systems from experience-based to evidence-based. To this end, this paper introduces a CPS-enabled rehabilitation system that collects, processes, and models the data from patient and rehabilitative training machines. This proposed system consists of a set of sensors to collect various physiological data as well as machine parameters. The sensors and data acquisition systems are connected to an edge computing unit that handles the data preprocessing, analytics, and results visualization. Advanced machine learning algorithms are used to analyze data from physiological data, machine parameters, and patients’ metadata to quantify each patient’s recovery progress, devise personalized treatment strategies, adjust machine parameters for optimized performance, and provide feedback regarding patient’s adherence to instructions. Moreover, the accumulation of the knowledge gathered by patients with different conditions can provide a powerful tool for better understanding the human-machine interaction and its impact on patient recovery. Such system can eventually serve as a ‘Virtual Doctor’, providing accurate feedback and personalized treatment strategies for patients.
Full-text available
Internet of Things (IoT) has rapidly developed in multidisciplinary research topics, particularly in Cyber-Physical infrastructures, such as smart-health care, transportation systems, vehicle management surveillance systems. The smart-video surveillance system has become an essential part of almost all security applications, including academic institutions. University campuses have rich video repositories comprising almost all kinds of academic and non-academic activities. Researchers have introduced many state-of-art activity recognition methods for various application domains with the availability of several activity data sets. Unfortunately, none of these data sets or methods have been developed explicitly for academia and do not cover academic activities. With the advancement of deep learning and IoT, the processing of large-scale video data has become convenient for performing various video analysis tasks. Thus, in this work, an automated deep learning-based academic activities recognition system is presented in smart-cyber infrastructure. We explore a new academic campus domain for research and proposed a novel Convolutional Neural Network (CNN) model for academic activities recognition utilizing a realistic campus dataset. The video database typically contains long, 24-hour video streams recorded by surveillance cameras installed in campus environments. The proposed model’s efficiency is tested through extensive experimentation in terms of accuracy, computation time, and memory requirement. The experimental results reveal that the proposed method attains good results with an accuracy of 98%.
Full-text available
The high-speed railway (HSR) transportation system in China has been growing rapidly during the past decade. In 2016, the total length of HSR in China has reached to 22,000 kilometers, and there are over 2,000 pairs of high speed trains operating daily. With the advancement of design and manufacturing technologies, the reliability and construction costs have been improved significantly. However, there is still great need for reduction of their operation and maintenance costs. With such incentive, a pilot project has been launched to develop a prognostics and health management system for rolling stock to transform the maintenance paradigm from preventive to predictive maintenance. Considering the high task variety and big data environment in HSR real-time monitoring system, a cyberphysical system (CPS) architecture is proposed as the framework for its PHM system. This paper reviews the needs of predictive maintenance for the HSR system, and then present a concept design of the CPS-enabled smart operation and maintenance system.
Full-text available
This paper considers the entropy based feature extraction method for the fault diagnosis of rolling bearings in automobile production line, where the fault information is difficult to identify due to the strong nonlinear and non-stationary characteristics of the fault vibration signals. In our work, a novel entropy based method called generalized composite multiscale diversity entropy (GCMDiEn) is developed. This method can effectively track the inside pattern changes of the time series by the description of cosine similarity between adjacent orbits. Unlike most of the existing entropy based results which concentrate on the static orderliness, we analyze the dynamic complexity characteristics of the arbitrary time series. Moreover, compared with the multiscale diversity entropy, GCMDiEn calculates all coarse-grained time series entropy value with the same scale and extends the first-order moment to second-order moment to mitigate the shortcomings of the short time series instability and losing part of the frequency region information. This paper proposed a new rolling bearing fault diagnosis framework which combined the GCMDiEn with the empirical wavelet transform (EWT), Laplacian score (LS), and particle swarm optimization-based support vector machine (PSO-SVM). Finally, the simulation results show the superiority of the GCMDiEn method over the multiscale diversity entropy method. The proposed framework has a higher fault recognition rate (99.38%) than the existing methods.
Full-text available
We focus on a small object detection model for digital twin, aiming to realize the dynamic synchronization between a physical manufacturing system and its virtual representation. Three significant elements, including equipment, product, and operator, are considered as the basic environmental parameters to represent and estimate the dynamic characteristics and real-time changes in building a generic DT system of smart manufacturing workshop. A hybrid deep neural network model based on the integration of MobileNet-v2, YOLOv4, and Openpose, is constructed to identify the real-time status from physical manufacturing environment to virtual space. A learning algorithm is then developed to realize the multi-type small object detection, in order to facilitate the modeling, monitoring, and optimizing of the whole smart manufacturing process in DT system. Experiments and evaluations using a real-world dataset demonstrate the effectiveness and usefulness of our proposed method which can achieve a higher detection accuracy for digital twinning in smart manufacturing.
Full-text available
Condition monitoring of industrial equipment, combined with machine learning algorithms, may significantly improve maintenance activities on modern cyber-physical production systems. However, data of proper quality and of adequate quantity, modeling both good operational conditions as well as abnormal situations throughout the operational lifecycle, are required. Nevertheless, this is difficult to acquire in a non-destructive approach. In this context, this study investigates an approach to enable a transition from preventive maintenance activities, that are scheduled at predetermined time intervals, into predictive ones. In order to enable such approaches in a cyber-physical production system, a deep learning algorithm is used, allowing for maintenance activities to be planned according to the actual operational status of the machine and not in advance. An autoencoder-based methodology is employed for classifying real-world machine and sensor data, into a set of condition-related labels. Real-world data collected from manufacturing operations are used for training and testing a prototype implementation of Long Short-Term Memory autoencoders for estimating the remaining useful life of the monitored equipment. Finally, the proposed approach is evaluated in a use case related to a steel industry production process.
Full-text available
Predictive maintenance of production lines is important to early detect possible defects and thus identify and apply the required maintenance activities to avoid possible breakdowns. An important concern in predictive maintenance is the prediction of remaining useful life (RUL), which is an estimate of the number of remaining years that a component in a production line is estimated to be able to function in accordance with its intended purpose before warranting replacement. In this study, we propose a novel machine learning-based approach for automating the prediction of the failure of equipment in continuous production lines. The proposed model applies normalization and principle component analysis during the pre-processing stage, utilizes interpolation, uses grid search for parameter optimization, and is built with multilayer perceptron neural network (MLP) machine learning algorithm. We have evaluated the approach using a case study research to predict the RUL of engines on NASA turbo engine datasets. Experimental results demonstrate that the performance of our proposed model is effective in predicting the RUL of turbo engines and likewise substantially enhances predictive maintenance results.
In recent years, deep learning methods, especially convolutional neural network (CNN), have received extensive attentions and applications in fault diagnosis. However, recent studies have shown that the shift-invariance of CNN is not good enough, resulting in fragile feature extraction and sharp reduction in model performance when the shift occurs in the input. To improve the shift-invariance of CNN, considering the periodic characteristics of vibration signals, a multiscale CNN with feature alignment (MSCNN-FA) is proposed for bearing fault diagnosis under different working conditions. First, by analyzing the operating principles of the convolutional layer and pooling layer, a feature alignment module including single-stride convolution layer, adaptive max-pooling layer, and global average pooling layer is designed to obtain aligned features. Next, to extract shift-invariant robust features from vibration signals, a multiscale convolution strategy is utilized, and a feature-aligned multiscale feature extractor is constructed. Finally, a classifier composed of fully connected (FC) layers is constructed for bearing fault diagnosis. The effectiveness of the method is verified by a rolling bearing experiment, which outperforms other related existing CNN-based methods in terms of diagnosis accuracy and feature robustness.
In this study, a data driven predictive maintenance system was developed for production lines in manufacturing. By utilizing the data generated from IoT sensors in real-time, the system aims to detect signals for potential failures before they occur by using machine learning methods. Consequently, it helps address the issues by notifying operators early such that preventive actions can be taken prior to a production stop. In current study, the effectiveness of the system was also assessed using real-world manufacturing system IoT data. The evaluation results indicated that the predictive maintenance system was successful in identifying the indicators of potential failures and it can help prevent some production stops from happening. The findings of comparative evaluations of machine learning algorithms indicated that models of Random Forest, a bagging ensemble algorithm, and XGBoost, a boosting method, appeared to outperform the individual algorithms in the assessment. The best performing machine learning models in this study have been integrated into the production system in the factory.