Content uploaded by Mohamed Benbouzid
Author content
All content in this area was uploaded by Mohamed Benbouzid on Jan 20, 2023
Content may be subject to copyright.
Electronics 2023, 12, 527. https://doi.org/10.3390/electronics12030527 www.mdpi.com/journal/electronics
Perspective
Quo Vadis Machine Learning-Based Systems Condition
Prognosis?—A Perspective
Mohamed Benbouzid 1,* and Tarek Berghout 2
1 Institut de Recherche Dupuy de Lôme (UMR CNRS 6027), University of Brest, 29238 Brest, France
2 Laboratory of Automation and Manufacturing Engineering, University of Batna 2, Batna 05000, Algeria
* Correspondence: mohamed.benbouzid@univ-brest.fr
Abstract: Data-driven prognostics and health management (PHM) is key to increasing the produc-
tivity of industrial processes through accurate maintenance planning. The increasing complexity of
the systems themselves, in addition to cyber-physical connectivity, has brought too many chal-
lenges for the discipline. As a result, data complexity challenges have been pushed back to include
more decentralized learning challenges. In this context, this perspective paper describes these
challenges and provides future directions based on a relevant state-of-the-art review.
Keywords: prognostics and health management; remaining useful life; machine learning;
data-driven; deep learning
1. Mains Challenges
Process health prognosis is essential to reducing downtime during operating condi-
tions, specifically time-to-repair, and increasing the productivity of operating systems
through accurate planning of condition-based maintenance tasks, whether the repair
process is scheduled under working or non-working conditions. Remaining useful life
(RUL), which is the expected time to complete system failure, is the primary focus in the
study of system deterioration, aging and damage propagation [1]. Determining RUL
requires run-to-failure samples labeled with the real RUL time, which is often difficult to
obtain. This being the case, the state of health (SOH) will be assessed instead via health
index (HI) and estimating the health stage (HS) [2]. Data-driven methods, especially
machine learning, are becoming dominant in the field due to the increasing complexity
issues of physical modeling [3]. As a result, increasing system complexity besides ad-
vanced cyber-physical connectivity means that machine learning will also be facing
challenges related to modeling complexity, decentralized learning, privacy and security
as illustrated by Figure 1 [4].
PHM challenges for RUL/(HI & HS)
predictions
Data availability Decentralized training Security
Data complexity
Precision
Data dynamism
and online decision-making
Uncertainty
Statistical heterogeneity
Systems heterogeneity
Communication efficiency
Data confidentiality, Integrity, and
availability
Privacy
Data is subject to conflict of intrests
Figure 1. Main challenges for RUL/(HI and HS) predictions.
Citation: Benbouzid, M.;
Berghout, T. Quo Vadis Machine
Learning-Based Systems Condition
Prognosis?—A Perspective.
Electronics 2023, 12, 527. https://doi.org
/10.3390/electronics12030527
Academic Editors: Rashid Mehmood
and Gwanggil Jeon
Received: 13 December 2022
Revised: 6 January 2023
Accepted: 18 January 2023
Published: 19 January 2023
Copyright: © 2023 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license
(https://creativecommons.org/license
s/by/4.0/).
Electronics 2023, 12, 527 2 of 7
1.1. Modeling Complexity
RUL modeling with machine learning faces many challenges in providing necessary
monitoring systems of real-world conditions in terms of generalizing the prediction
model to unseen samples for the same system or new similar systems to the studied one.
This poor generalization is the result of several facts related to data availability, data
complexity, lack of precision, uncertainty of predictions especially for time long-term
forecasts, data dynamism and online decision-making.
Data availability: Due to the lack of labeled datasets with real RUL timing, many
available works are moving towards accelerated life testing [5,6]. Accelerated degrada-
tion experiments provide real-world-like conditions, but lack some real run-to-failure
patterns. This is because these experiments are subject to even harsher environments
than real ones and recorded samples may suffer from a higher level of non-stationarity.
Lack of patterns is the main reason for the poor generalization of training models over
unseen samples driven by systems operating in real conditions and not accelerated test-
ing.
Data complexity: It is mentioned that recorded samples from accelerated life ex-
periments resemble incomplete data patterns containing samples with a higher level of
complexity due to harsh non-stationary conditions. In addition, even if this data is rec-
orded based on real degradation experiments with real run-to-failure time, continuous
change in working conditions due to internal and external constraints (environmental or
system-related conditions) drives data with a higher level of dynamicity, massive and
rapid change, and produces a very complex feature space that is difficult to manage even
with deep representations [1].
Lack of precision: It is undeniable that many works devoted their efforts in estimate
model approximate accuracy to well-known metrics such as root mean squared errors
(RMSE) and similar metrics. However, it is worth mentioning that prognosis models are
not merely a matter of approximation. Indeed, in general, for long-term predictions, the
distances between the predicted samples and the desired responses become more distant
as the predictions become longer. In this context, it seems that prognosis is a matter of
early and late prediction distribution more than of approximation. In this context, preci-
sion analysis is considered mandatory to assess the accuracy of the time-to-predict fail-
ures.
Precision analysis requires projecting predicted samples into a specific probability
distribution function (PDF) that helps determine the amount of early and late predictions
as well as their dispersion from the reference value that is assumed to be solutions opti-
mal [7]. For example, Figure 2 shows three different cases of predicted RUL/HI, where
the predictions are early, late and good, respectively. It should be mentioned that the
data used for illustration, in this case, are related to publicly available linear and cyclic
degradation trends of Li-ion batteries [8]. Such a classification could be driven by many
approaches, including human-centric approaches and expert backgrounds and assess-
ments of maintenance resource consumption and other potential effects of failures (e.g.,
damage to reputation and life, financial loss, etc.). Therefore, the decision of whether
predictions are good or bad depends on the nature of the system and the predictions. For
example, if the system is safety-critical and a failure is completely prohibited, accuracy
should be maximized as much as possible. However, deciding what the exact threshold is
for this sub-classification depends on the optimal threshold selection standards defined
for a specific system [9]. For Figure 2a, prediction errors are scattered far from the center
in both early and late cases. Similarly, Figure 2b gives more precision while late predic-
tions dominate. Conversely, Figure 2c shows acceptable prediction results exhibiting less
scatter and more concentration towards the preferred reference value.
Electronics 2023, 12, 527 3 of 7
Figure 2. Example of distributing RUL/HI predictions under different precision levels: (a) bad
predictions. Predictions are far from the center. Early and late predictions are equally distributed;
(b) acceptable predictions. The model is even more precise than in (a). It is an early predictor be-
cause late predictions are almost neglected; (c) good predictions. More precision is provided in this
case. More concentration towards the center and less dispersion toward early and late predictions.
The distinctive feature of prognostic predictions is that early and late predictions are
two different issues and not just a distance from the exact prediction. Indeed, early pre-
dictions consume maintenance resources, while late predictions are too harmful and can
lead to catastrophic situations and loss of life. Therefore, it is very important to consider
penalizing their distributions differently in the PDF function to minimize late predictions
as much as possible.
In this case, the main challenge facing machine learning models is to provide a
larger concentration of predictions toward the center of the PDF while also balancing
(i.e., providing a sort of symmetry) early and late predictions in terms of dispersion to
keep the maintenance decision as accurate as possible.
Uncertainty: Uncertainty in RUL/HI prediction models is the result of many factors, in-
cluding hyperparameters, model structure, approximations, algorithmic and experimental
conditions (conditions when aging experiments are made). Uncertainty quantification is
necessary to reduce the number of uncertainties in prediction as well as maintenance related
to decision-making. Two main categories, namely Bayesian techniques and ensemble learn-
ing, are widely investigated [10,11]. The challenge is that existing approaches suffer from
some problems. Many of them are computationally prohibitive and can be difficult to cali-
brate, which can lead to high sampling complexity and may also require major changes in
model architecture and training. Figure 3 is introduced to showcase an example of uncer-
tainty quantification of RUL predictions with a 99% confidence interval (CI). In this particu-
lar example, predicted samples that fall outside the CI are considered uncertain. The measure
of uncertainty in this case is the ratio of samples outside the CI over those inside the CI.
Figure 3. Uncertainty quantification of RUL predictions with a 99% confidence interval.
RUL
Electronics 2023, 12, 527 4 of 7
Data dynamism and online decision-making: RUL prediction models have diffi-
culty addressing real online adaptive learning, such as reinforcement learning, in the
context of data availability and the difficulty of obtaining such experience in real-world
scenarios due to possible mishaps. In addition, simulation is difficult to approach due to
the complexity of physical modeling. Thus, most of the studies on this topic are online
models based on offline data already collected. These simulations do not address the re-
ality of condition monitoring but they remain theoretically possible [12].
1.2. Decentralized Training
Recent cyberphysical connectivity and decentralized architectures of industrial
processes make it difficult to achieve global generalization of machine learning models
due to too many challenges such as statistical heterogeneity, systems heterogeneity and
communication efficiency in smart infrastructures [13].
Statistical heterogeneity refers to data distribution. Generally speaking, data come
from different devices with different working conditions. This means that the data may
be non-independent and identically distributed (Non-IDD). In this context, it is therefore
very challenging to have a model that can handle this type of collaborative training
without experiencing a performance drop. As a result, differences in devices and con-
nectivity methods lead also to differences in data characteristics. This makes it difficult to
account for these variations in each training run. Regarding communication efficiency,
the challenges remain communication overhead, especially for mobile devices with lim-
ited resources (e.g., battery-powered devices). Synchronization between these devices
must also be considered when simulating machine learning models due to the nature of
communications in smart infrastructure networks.
1.3. Security and Privacy
In smart infrastructures, decentralized learning and data sharing do not satisfy data
privacy conflicts of interest. The main challenges in this case are thus to ensure decen-
tralized training under less data sharing. In addition, connectivity makes the immunity of
the entire smart infrastructure prone to cyberthreats, which leads to many consequences
such as breach of confidentiality, integrity and availability of data.
2. What Do We Have to Work Towards?
Advances in PHM should not be limited to the performance of the prognosis model.
Indeed, for recent smart infrastructure technologies, connectivity, privacy and security
must be considered. Therefore, prognosis models should be improved in the context of
modeling complexity, involving federated learning and secured learning process and
information sharing. In other words, what we should design is: “an accurate, precise, se-
cure, and online adaptive decentralized federated learning system” (see last statement from
Section 1.1 of [1]).
2.1. Reducing Complexity
In the context of reducing modeling complexity while keeping generalization capa-
bility, data generation (e.g., generative models), reducing model architecture, precision
analysis, uncertainty quantification and adaptive learning are very important.
Data generation: Generative models, such as autoencoders, including denoising
autoencoders [14] and more specifically generative adversarial networks (GANs) [15], are
very popular in this field. Denoising autoencoders allow producing robust meaningful
representation by training learning models to produce accurate representation under the
presence of data corruption. Unlike autoencoders, which are completely unsupervised
networks, GANs are used to generate data from different random noises based on two
parts, namely the generator, which generates new samples, and the discriminator, which
classifies samples as real or false. Generative models are very important in augmenting
Electronics 2023, 12, 527 5 of 7
data by generating new samples that are statistically similar to the training data. These
samples are synthetic instances of data very similar to the real ones. In the context of
PHM, generative models will help in filling the gap of poor generalization related to the
absence of degradation patterns.
In addition, transfer learning also will help generate meaningful representations
from different source domains and working conditions to fill in the gap of lack of sam-
ples [12].
Improving model architecture: Model architecture increases complexity in terms of
computational cost; therefore, more effort should be devoted to developing less complex
architectures while keeping accuracy the same as deep networks. In this context, least
squares variants and the Kalman filter can be considered when training deep learning
models [16–18].
Precision analysis: To help the prediction errors to surround the desired PDF value,
more effort should be devoted to improving learning models in a prognosis context and
not only to accurate approximation. Therefore, the following solutions can be considered:
(i) Defining the appropriate loss function to be minimized during the training process,
such as the same PDF function of the precision; (ii) Learning from labels autoencoding
has also proven its capability to reshape the predicted responses and can contribute to a
better fit of the desired results [19].
Uncertainty quantifications: Uncertainty quantification is essential in reducing
prediction models’ uncertainty. Accordingly, more efforts should be focused on analyz-
ing the uncertainty of predictions under different algorithmic architectures and data
complexity. In this context, the following research areas can be further explored (see [10],
§ 7.1.2. “Future directions based on applications”): (i) Use of meta-reinforcement learning
models for better decision-making with a better certainty; (ii) Approximate Bayesian in-
ference in sequential decision-making applications should be used as an internal proce-
dure of larger methods; (iii) Density Filtering Techniques (ADF); (iii) Ensemble-based
sampling; and (iv) Quantification of uncertainties for multi-agent systems.
Adaptive online learning: Reinforcement learning features give the most interesting
insights in modern online learning, which allows agents to learn in an interactive envi-
ronment by trying and correcting their mistakes; it makes predictions based on the
feedback of its actions. Additional efforts should be made in a PHM context, as there are
only a few contributions of RUL model reconstruction, [20–22].
2.2. Federated Learning
Federated learning is the available solution for data privacy in decentralized learn-
ing. Its core idea is to train a generalized and global learning model without data sharing
[23]. However, works on federated learning are scarce in the context of PHM. Only a few
papers are available on this topic [24,25]. Federated learning faces decentralized learning
challenges besides privacy concerns related to data sharing. In this context, more em-
phasis should be placed on considering federated learning for PHM.
2.3. Security
Machine learning-based prognosis models, which are supposed to be federated
learning ones, are subject to external threats. In this context, there is almost a complete
lack of studies related to PHM while it is the most important area in the field of condition
monitoring, especially cyberphysical connectivity and internet of things technologies. As
such, more investigation efforts should be devoted to this topic in the future.
3. Conclusions
Most machine learning-based systems condition prognosis studies available in the
literature deal with model performance while ignoring intelligent infrastructures’ most
important factors, namely decentralized learning, privacy and security of the learning
Electronics 2023, 12, 527 6 of 7
models. In this context, this paper briefly provided readers with the most important
challenges faced by data-driven PHM and specifically suggested future guidance to ad-
dress these challenges. As a result, challenges range from data availability and complex-
ity and drift to statistical heterogeneity, system heterogeneity, communications efficien-
cy, privacy and security of cyber-physical connectivity.
Author Contributions: Conceptualization, M.B. and T.B.; methodology, M.B. and T.B.; software,
M.B. and T.B.; validation, M.B. and T.B.; formal analysis, M.B. and T.B.; investigation, M.B. and
T.B.; resources, M.B. and T.B.; data curation, M.B. and T.B.; writing—original draft preparation,
M.B. and T.B.; writing—review and editing, M.B. and T.B.; visualization, M.B. and T.B.; All authors
have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Data Availability Statement: Not Applicable
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Berghout, T.; Mouss, M.-D.; Mouss, L.; Benbouzid, M. ProgNet: A Transferable Deep Network for Aircraft Engine Damage
Propagation Prognosis under Real Flight Conditions. Aerospace 2022, 10, 10. https://doi.org/10.3390/aerospace10010010.
2. Lei, Y.; Li, N.; Guo, L.; Li, N.; Yan, T.; Lin, J. Machinery health prognostics: A systematic review from data acquisition to RUL
prediction. Mech. Syst. Signal Process. 2018, 104, 799–834. https://doi.org/10.1016/j.ymssp.2017.11.016.
3. Berghout, T.; Benbouzid, M. A Systematic Guide for Predicting Remaining Useful Life with Machine Learning. Electronics 2022,
11, 1125. https://doi.org/10.3390/electronics11071125.
4. Berghout, T.; Benbouzid, M.; Muyeen, S.M. Machine Learning for Cybersecurity in Smart Grids: A Comprehensive
Review-based Study on Methods, Solutions, and Prospects. Int. J. Crit. Infrastruct. Prot. 2022, 38, 100547.
https://doi.org/10.1016/j.ijcip.2022.100547.
5. Nectoux, P.; Gouriveau, R.; Medjaher, K.; Ramasso, E.; Chebel-Morello, B.; Zerhouni, N.; Varnier, C. PRONOSTIA : An
experimental platform for bearings accelerated degradation tests. In Proceedings of the IEEE International Conference on
Prognostics and Health Management, PHM’12; Denver, CO, USA, 18—21 June 2012; pp. 1–8.
6. Qiu, H.; Lee, J.; Lin, J.; Yu, G. Wavelet filter-based weak signature detection method and its application on rolling element
bearing prognostics. J. Sound Vib. 2006, 289, 1066–1090. https://doi.org/10.1016/j.jsv.2005.03.007.
7. Gouriveau, R.; Medjaher, K.; Ramasso, E.; Zerhouni, N. PHM—Prognostics and health management De la surveillance au
pronostic de défaillances de systèmes complexes. Tech. Ing. Fonct. Strat. Maint. 2013, 148625958.
8. Saha, B.; Goebel, K. Battery Data Set. NASA AMES Prognostics Center of Excellence Data Set Repository: Washington, DC,
USA, 2007.
9. Juričić, Ð.; Kocare, N.; Boškoski, P. On Optimal Threshold Selection for Condition Monitoring. In Advances in Condition
Monitoring of Machinery in Non-Stationary Operations, Proceedings of the Fourth International Conference on Condition Monitoring of
Machinery in Non-Stationary Operations, CMMNO'2014, Lyon, France, 15–17 December 2014; Chaari, F., Zimroz, R., Bartelmus, W.,
Haddar, M., Eds.; Springer, Cham: Cham, Switzerland, 2016; Volume 4, pp. 237–249.
https://doi.org/10.1007/978-3-319-20463-5_18.
10. Abdar, M.; Pourpanah, F.; Hussain, S.; Rezazadegan, D.; Liu, L.; Ghavamzadeh, M.; Fieguth, P.; Cao, X.; Khosravi, A.; Acharya,
U.R.; et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Inf. Fusion 2021,
76, 243–297. https://doi.org/10.1016/j.inffus.2021.05.008.
11. Alves, D.S.; Daniel, G.B.; de Castro, H.F.; Machado, T.H.; Cavalca, K.L.; Gecgel, O.; Dias, J.P.; Ekwaro-Osire, S. Uncertainty
quantification in deep convolutional neural network diagnostics of journal bearings with ovalization fault. Mech. Mach. Theory
2020, 149, 103835. https://doi.org/10.1016/j.mechmachtheory.2020.103835.
12. Berghout, T.; Mouss, L.-H.; Bentrcia, T.; Benbouzid, M. A Semi-Supervised Deep Transfer Learning Approach for
Rolling-Element Bearing Remaining Useful Life Prediction. IEEE Trans. Energy Convers. 2022, 37, 1200–1210.
https://doi.org/10.1109/TEC.2021.3116423.
13. Yang, Q.; Liu, Y.; Cheng, Y.; Kang, Y.; Chen, T.; Yu, H. Federated Learning; Synthesis Lectures on Artificial Intelligence and
Machine Learning; Morgan & Claypool Publishers: San Rafael, CA, USA, 2020; Volume 13, pp. 1–207.
https://doi.org/10.2200/S00960ED2V01Y201910AIM043.
14. Berghout, T.; Mouss, L.H.; Kadri, O.; Saïdi, L.; Benbouzid, M. Aircraft engines Remaining Useful Life prediction with an
adaptive denoising online sequential Extreme Learning Machine. Eng. Appl. Artif. Intell. 2020, 96, 103936.
https://doi.org/10.1016/j.engappai.2020.103936.
15. Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative
Adversarial Networks. arXiv 2014, arXiv:1406.2661.
Electronics 2023, 12, 527 7 of 7
16. Berghout, T.; Bentrcia, T.; Ferrag, M.A.; Benbouzid, M. A Heterogeneous Federated Transfer Learning Approach with Extreme
Aggregation and Speed. Mathematics 2022, 10, 3528. https://doi.org/10.3390/math10193528.
17. Ma, X.; Wen, C.; Wen, T. An Asynchronous and Real-time Update Paradigm of Federated Learning Diagnosisfor Fault. IEEE
Trans. Ind. Inform. 2021, 3203, 8531–8540. https://doi.org/10.1109/TII.2021.3063482.
18. Xue, M.A.; Chenglin, W.E.N. An Asynchronous Quasi-Cloud/Edge/Client Collaborative Federated Learning Mechanism for
Fault Diagnosis. Chin. J. Electron. 2021, 30, 969–977. https://doi.org/10.1049/cje.2021.07.008.
19. Berghout, T.; Benbouzid, M. EL-NAHL: Exploring labels autoencoding in augmented hidden layers of feedforward neural
networks for cybersecurity in smart grids. Reliab. Eng. Syst. Saf. 2022, 226, 108680. https://doi.org/10.1016/j.ress.2022.108680.
20. Bellani, L.; Compare, M.; Baraldi, P.; Zio, E. Towards Developing a Novel Framework for Practical PHM: A Sequential Decision
Problem solved by Reinforcement Learning and Artificial Neural Networks. Int. J. Progn. Heal. Manag. 2019, 31, 211051503.
21. Jha, M.S.; Weber, P.; Theilliol, D.; Ponsart, J.C.; Maquin, D. A reinforcement learning approach to health aware control strategy.
In Proceedings of the 27th Mediterranean Conference on Control and Automation, Akko, Israel, 1–4 July 2019; pp. 171–176.
https://doi.org/10.1109/MED.2019.8798548.
22. Skordilis, E.; Moghaddass, R. A deep reinforcement learning approach for real-time sensor-driven decision making and
predictive analytics. Comput. Ind. Eng. 2020, 147, 106600. https://doi.org/10.1016/j.cie.2020.106600.
23. McMahan, H.B.; Moore, E.; Ramage, D.; Hampson, S.; Arcas, B.A. Communication-Efficient Learning of Deep Networks from
Decentralized Data. arXiv 2016, arXiv:1602.05629.
24. Rosero, R.L.; Silva, C.; Ribeiro, B. Remaining Useful Life Estimation in Aircraft Components with Federated Learning. In
Proceedings of the 5th European Conference of the PHM Society 2020, Virtual Event, 27–21 July 2020; PMH Society: Portland,
OR, USA, 2020; Volume 5, pp. 1–8.
25. Dhada, M.; Jain, A.K.; Parlikad, A.K. Empirical convergence analysis of federated averaging for failure prognosis. IFAC
PapersOnLine 2020, 53, 360–365. https://doi.org/10.1016/j.ifacol.2020.11.058.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury
to people or property resulting from any ideas, methods, instructions or products referred to in the content.
