Content uploaded by Dmitriy Fradkin
Author content
All content in this area was uploaded by Dmitriy Fradkin on Apr 20, 2015
Content may be subject to copyright.
A preview of the PDF is not available
Log-based predictive maintenance
Abstract and Figures
Success of manufacturing companies largely depends on reliability of their products. Scheduled maintenance is widely used to ensure that equipment is operating correctly so as to avoid unexpected breakdowns. Such maintenance is often carried out separately for every component, based on its usage or simply on some fixed schedule. However, scheduled maintenance is labor-intensive and ineffective in identifying problems that develop between technician's visits. Unforeseen failures still frequently occur. In contrast, predictive maintenance techniques help determine the condition of in-service equipment in order to predict when and what repairs should be performed. The main goal of predictive maintenance is to enable pro-active scheduling of corrective work, and thus prevent unexpected equipment failures.
Figures - uploaded by Dmitriy Fradkin
Author content
All figure content in this area was uploaded by Dmitriy Fradkin
Content may be subject to copyright.
... Production downtime is one of the most significant contributors to production inefficiency (Liu et al., 2012), resulting in lost profit. While planned production downtime occurs, for example, for scheduled maintenance based on regular schedules or predictive models (Boudjelida, 2019;Khatab, 2018;Liu et al., 2019;Sipos et al., 2014), unforeseen production stops are a result of failures in the production process, e.g., misconfiguration of a machine, intervention of a worker, or defective raw material. In the case of an unforeseen production stop, direct action from production workers is required to resolve the issue promptly and limit the financial loss (Mobley, 2002). ...
... This information is commonly stored while monitoring the production process and the product quality to detect defects. Harnessing this log data vault beyond monitoring opens the opportunity for a datadriven examination of predictive maintenance or automatic Root Cause Analysis (RCA), e.g., for increasing production efficiency, reducing defects, or decreasing unforeseen production downtime (Davis et al., 2015;Gutschi et al., 2019;Li et al., 2020;Nikula et al., 2019;Rodríguez et al., 2019;Sipos et al., 2014;Sun et al., 2021;Wang et al., 2017;Wuest et al., 2016). In this context, random forests are used to derive models for predicting general machine breakdown (Gutschi et al., 2019;Wang et al., 2017), or they are used to select possible causes of faults within a manufacturing process (Chien & Chuang, 2014). ...
... Furthermore, in our work, we elucidate data preprocessing and the application of domain knowledge during the causal structure learning process. Beyond causal structure learning, several research works investigate the use of log data for predictive maintenance (Gutschi et al., 2019;Sipos et al., 2014;Wang et al., 2017). In this context, the preprocessing of log data follows similar steps to aggregate data within time windows and select relevant features for model creation. ...
In discrete manufacturing, the knowledge about causal relationships makes it possible to avoid unforeseen production downtimes by identifying their root causes. Learning causal structures from real-world settings remains challenging due to high-dimensional data, a mix of discrete and continuous variables, and requirements for preprocessing log data under the causal perspective. In our work, we address these challenges proposing a process for causal reasoning based on raw machine log data from production monitoring. Within this process, we define a set of transformation rules to extract independent and identically distributed observations. Further, we incorporate a variable selection step to handle high-dimensionality and a discretization step to include continuous variables. We enrich a commonly used causal structure learning algorithm with domain-related orientation rules, which provides a basis for causal reasoning. We demonstrate the process on a real-world dataset from a globally operating precision mechanical engineering company. The dataset contains over 40 million log data entries from production monitoring of a single machine. In this context, we determine the causal structures embedded in operational processes. Further, we examine causal effects to support machine operators in avoiding unforeseen production stops, i.e., by detaining machine operators from drawing false conclusions on impacting factors of unforeseen production stops based on experience.
... The method has been tested on both synthetic data and real-life TwoLeadECG dataset. The performance of alert generation systems in the predictive maintenance domain has been assessed by Sipos et al (Sipos et al, 2014). The authors are predicting equipment failures by mining the data from event logs of medical devices. ...
Money laundering is a global threat to society nowadays. Governments and governmental authorities fight money laundering, in part, by regulating banks and financial institutions. Financial institutions, in turn, are obligated to implement mechanisms to prevent money laundering. Usually, these prevention mechanisms include automated monitoring systems. In this paper, we propose an Anti-Money Laundering monitoring system based on machine learning techniques, which addresses three requirements: (i) generating accurate and non-redundant alerts; (ii) generating timely alerts; and (iii) associating explanations and risk estimates to each alert. The first requirement is addressed by training a machine learning classification model on different snapshots of customers’ history and then feeding the classification scores generated by this model into an alerting policy designed to prevent redundant alerts. The second requirement is addressed via custom metrics for assessing the performance of classification models, which take into account the timeliness requirements that arise in the Anti-Money Laundering domain. Finally, the third requirement is addressed by applying a method for class probability calibration and by an interpretability layer based on Shapley values. The proposed monitoring system has been designed based on requirements provided by an investigation unit at a financial institution and evaluated using real-life data as well as multiple rounds of feedback from specialized subject-matter experts.
... Each application displays its own distinctive characteristics that have great effect on the design of the corresponding algorithm. Some application domains where failure prediction is important and prediction methods have been successfully applied are web servers [5], medical equipment [6,7], and Hard drives [8,9]. As there is a great development in the processing of big data, IOT, and storage, when the domain of expertise is not available the best approach is the Data-driven [10,11]. ...
... The authors used multiple instances learning approaches to structure the model as a regression problem to approximate the risk of a target event occurring. Sipos et al. [36] developed a data-driven approach based on multiple-instance learning for predicting equipment failures. Evgeny [10] developed a data-driven rare failure prediction model using event matching for aerospace applications. ...
The use of aircraft operation logs to develop a data-driven model to predict probable failures that could cause interruption poses many challenges and has yet to be fully explored. Given that aircraft is high-integrity assets, failures are exceedingly rare. Hence, the distribution of relevant log data containing prior signs will be heavily skewed towards the typical (healthy) scenario. Thus, this study presents a novel deep learning technique based on the auto-encoder and bidirectional gated recurrent unit networks to handle extremely rare failure predictions in aircraft predictive maintenance modelling. The auto-encoder is modified and trained to detect rare failures, and the result from the auto-encoder is fed into the convolutional bidirectional gated recurrent unit network to predict the next occurrence of failure. The proposed network architecture with the rescaled focal loss addresses the imbalance problem during model training. The effectiveness of the proposed method is evaluated using real-world test cases of log-based warning and failure messages obtained from the fleet database of aircraft central maintenance system records. The proposed model is compared to other similar deep learning approaches. The results indicated an 18% increase in precision, a 5% increase in recall, and a 10% increase in G-mean values. It also demonstrates reliability in anticipating rare failures within a predetermined, meaningful time frame.
... Chen et al. [50] predicted and diagnosed the failures by the decision tree classifiers, which does not have the most competitive classification performance, but the important comprehensibility. Sipos et al. [48] proposed an approach of equipment failure prediction based on multi-instance learning, which meet the characteristics of the normal and abnormal equipment logs. Zhou et al. [45] proposed MEPFL for failure diagnosis of the micro-service system, adopted random forest, KNN, and Multilayer Perceptron as prediction models, set up four prediction tasks for potential error diagnosis. ...
With the development of Artificial Intelligence (AI), Internet of Things (IoT), cloud computing, new-generation mobile communication, etc., digital transformation is changing the technical architecture of IT systems. It brings more requirements for performance and reliability. The traditional human-dependent development and maintenance methods are overwhelmed, and need to transform to Artificial Intelligence for IT Operations (AIOps). As one of the most useful data resources in IT system, the log plays an important role in AIOps. There are many research on enhancing log quality, analyzing log structure, understanding system behavior, helping users to mine the effective information in logs. Based on the characteristics of logs and different strategies, this paper reviews and categorizes the existing works around the three key processes in the log processing framework of log enhancement, log parsing, and log analysis in academia, and establishes evaluation indicators for comparison and summary. Finally, we discussed the potential directions and future development trends.
Prediction accuracy plays an important role in order to improve the maintenance of machines and plants. In this context, the extension of the data-driven prediction models by relevant information from repair and maintenance reports can help to detect anomalies and to differentiate between healthy and abnormal operating states of machines and plants. Up to now, determining a plant’s operating status based only on sensor data is quite challenging. The present paper concentrates on showing the applications of using maintenance reports to aid sensory data analysis by means of two approaches: manual keywords and using NLP (Natural Language Processing) to create clusters. The results show that analyzing repair and maintenance reports can optimize predictive maintenance and support the development of failure catalogues, fault lists, time interval definitions, and work instructions or trouble shooting guides.
The Fourth Industrial Revolution, under the name of Industry 4.0, focuses on obtaining and using data to facilitate decision-making and thus achieve a competitive advantage. Industry 4.0 is about smart factories. For this, a series of technologies have emerged that communicate the physical and the virtual world, including Internet of Things, Big Data, and Artificial Intelligence. These technologies can be applied in many areas of the industry such as production, manufacturing, quality, logistics, maintenance, or security to improve the optimization of the production capacity or the control and monitoring of the production process. An important area of application is maintenance. Predictive maintenance is focused on monitoring the performance and condition of equipment during normal operation to reduce the likelihood of failures with the help of data-driven techniques. This chapter aims to explore the possibilities of using artificial intelligence to optimize the maintenance of the machinery and equipment components so that product costs are reduced.
In recent scientific work, a classification-based approach to the machinery prognostics problem has been elaborated as an alternative to the Remaining Useful Life approaches. The classification-based approaches rely on a prediction horizon parameter, to which the model quality is sensitive. However, existing studies do not provide any means of determining this critical parameter. Instead, they rely on assumptions. We argue that the prediction horizon should be learned from data in order to overcome the challenges of its uncertainty. We propose a heuristic algorithm to learn the prediction horizon from data, as the first of its kind in the literature. We test its effectiveness with an ablation study based on a rich set of data. The results indicate a statistically significant improvement in model quality. This in turn increases the usability and generalizability of classification-based failure prediction approaches in the industry.
This paper presents two new formulations of multiple-instance learning as a maximum margin problem. The proposed extensions of the Support Vector Machine (SVM) learning approach lead to mixed integer quadratic programs that can be solved heuristically. Our generalization of SVMs makes a state-of-the-art classication technique, including non-linear classication via kernels, available to an area that up to now has been largely dominated by special purpose methods. We present experimental results on a pharma- ceutical data set and on applications in automated image indexing and document categorization.
The project integrates work in natural language processing, machine learning, and the semantic web, bringing together these diverse disciplines in a novel way to address a real problem. The objective is to extract and categorize machine components and subsystems and their associated failures using a novel approach that combines text analysis, unsupervised text clustering, and domain models. Through industrial partnerships, this project will demonstrate effectiveness of the proposed approach with actual industry data.
Text categorization is the problem of automatically assigning text documents into one or more categories. Typically, an amount of labelled data, positive and negative examples for a category, is available for training automatic classifiers. We are particularly concerned with text classification when the training data is highly imbalanced, i.e., the number of positive examples is very small. We show that the linear support vector machine (SVM) learning algorithm is adversely affected by imbalance in the training data. While the resulting hyperplane has a reasonable orientation, the proposed score threshold (parameter b) is too conservative. In our experiments we demonstrate that the SVM-specific cost-learning approach is not effective in dealing with imbalanced classes. We obtained better results with methods that directly modify the score threshold. We propose a method based on the conditional class distributions for SVM scores that works well when very few training examples is available to the learner.
Traditional approaches to system management have been largely based on domain experts through a knowledge acquisition process that translates domain knowledge into operating rules and policies. This has been well known and experienced as a cumbersome, labor intensive, and error prone process. In addition, this process is difficult to keep up with the rapidly changing environments. In this paper, we will describe our research efforts on establishing an integrated framework for mining system log files for automatic management. In particular, we apply text mining techniques to categorize messages in log files into common situations, improve categorization accuracy by considering the temporal characteristics of log messages, develop temporal mining techniques to discover the relationships between different events, and utilize visualization tools to evaluate and validate the interesting temporal patterns for system management.
Traditional supervised learning requires a training data set that consists of inputs and corre-sponding labels. In many applications, however, it is difficult or even impossible to accurately and consistently assign labels to inputs. A relatively new learning paradigm called Multi-ple Instance Learning allows the training of a classifier from ambiguously labeled data. This paradigm has been receiving much attention in the last several years, and has many useful applications in a number of domains (e.g. computer vision, computer audition, bioinformat-ics, text processing). In this report we review several representative algorithms that have been proposed to solve this problem. Furthermore, we discuss a number of existing and potential applications, and how well the currently available algorithms address the problems presented by these applications.
The multiple instance regression (MIR) problem arises when a data set is a collection of bags, where each bag contains multiple instances sharing the identical real-valued label. The goal is to train a regression model that can accurately predict label of an unlabeled bag. Many remote sensing applications can be studied within this setting. We propose a novel probabilistic framework for MIR that represents bag labels with a mixture model. It is based on an assumption that each bag contains a prime instance which is responsible for the bag label. An expectation-maximization algorithm is proposed to maximize the likelihood of the mixture model. The mixture model MIR framework is quite flexible, and several existing MIR algorithms can be described as its special cases. The proposed algorithms were evaluated on synthetic data and remote sensing data for aerosol retrieval and crop yield prediction. The results show that the proposed MIR algorithms achieve higher accuracy than the previous state of the art.
In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. This paper provides a survey on this topic. At first, it introduces the origin of multi-instance learning. Then, developments on the study of learnability, learning algorithms, applications and extensions of multi-instance learning are reviewed. In particular, this paper employs a unified view to look into multi-instance learning algorithms. Some important issues to be addressed are also discussed. Abstract In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. This paper provides a survey on this topic. At first, it introduces the origin of multi-instance learning. Then, developments on the study of learnability, learning algorithms, applications and extensions of multi-instance learning are reviewed. In particular, this paper employs a unified view to look into multi-instance learning algorithms. Some important issues to be addressed are also discussed.
The multiple instance problem arises in tasks where the training examples are ambiguous: a single example object may have many alternative feature vectors (instances) that describe it, and yet only one of those feature vectors may be responsible for the observed classification of the object. This paper describes and compares three kinds of algorithms that learn axis-parallel rectangles to solve the multiple instance problem. Algorithms that ignore the multiple instance problem perform very poorly. An algorithm that directly confronts the multiple instance problem (by attempting to identify which feature vectors are responsible for the observed classifications) performs best, giving 89% correct predictions on a musk odor prediction task. The paper also illustrates the use of artificial data to debug and compare these algorithms.
The importance of event logs, as a source of information in systems and network management cannot be overempha- sized. With the ever increasing size and complexity of to- day's event logs, the task of analyzing event logs has become cumbersome to carry out manually. For this reason recent research has focused on the automatic analysis of these log les. In this paper we present IPLoM (Iterative Partitioning Log Mining), a novel algorithm for the mining of clusters from event logs. Through a 3-Step hierarchical partition- ing process IPLoM partitions log data into its respective clusters. In its 4th and nal stage IPLoM produces clus- ter descriptions or line formats for each of the clusters pro- duced. Unlike other similar algorithms IPLoM is not based on the Apriori algorithm and it is able to nd clusters in data whether or not its instances appear frequently. Evalu- ations show that IPLoM outperforms the other algorithms statistically signicantly, and it is also able to achieve an average F-Measure performance 78% when the closest other algorithm achieves an F-Measure performance of 10%.
We empirically study the relationship be- tween supervised and multiple instance (MI) learning. Algorithms to learn various con- cepts have been adapted to the MI represen- tation. However, it is also known that con- cepts that are PAC-learnable with one-sided noise can be learned from MI data. A rel- evant question then is: how well do super- vised learners do on MI data? We attempt to answer this question by looking at a cross section of MI data sets from various domains coupled with a number of learning algorithms including Diverse Density, Logistic Regres- sion, nonlinear Support Vector Machines and FOIL. We consider a supervised and MI ver- sion of each learner. Several interesting con- clusions emerge from our work: (1) no MI al- gorithm is superior across all tested domains, (2) some MI algorithms are consistently su- perior to their supervised counterparts, (3) using high false-positive costs can improve a supervised learner's performance in MI do- mains, and (4) in several domains, a super- vised algorithm is superior to any MI algo- rithm we tested.