Available via license: CC BY 3.0
Content may be subject to copyright.
IOP Conference Series: Earth and Environmental Science
PAPER • OPEN ACCESS
A Fault Diagnosis System Based on Case Decision Technology for UAV
Inspection of Power Lines
To cite this article: Jianyang Li et al 2021 IOP Conf. Ser.: Earth Environ. Sci. 632 042077
View the article online for updates and enhancements.
This content was downloaded from IP address 185.36.195.36 on 15/01/2021 at 16:00
Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution
of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
Published under licence by IOP Publishing Ltd
2020 Asia Conference on Geological Research and Environmental Technology
IOP Conf. Series: Earth and Environmental Science 632 (2021) 042077
IOP Publishing
doi:10.1088/1755-1315/632/4/042077
1
A Fault Diagnosis System Based on Case Decision Technology
for UAV Inspection of Power Lines
Jianyang Li1, *, Hui Wu1, Chunhua Hu2, Chenglie Yu2
1School of Electrical Engineering, Zhenjiang Institute of Technology, Zhenjiang,
China
2Department of Technology, Jiangsu Capital Holdings Manufacturing, Zhenjiang,
China
*Corresponding author email: jianyangli@zjit.edu.cn
Abstract. Though Deep Learning CNN is mostly used in UAV power line-inspection
system for the application of intelligent image recognition technology, can design image
features easily and has strong adaptability to complex environments, but three problems
deafly influence the actual results of application system such as insufficient image
samples library, scarce labeling samples, and absent open-data source. To conquer these
problems, CBR is proposed as a strategy for knowledge reasoning, which transform the
similar case-space to a new situation for problem-solving, so the combination of RBR
and CBR is expected to construct our flexible case- decision diagnosis system, which
integrates efficient machine learning methods to give their full advantages to guarantee
the good performance of the system for fault detection. The on spot experimental results
indicates our system performs efficiently, assist people in decision-making and can find
potential equipment faults.
1. Introduction
Power lines is the main channel of power supply, which has more than 1.15 million kilometers long in
China, and how to ensure power lines unobstructed is the key for power supply. The power lines
inspection is extremely performed, and normally the inspectors should regularly carry out manual
inspection, climb the high tower, and walk back and forth between the lines to find potential problems,
so the task is tremendous. Moreover, the terrain of our country is complex and diverse, where all kinds
of geographical conditions are staggered, thus the task is arduous.
Restricted by the working conditions, personnel ability and other factors, manual line-inspection has
many deficits like high life risk, high labor intensity, professional skills requirements, and blind spot
exist [1]. Benefiting from modern communication technology and AI technology, UAV line-inspection
has made many progresses for lightweight UAV materials, miniaturization and integration of sensors,
image recognition becomes the key process.
image recognition has got rapid progress in recent years, the most popular deep learning image
recognition based on convolution neural network combines artificial neural network with deep learning
technology, which has the characteristics of local perception region, hierarchical structure, feature
extraction and global training combined with classification process, so it is widely used for image
recognition in all kinds of domains. but for a specific task, it is still difficult to determine which network
2020 Asia Conference on Geological Research and Environmental Technology
IOP Conf. Series: Earth and Environmental Science 632 (2021) 042077
IOP Publishing
doi:10.1088/1755-1315/632/4/042077
2
structure is used, how many layers are used, and how many neurons are used in each layer [2]. Empirical
knowledge are still needed to select reasonable values such as learning rate, intensity of regularization,
etc.
From the view of recognition, CBR involves human thinking in an integrated sense, logics and
creativity, and can avoid abstract knowledge insufficient within the field of the traditional Rule-based
reasoning (RBR) which is just a simple simulation of abstract thinking [3]. Case intelligent system can
use the past experience of success and failures to guide problem-solving, assist user in refining the
details, access the knowledge and experience in the case library, so the case decision system proposed
for fault diagnosis can help user finding potential problem in automatic line-inspection, and use more
imaginative thinking in decision-making really.
2. Characteristic of UAV Imagine Recognition
Image recognition and classification technology is to extract potentially useful, unknown, and implicit
knowledge from picture data, which combines multiple disciplines such as data mining, machine learning,
computer vision, picture retrieval and image processing, and other artificial intelligent technology [4].
Image recognition of UAV line-inspection mainly concerns about such three common technologies,
target segmentation, target detection and target tracking.
2.1. General Recognition Methods)
2.1.1. Target segmentation. Target segmentation also known as image semantic segmentation, refers to
the use of a computer to classify the pixels of an image according to the semantic information, for
general optical images, segmentation pixels is a common goal, which pixels extracted are used to express
these known targets. At present, main image semantic segmentation methods are designed based on
deep learning. This kind of segmentation can be a classification problem, which is to label each pixel
and propose the interesting pixel of the same type. It can also be a clustering problem, which means you
don't know the label, but you need to meet some optimality conditions, such as the minimum correlation.
2.1.2. Target detection. This is a recognition problem based on classification to find all objects from
the given data, which samples are classified as targets and distinguish which ones are not. The level of
this classification is often not pixels, it is the given some segments, or defined objects through target
segmentation. At the same time it determines the position of each object in the picture, which includes
two subtasks: target classification and target positioning. In the field of power lines recognition
application, image semantic segmentation mainly realizes the classification of power equipment,
components and operators' images at a fine-grained level. It can improve the computer's cognition level
of power lines images, and promote the advanced level of video image recognition in such application
systems.
2.1.3. Target tracking. Given the size and position of the target in the initial frame of a video sequence,
predict the size and position of the target in subsequent frames, which can be divided into single target
tracking and multi-target tracking. The first important task is target locating to find certain object, and
the data designed is Temporal Data (time series). after the target is identified, the algorithm needs to
relocate the given target quickly and efficiently in the next sequential data. So, the algorithm must
distinguish similar goals, avoid unnecessary recalculations, and make full use of temporal correlation.
2.2. Special Domain Problems
Deep learning CNN is widely used in UAV power line-inspection system for the application of intelligent
image recognition technology, and has got much rapid progress, but the two problems exist in the actual
application systems, greatly restrict its recognition ability.
2020 Asia Conference on Geological Research and Environmental Technology
IOP Conf. Series: Earth and Environmental Science 632 (2021) 042077
IOP Publishing
doi:10.1088/1755-1315/632/4/042077
3
2.2.1. Samples problem. Machine learning is a data-driven process that uses algorithms to dig out the
hidden laws from a large amount of historical data, attempt to parse data, learn from it and use them for
prediction or classification. The so-called training data is a data set that has been preprocessed (usually
manually labeled) and has relatively stable and accurate feature descriptions. Training data selection
generally has the following requirements: the data sample is as large as possible, the data is diversified,
and the data sample quality is high.
AS the VC dimension indicates that for a set of functions {f(α)} is defined as the maximum number
of training points which can be shattered by {f(α)}, the Empirical Risk Minimization is like:
, (1)
Then
(2)
The more complex the model has, the greater h (VC dimension) has. So in order to make the model
have a good generalization, it is necessary to have a large N to lower the model complexity penalty, that
is why deep learning models require millions of training samples.
However, in the actual application field of UAV power line-inspection, neither the equipment defect
inspection image database nor the equipment environment image database has been built on a large scale,
especially the abnormal image samples are very few.
2.2.2. Tagging problem. Most intelligent image recognition learn the deep features of the data by
supervised learning algorithms, which requires that the image dataset not only satisfy the condition of
sufficient quantity and quantity equilibrium in all kinds of samples, but also ensure that these grid image
data are accurately labeled.
considering the field of power line-inspection system, the image data of the power grid has certain
secret property, it is not suitable for outsourcing companies or professional firms to tag samples. In
addition, many image data need experienced professional to judge whether the equipment has defects
or the severity of defects, which makes tagging problem even worse.
moreover there are many kinds of power equipments with many models and mutations suiting for
different running conditions, due to their high reliability, which will inevitably lead to the lack of defect
samples. In addition, many image data in the power industry are defined as confidential, and it is less
likely to establish a large training sample data set with open sharing.
3. Fault Diagnosis System construction
As a significant branch of AI, CBR has found to be an elective approach to the solution of problems in a
variety of domains when generalized knowledge is lacking just like medical diagnosis, time series
prediction, product design, planning, and online decision guides, etc. CBR system has grown from a quite
new area to a subject of major influence, and has good flexibility which indicates that CBR is a hybrid
system [5], so CBR is proposed in line-inspiration to find potential faults, guiding DSS practitioners in
improving their core knowledge acquiring for effective support.
3.1. Problem Solving
Scott-Morton first articulated the concept DSS in the early 1970s, emphasized the positive auxiliary role
to the decision process. Bonczek and others put forward the idea of Intelligent DSS in the early 1980s,
combined the Expert System and knowledge processing in AI semi-structured and unstructured system.
It is generally believed that intelligent DSS uses AI technology for gaining competitive advantage,
1
1
() | (, ))|
2
n
emp i i
i
Ryfx
n
(log(2 / ) 1) log( 4)
() ()
emp hnh
RR n
/
(log(2 / ) 1) log( 4)
()1
test training hnh
Perror error n
/
2020 Asia Conference on Geological Research and Environmental Technology
IOP Conf. Series: Earth and Environmental Science 632 (2021) 042077
IOP Publishing
doi:10.1088/1755-1315/632/4/042077
4
simulate human thinking methods and decision-making processes, formulating better problem-solving
processes, improving decision quality.
Classical knowledge- or rule-based DSS draw conclusions by applying generalized rules, successful
in many application areas except for several problems in knowledge acquisition. Their traditional
analysis methods can only work in a given matching mode, whose adaptability to the environment is
poor, and the traditional hard knowledge processing method can not suit for dealing with uncertain
knowledge [6]. Therefore, the theoretical and technical of flexible information processing must be
studied.
CBR is a strategy for humans to apply past experience to solve new problems by means of analogical
reasoning and hypothetical reasoning, which compares two similar things and find out their similar
relationship at a certain level of abstraction, projects the problem space mapping, and then the solution
is obtained after proper knowledge transformation. It is a reasoning process from a specific instance to
another specific instance, the conclusion may be valid or invalid, so objective verification is required,
and adjustments can be made when new knowledge is acquired or contradictions are introduced.
Incomplete inductive reasoning, analogical reasoning, and hypothetical reasoning are all processes that
can discover and improve new knowledge, whose flexibility can be introduced in logic theory for
decision-making.
3.2. System Modeling
Based on the above theoretical model, we construct a practical intelligent fault-diagnosis system. as
shown in fig 1, main components on comprehensive reasoning technology are described as follows (no
consideration of the same components in DSS).
Fig 1. Fault Diagnosis System frame
Case base system and knowledge base system store a variety of formal knowledge and environmental
descriptions, including expert experience and related knowledge expressed in rules. the entire case
library can be partitioned into several parts: case library, rule library, method library, model library, the
2020 Asia Conference on Geological Research and Environmental Technology
IOP Conf. Series: Earth and Environmental Science 632 (2021) 042077
IOP Publishing
doi:10.1088/1755-1315/632/4/042077
5
case library is connected with other libraries, to reduce the size of the entire case library and improve
the system efficiency.
Multi-base coordinator is the key to meet the needs of problem solving in the hybrid system, a certain
data extraction strategies are performed to realize the coordination of the model base system, method
base system, knowledge base system, and database system, and complete model linking operation,
knowledge scheduling, and information query etc. Overall, we propose the corresponding retrieval with
case organization to store cases, convert the case into an autonomously operation that can actively serve
users, improve the efficiency system reasoning, decrease the system complexity.
4. Experiments and Analysis
4.1. System Validation
“Iris Plants Database” is downloaded from UCI ML Repository, which has 150 instances, 3 classes and
4 attributes. The four attributes are represented the key impact vectors as extracted from the functional
equipment by Image Recognition tools. In order to simulate real cases, attributes are extends from 4 to 6
(A5, A6) with values adding in randomly, and the additional 2 attributes are represented faults vector, for
the faulty equipment like cracks in insulator must have the more dimensions by Image Recognition.
Table 1. Partial and overall similarity for Retrieval
Id A1 A2 A3 A4 A5 A6 C t1 t2 t3
1 4.7 3.2 1.3 0.2 0 0 1 0.999 0.006 0.007
2 5.1 3.7 1.5 0.4 0 0 1 0.217 0.004 0.203
3 5.8 2.7 4.1 1.0 0 0 2 0.004 0.909 0.000
4 6.8 2.8 4.8 1.4 0 0 2 0.005 0.780 0.000
5 6.2 3.4 5.4 2.3 0 0 3 0.002 0.003 0.811
6 6.1 3.0 4.9 1.8 0 0 3 0.001 0.006 0.568
7 7.2 3.0 5.8 1.6 0 0 3 0.002 0.001 0.953
8 6.3 2.3 4.4 1.3 0 0 2 0.001 0.797 0.000
9 5.1 3.3 1.7 0.5 0 0 1 0.718 0.006 0.049
t1 4.3 3.0 1.1 0.1 0.7 5.2 1
t2 5.8 2.7 4.1 1.0 0.3 0.8 2
t3 7.7 3.0 6.1 2.3 6.3 3.9 3
Simulating results are shown in table 1, where 9 records are selected randomly from the database as
cases library, with 3 records as target case marked in t1-t3. Overall similarity and partial similarity are
used for case retrieval to find potential deficits, where the same cases have the max similarity and have
been classified perfectly. By applying feature weights we can put special emphasis on some features for
the similarity calculation, which can be seen in our former work [7] to improve the system accuracy and
efficiency.
4.2. Results with Outlook
A certain actual UAV line- inspection -110kV lines fine inspection is operated as a live process to test
our system performance. As we have described equipment eigenvector of the power lines in good function
have been extracted by Image Recognition tools, for example insulator is a common used equipment in
power lines, with various sizes, weights, dimensions, and different manufacturing materials, they are
formatted as a feature vectors in our diagnosis system.
2020 Asia Conference on Geological Research and Environmental Technology
IOP Conf. Series: Earth and Environmental Science 632 (2021) 042077
IOP Publishing
doi:10.1088/1755-1315/632/4/042077
6
While an UAV automatically performs inspection of power lines, live video and many special
pictures can be transmitted by Data Link system to Control Centre (UAV keeps such information in its
memory as a backup). On-site equipments are distinguished in feature vectors in Control Centre, a
feature-based approach that described the statistical properties of an image. Many other AI methods are
applied in our hybrid systems as co-reasoners to define alternative partial solutions, including RBR,
qualitative reasoning, constraint satisfaction, and model-based reasoning, with which the CBR
mechanism operates in parallel under the supervision of a control module.
Table 2. Summary sheet of line defeats
Id Location Fault description Suggestion Risk
1
No. 8 pole tower of
110kV station line, big
side, shockproof
hammer fittings
Protective fittings, the
shockproof hammer is
corroded, and the fitting has
slight corrosion
replace general
2
No. 9 pole tower of
110kV station line,
small side, right lower
phase, insulator,
composite insulator
Composite insulators aging
and brittle, the umbrella skirts
of composite insulators are
damaged, and the surface of
the material appears chalking,
cracking
repair general
3
Between No. 5 pole
tower and No. 6 of
110kV station line
Dangerous construction
operations that endanger the
line under or near the line
annunciate
Fig 2. Faults Pictures in Lines (Corresponding with Summary Sheet Id)
AS shown in table 2 and fig 2, some defeats have been found by our hybrid application system
(selection of representative faults, which means the others omitted). The staff must firstly recall the
original pictures and examine the results, then judges the inspection results according to the inspection
contents, and should discuss the final solution.
The final manual review confirms the system results, indicates our system constructing for UAV to
carry out power line inspections, can accurately find line faults, give their detail ed information, and
specific location information. The hybrid system can improve the efficiency of fault diagnosis process,
and has a wider perspective.
5. Conclusion
Deep Learning CNN is mostly used in Imagine Recognition, and attract much more researches to
construct their applications, for it can draw image features easily and has strong adaptability to complex
environments, which is largely benefiting from representative and massive training. But for UAV power
line-inspection application it can not well work for insufficient image samples library, scarce labeling
samples, and absent open-data source. So, the hybrid diagnosis system is proposed to conquer the
2020 Asia Conference on Geological Research and Environmental Technology
IOP Conf. Series: Earth and Environmental Science 632 (2021) 042077
IOP Publishing
doi:10.1088/1755-1315/632/4/042077
7
deficient knowledge extracted from imagine recognition, which performs synthesis reasoning to access
the knowledge and experience in the case library, assisting user in refining the reasoning details, and use
more imaginative thinking for fault diagnosis to help user finding potential problem in automatic line-
inspection really. The onsite UAV inspection of 110kV power lines indicates that our system can find
potential defeats and performs efficiently for fault- diagnosis, our future works lies in equipment image
sampling and labeling through the system new cases.
References
[1] QIAN Jinju, Application and Effect of Large-scale Inspection on Power Lines by Using Large
Unmanned Helicopter in Guangdong Power Grid. Guangdong Electric Power. 2016
(29)5,pp.124-129.
[2] ZHOU Feiyan, JIN Linpeng, Review of Convolutional Neural Network . Chinese Journal of
Computers, 2017(6), pp.1229-1251.
[3] Jianyang Li, Xiaoping Liu. Personalized Recommendation System on Massive Content
Processing Using Improved MFNN. Lecture Notes in Computer Science ,Vol:7529, pp.183-
190
[4] Kareem Amin, Stelios Kapetanakis, et al. Answering with Cases: A CBR Approach to Deep
earning. 26th International Conference on Case-Based Reasoning .Springer 2018 ,pp. 15-27.
[5] Mohammad B. Chawki. Tetra: A Case-Based Decision Support System for Assisting Nuclear
Physicians with Image Interpretation. 25th International Conference on Case-Based
Reasoning . 2017,p. 108-122
[6] Zhiwei Ni, Survey of Case Decision Techniques and Case Decision Support System .Computer
Science in Chinese . 2009 (36) 11, pp.18-23
[7] Li Jianyang, Song Changtong. Attribute Weights Mining for Case-Intelligent System Reasoning
on Similarity Rough Sets. 3rd International Conference on Insulating Materials, Material
Application and Electrical Engineering,2018
