Conference PaperPDF Available

Object Detection for P&ID Images

Authors:
Jae-Min Cha at Samsung
Jae-Min Cha
Taekyong Lee at Institute for Advanced Engineering
Taekyong Lee
Shelly Salim at Chonnam National University
Shelly Salim
Bohyun Ryu
Bohyun Ryu
Bohyun Ryu
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Download full-text PDF
Read full-text
Download citation

Abstract

Piping and instrumentation diagram(P&ID) is an essential design document which is continuously modified and managed from design phase to O&M phase. For the ease of data transfer, P&IDs are generally converted into PDF which is hard to be modified. Therefore, engineering companies who need to manage P&IDs should manually re-convert their P&ID images into the P&IDs in CAD formats. To reduce the inefficiency of the P&ID re-conversion, various symbols and texts in P&ID images should be automatically detected before the re-conversion. As the first step to the auto P&ID conversion, in this study, we propose methods detecting symbols and texts in P&ID image using geometrical and deep learning-based approaches.
Content uploaded by Jae-Min Cha
Author content
All content in this area was uploaded by Jae-Min Cha on Nov 07, 2019
Content may be subject to copyright.
International Congress and Conferences on Computational Design and Engineering 2019 (I3CDE 2019)
Object Detection for P&ID Images
Taekyong Leea, Jae-min Chaa, Bohyun Ryua and Shelly Salimb
aPlant Engineering Center, Institute for Advanced Engineering, Korea
bGwangju Institute of Science and Technology, Korea
E-mail: tklee@iae.re.kr
Abstract
Piping and instrumentation diagram(P&ID) is an
essential design document which is continuously
modified and managed from design phase to O&M phase.
For the ease of data transfer, P&IDs are generally
converted into PDF which is hard to be modified.
Therefore, engineering companies who need to manage
P&IDs should manually re-convert their P&ID images
into the P&IDs in CAD formats. To reduce the
inefficiency of the P&ID re-conversion, various symbols
and texts in P&ID images should be automatically
detected before the re-conversion. As the first step to the
auto P&ID conversion, in this study, we propose methods
detecting symbols and texts in P&ID image using
geometrical and deep learning-based approaches.
Keywords –
P&ID, Symbol Detection, Text Detection
1 Introduction
Piping and Instrumentation Diagram(P&ID) is a
diagram representing core information of a plant such as
equipment, pipelines, instruments and their logical,
physical connections. As P&IDs contain core
information of processes and configurations of a plant,
they are continuously modified and managed throughout
a whole engineering lifecycle. P&IDs are drawn in CAD
systems designed for P&ID such as AutoCAD P&ID,
however in some cases, P&IDs are handed over to a
customer as an image or PDF format. To modify and
manage P&ID images, they are need to be manually re-
converted to the customer company’s CAD format and
this causes great losses in time and human resources.
To reduce the inefficiency of the P&ID re-conversion,
it is required to develop a technology automatically
converting P&ID images into CAD formats. And
primarily elements of P&ID such as equipment, pipelines,
instruments and texts should be automatically detected.
Studies on detecting various engineering diagrams have
been conducted for several decades. Most of previous
studies on diagram detections[1][2] were based on
geometrical features of diagram elements, and recently
several studies have presented detection technologies
based on Convolutional Neural Network(CNN). But it is
still rare to find a study on P&ID detection.
So, in this study, we have studied on methods to
detect elements in P&ID images and have tested the
proposed geometrical and deep learning based methods
for detecting various elements in P&ID images except
pipelines.
2 P&ID Image Detection Methods and
Test Results
2.1 Pre-process
Even though a P&ID template can be changed in each
project due to owners’ requirements, the overall structure
is similar. A P&ID is generally consisted of outer boarder
and inner boarder lines, a note area at right side and a title
area at lower-right side and a main area where a diagram
located. Prior to detecting a main area of a P&ID, it is
recommended to remove boarder lines, note and title
areas for an improved recognition result.
In this study, using Python and OpenCV, we have
developed a simple algorithm to detect and remove
boarders, note and title areas. Figure 1 shows the result.
2.2 Texts
A P&ID contains a lot of text information indicating
ids, numbers and connections of equipment, pipelines
and instruments. Once texts in a P&ID are detected, their
information can be extracted through OCR technologies
and then removed from the P&ID to reduce unnecessary
noises for a deep-learning based symbol detection
training. Detecting Texts is a relatively easier task as
texts in a P&ID are generally in single font, not randomly
oriented, and written clearly in black. We have applied
Figure 1. Main area detection and cropping[3]
Object Detection for P&ID Images
Connectionist Text Proposal Network(CTPN)[4] pre-
trained with ICDAR 2013 MLT dataset to detect texts in
a sample P&ID. The result is shown in Figure 2.
In the test, 164 text elements were correctly detected
from 184 text elements in the P&ID (about 89.1%).
Once texts are localized correctly, then a non-AI
supported OCR technology such as Tesseract[5] can be
applied to recognize the localized texts.
2.3 Instrumentation Symbols
Compared to equipment symbols, Instrumentation
symbols are in simple geometrical shapes. And functions
of Instruments are distinguished by texts inside symbols.
So, as texts can be detected as explained in 2.2, the key
is to detect geometrical shapes. In this study, we have
used Hough Circle Transform[6] in OpenCV library.
With the adjusted parameters (dp=3, minDist=100,
Param2=100, minRadius = 20, maxRadius = 30), we
detected circular Instrumentation symbols as shown in
Figure 4. Though some texts and part of equipment
symbols were detected as circles, all the circular
instrumentation symbols were detected.
2.4 Equipment Symbols
We have used Faster Regional Convolutional Neural
Network(R-CNN)[7] which is widely known for its
reliability and performance, for detecting pump symbols
in P&IDs[8]. Nine P&ID images containing pump
symbols were used for training and ten P&ID images for
testing. All the 25 pumps in testing P&ID images have
been correctly detected.
3 Conclusion
In this study, several detection methods have been
proposed and tested for elements of P&ID images: texts,
instrumentation symbols and equipment symbols. The
results have showed that appropriate applications of both
geometrical and deep learning based approaches can
effectively detect various elements of P&ID images.
Acknowledgement
This work was supported by the Technology
Innovation Programs (10072058, 20000725) funded by the
Ministry of Trade, Industry & Energy(MOTIE, Korea).
References
[1] Bae, C. and Min, B.
Automatic Drawing Input by
Segmentation of Text Region and Recognition of
Geometric Drawing Element
.
The Institute of Electronics
Engineers of Korea
, 31(6):790–802, 1994.
[2] Jang, W. and Kwon, Y. An Effective Vector
Extraction Method based on Drawing Characteristics
. In
Proceedings of Korea Information Science Society
Fall Conference, pages 544–546, 2000.
[3] ISA. PIP PIC 001 Piping and Instrumentation
Diagram Documentation Criteria. Process Industry
Practices, 2008.
[4] Tian, Z. et al. Detecting Text in Natural Image with
Connectionist Text Proposal Network. European
Conference on Computer Vision, 2016.
[5] Smith, L. An Overview of the Tesseract OCR
Engine. In Proceedings of ICDAR 2007, 2007.
[6] OpenCV dev Team. Hough Circle Transform.
Accessed at https://docs.opencv.org/2.4.13.7/doc/
tutorials/imgproc/imgtrans/hough_circle, 2019.
[7] Ren, S. et al. Faster R-CNN: Toward Real-Time
Object Detection with Region Proposal Networks.
arXiv e-prints, arXiv:1506.01497, 2015.
[8] Cha, J. et al. A Preliminary Study on Recognition
of Equipment Symbols from Image P&IDs based on
Deep Learning. In Proceedings of The Korean
Society of Mechanical Engineers Conference, 2018.
Figure 2. Part of the texts localization result
Figure 3. Basic instrumentation symbols (part)
Figure 4. Instrumentation symbols detection[3]
Figure 5. Result of the pump symbols detection
... We reviewed the methods developed for the analysis of P&IDs which include recognition and classification of symbols [1][2][3][4][5][6][7][8][21][22][23][24][25][26][27][28][29][30][31][32], pipelines [2,4,7,8,23,27], texts [1,2,4,5,7,8,23,26,27,29,30] and the inference of their interconnectivity relationships [3,5,7,8,23,27]. These methods utilize image processing techniques, machine learning and deep learning-based algorithms. ...
... We reviewed the methods developed for the analysis of P&IDs which include recognition and classification of symbols [1][2][3][4][5][6][7][8][21][22][23][24][25][26][27][28][29][30][31][32], pipelines [2,4,7,8,23,27], texts [1,2,4,5,7,8,23,26,27,29,30] and the inference of their interconnectivity relationships [3,5,7,8,23,27]. These methods utilize image processing techniques, machine learning and deep learning-based algorithms. ...
... Symbols Detection: Elyan et al. [2] and Lee et al. [29] developed heuristic-based methods for circular symbol recognition using Hough Transform [33]. Howie et al. [3], Arroyo et al. [5] and Ghadekar et al. [27] performed symbol recognition using template matching method. ...
Semi-supervised symbol detection for piping and instrumentation drawings
Article
  • Jan 2024
  • AUTOMAT CONSTR
Current computer vision methods for symbol detection in piping and instrumentation diagrams (P&IDs) face limitations due to the manual data annotation resources they require. This paper introduces a versatile two-stage symbol detection pipeline that optimizes efficiency by (1) labeling only data samples with minimal cumulative informational redundancy, (2) restricting annotation to the minimal effective training dataset size, and (3) expanding the training dataset using pseudo-labels. In Stage-1, the method performs generic symbol detection, while Stage-2 focuses on symbol differentiation through metric learning. To enhance robustness and general-izability, the model is trained on a diverse dataset collected from both industry sources and web scraping. The achieved Top-1 accuracy is 85.39%, with a Top-5 accuracy of 95.19% on a test dataset containing 102 symbol classes. These results suggest the potential for a shift from resource-intensive supervised learning approaches to a more efficient semi-supervised paradigm.
View
Object Detection for P&ID Images using various Deep Learning Techniques
Conference Paper
  • Jan 2021
Piping and Instrumentation Diagrams are detailed diagrams that show the piping and process equipment together with the instrumentation and control devices used in the process industry. These are updated during the plant life cycle to depict the latest changes and modifications done in the plant. These diagrams are detailed and are usually in the form of PDF, they are difficult to modify, and inferring details from these diagrams requires a deep understanding of plant process and engineering. To reduce the difficulty in finding and retrieving information from the diagrams, the proposal of automating the task of detection of the symbols and the texts embedded is explored in this paper. This paper compares various Deep Learning models on P&I Diagrams by using object detection for symbols using the Transfer Learning technique. Initially, the Connectionist Text Proposal Network algorithm is used for text detection in the images, then for the geometrically shaped objects, OpenCV library is used. The models trained by Transfer Learning are exported and the results are compared by performing symbol detection. This paper is an overall study of various deep learning methods implemented on P&I Diagrams.
View
Detecting Text in Natural Image with Connectionist Text Proposal Network
Conference Paper
Full-text available
  • Oct 2016
  • Lect Notes Comput Sci
We propose a novel Connectionist Text Proposal Network (CTPN) that accurately localizes text lines in natural image. The CTPN detects a text line in a sequence of fine-scale text proposals directly in convolutional feature maps. We develop a vertical anchor mechanism that jointly predicts location and text/non-text score of each fixed-width proposal, considerably improving localization accuracy. The sequential proposals are naturally connected by a recurrent neural network, which is seamlessly incorporated into the convolutional network, resulting in an end-to-end trainable model. This allows the CTPN to explore rich context information of image, making it powerful to detect extremely ambiguous text. The CTPN works reliably on multi-scale and multi-language text without further post-processing, departing from previous bottom-up methods requiring multi-step post filtering. It achieves 0.88 and 0.61 F-measure on the ICDAR 2013 and 2015 benchmarks, surpassing recent results [8, 35] by a large margin. The CTPN is computationally efficient with 0.14 s/image, by using the very deep VGG16 model [27]. Online demo is available: http:// textdet. com/ .
View
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Conference Paper
  • Jan 2016
State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet [7] and Fast R-CNN [5] have reduced the running time of these detection networks, exposing region pro-posal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully-convolutional network that simultaneously predicts object bounds and objectness scores at each position. RPNs are trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. With a simple alternating optimization, RPN and Fast R-CNN can be trained to share convolu-tional features. For the very deep VGG-16 model [18], our detection system has a frame rate of 5fps (including all steps) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007 (73.2% mAP) and 2012 (70.4% mAP) using 300 proposals per image. The code will be released.
View
An Overview of the Tesseract OCR Engine
Conference Paper
  • Oct 2007
The Tesseract OCR engine, as was the HP Research Prototype in the UNLV Fourth Annual Test of OCR Accuracy, is described in a comprehensive overview. Emphasis is placed on aspects that are novel or at least unusual in an OCR engine, including in particular the line finding, features/classification methods, and the adaptive classifier.
View
Automatic Drawing Input by Segmentation of Text Region and Recognition of Geometric Drawing Element
  • Jan 1994
  • 790-802
  • C Bae
  • B Min
Bae, C. and Min, B. Automatic Drawing Input by Segmentation of Text Region and Recognition of Geometric Drawing Element. The Institute of Electronics Engineers of Korea, 31(6):790-802, 1994.
An Effective Vector Extraction Method based on Drawing Characteristics
  • Jan 2000
  • 544-546
  • W Jang
  • Y Kwon
Jang, W. and Kwon, Y. An Effective Vector Extraction Method based on Drawing Characteristics. In Proceedings of Korea Information Science Society Fall Conference, pages 544-546, 2000.
PIC 001 Piping and Instrumentation Diagram Documentation Criteria
  • Jan 2008
  • Isa Pip
ISA. PIP PIC 001 Piping and Instrumentation Diagram Documentation Criteria. Process Industry Practices, 2008.
A Preliminary Study on Recognition of Equipment Symbols from Image P&IDs based on Deep Learning
  • Jan 2018
  • J Cha
Cha, J. et al. A Preliminary Study on Recognition of Equipment Symbols from Image P&IDs based on Deep Learning. In Proceedings of The Korean Society of Mechanical Engineers Conference, 2018.