No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Abstract
Particle defects on the cathodic copper plate surface always happen due to the immaturity of electrolytic copper processing. The removal of defects mainly depends on their height exceeding the plate and current removal requires manual measurement and operation, which is time-consuming and laborious. To automate the removal process, machine vision-based defect detection methods need to be developed. However, copper defects are of very small size, which increases the difficulty of feature extraction and prediction. Therefore, this paper proposes a novel Anchor-based Detection and Height Estimation (ADHE) framework, to locate the defect out and estimate the height of the defect in an end-to-end way. Large-scale raw images are transformed into several image blocks as input. Defect features are obtained by Defect Region Extraction Network and then sent into Height-RCNN for defect detection and height prediction. Dataset of cathodic copper plate surface defects has been collected from a real-world manufacturing factory. Experimental results show that the proposed ADHE method can effectively address the small size problem of copper defects and achieve excellent results in detection and height estimation.
ResearchGate has not been able to resolve any citations for this publication.
Automatically detecting surface defects from images is an essential capability in manufacturing applications. Traditional image processing techniques were useful in solving a specific class of problems. However, these techniques were unable to handle noise, variations in lighting conditions, and background with complex textures. Increasingly deep learning is being explored to automate defect detection. This survey paper presents three different ways of classifying various efforts. These are based on defect detection context, learning techniques, and defect localization and classification method. The existing literature is classified using this methodology. The paper also identifies future research directions based on the trends in the deep learning area.
Surface defect detection has received increased attention in relation to the product quality and industry safety. This paper develops an image pyramid convolution neural network (IPCNN) model to detect surface defects in images. The IPCNN is an improvement of the Mask rcnn model. It combines image pyramid and deep convolution neural network to extract pyramid features for defect detections. First, it constructs an image pyramid. Then, images from the pyramid are processed with a deep residual neural network to extract features. Extracted features are then fused and processed with a feature pyramid network to generate pyramid features. A region proposal neural network then operates on the pyramid features to generate defect bounding boxes and make classifications. Finally, a fully convolutional neural network generates defect masks in the detected bounding boxes. The IPCNN was evaluated on the publicly available COCO dataset and improved the Mask rcnn by approximately one point. Additionally, it generated high precision and recall values on inspecting the oil leak defect on freight train with a small dataset. The comparison with state-of-the-art methods on freight train defect detection showed the promising results of IPCNN. The efficiency of the IPCNN is 0.243 seconds per frame.
Object detection, one of the most fundamental and challenging problems in computer vision, seeks to locate object instances from a large number of predefined categories in natural images. Deep learning techniques have emerged as a powerful strategy for learning feature representations directly from data and have led to remarkable breakthroughs in the field of generic object detection. Given this period of rapid evolution, the goal of this paper is to provide a comprehensive survey of the recent achievements in this field brought about by deep learning techniques. More than 300 research contributions are included in this survey, covering many aspects of generic object detection: detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics. We finish the survey by identifying promising directions for future research.
We present a method for detecting objects in images using a single deep neural network. Our approach, named SSD, discretizes the output space of bounding boxes into a set of default boxes over different aspect ratios and scales per feature map location. At prediction time, the network generates scores for the presence of each object category in each default box and produces adjustments to the box to better match the object shape. Additionally, the network combines predictions from multiple feature maps with different resolutions to naturally handle objects of various sizes. SSD is simple relative to methods that require object proposals because it completely eliminates proposal generation and subsequent pixel or feature resampling stages and encapsulates all computation in a single network. This makes SSD easy to train and straightforward to integrate into systems that require a detection component. Experimental results on the PASCAL VOC, COCO, and ILSVRC datasets confirm that SSD has competitive accuracy to methods that utilize an additional object proposal step and is much faster, while providing a unified framework for both training and inference. For input, SSD achieves 74.3 % mAP on VOC2007 test at 59 FPS on a Nvidia Titan X and for input, SSD achieves 76.9 % mAP, outperforming a comparable state of the art Faster R-CNN model. Compared to other single stage methods, SSD has much better accuracy even with a smaller input image size. Code is available at https:// github. com/ weiliu89/ caffe/ tree/ ssd.
Suffering from the diversity, complexity and miniaturization of PCB (Printed Circuit Board) defects, traditional detection methods are difficult to detect. Despite object detection has made significant advances based on deep neural networks, it remains a challenge to focus on small objects. We address this challenge by allowing multiscale fusion. We introduce a PCB defect detection algorithm based on extended Feature Pyramid Network model in this paper. The backbone is constructed by part of ResNet-101, in order to accurately locate and identify small objects, this paper constructs a feature layer which integrates high-level semantic information and low-level geometric information. Based on FPN(Feature Pyramid Networks) network structure, using 1×1 convolution lateral fusion of the previous semantic information, the fused features use 3×3 convolution to obtain the final feature layer. The problem that PCB defects are difficult to classify is considered, the focal loss function is introduced. In order to reduce over-fitting in the training process, the original data is enhanced by image clipping and rotation. Through the quantitative analysis on PCB defect dataset, the results is the best to use the fused low-level feature layer for detection. The mAP(mean Average Precision) is 96.2% on the public PCB dataset, which is surpassing the state-of-the-art methods.
In the integrated circuit (IC) packaging, the surface defect detection of flexible printed circuit boards (FPCBs) is important to control the quality of IC. Although various computer vision (CV)-based object detection frameworks have been widely used in industrial surface defect detection scenarios, FPCB surface defect detection is still challenging due to non-salient defects and the similarities between diverse defects on FPCBs. To solve this problem, a decoupled two-stage object detection framework based on convolutional neural networks (CNNs) is proposed, wherein the localization task and the classification task are decoupled through two specific modules. Specifically, to effectively locate non-salient defects, a multi-hierarchical aggregation (MHA) block is proposed as a location feature (LF) enhancement module in the defect localization task. Meanwhile, to accurately classify similar defects, a locally non-local (LNL) block is presented as a SEF enhancement module in the defect classification task. What is more, an FPCB surface defect detection dataset (FPCB-DET) is built with corresponding defect category and defect location annotations. Evaluated on the FPCB-DET, the proposed framework achieves state-of-the-art (SOTA) accuracy to 94.15% mean average precision (mAP) compared with the existing surface defect detection networks. Soon, source code and dataset will be available at
https://github.com/SCUTyzy/decoupled-two-stage-framework
.
In modern manufacturing, vision-based defect recognition is an essential technology to guarantee product quality, and it plays an important role in industrial intelligence. With the developments of industrial big data, defect images can be captured by ubiquitous sensors. And, how to realize accuracy recognition has become a research hotspot. In the past several years, many vision-based defect recognition methods have been proposed, and some newly-emerged techniques, such as deep learning, have become increasingly popular and have addressed many challenging problems effectively. Hence, a comprehensive review is urgently needed, and it can promote the development and bring some insights in this area. This paper surveys the recent advances in vision-based defect recognition and presents a systematical review from a feature perspective. This review divides the recent methods into designed-feature based methods and learned-feature based methods, and summarizes the advantages, disadvantages and application scenarios. Furthermore, this paper also summarizes the performance metrics for vision-based defect recognition methods. And some challenges and development trends are also discussed.
The existing object detection algorithms based on the convolutional neural network (CNN) are always devoted to the detection of natural objects and have achieved admirable detection effects. At present, these detection algorithms have been applied to the detection of defect data. In fact, the detection of defect data is different from the detection of general natural object data. First, it has a large number of images without annotations (that is, normal images), and they each contain different background information. Second, its processing principles are fundamentally different from general object detection problems. Therefore, the application of a general object detection algorithm based on CNN may not be perfect in this problem. In this article, a novel defect detection network (DefectNet) is proposed to solve the problem of defect detection. It first uses a shared weight binary classification network to determine whether an image contains the targets and then uses the detection network to detect the targets. Theoretical deduction and experimental results fully confirm that it can effectively improve the detection speed and effect of the general object detection network based on CNN. (Our code is available at
https://github.com/li-phone/DefectNet.git
)
In vision-based defect recognition, deep learning (DL) is a research hotspot. However, DL is sensitive to image quality, and it is hard to collect enough high-quality defect images. The low-quality images usually lose some useful information and may mislead the DL methods into poor results. To overcome this problem, this paper proposes a generative adversarial network (GAN)-based DL method for low-quality defect image recognition. A GAN is used to reconstruct the low-quality defect images, and a VGG16 network is built to recognize the reconstructed images. The experimental results under low-quality defect images show that the proposed method achieves very good performances, which has accuracies of 95.53%-99.62% with different masks and noises, and they are improved greatly compared with the other methods. Furthermore, the results on PSNR, SSIM, cosine and mutual information indicate that the quality of the reconstructed image is improved greatly, which is very helpful for defect analysis.
Recently, many methods based on low-rank representation have been proposed for fabric defect detection. Most of them relax the low-rank decomposition problem to a nuclear norm minimization (NNM) problem to pursue the convexity of the objective function. When solving the standard NNM problem, matrix singular values have to be treated equally. This, however, would be impractical in the scenario of fabric defect detection as the matrix singular values have clear physical meanings, and thus, they should be treated differently. In this article, we propose a weighted double-low-rank decomposition method (WDLRD) to treat the matrix singular values differently by assigning different weights. Thus, the most important/distinguishing characteristics of a fabric image can be preserved. Another difference between WDLRD and the other existing low-rank-based methods is that WDLRD considers a defective fabric image being decomposed to two low-rank matrices, i.e., low-rank defect-free matrix and low-rank defect matrix, as the defect-free and defective regions are usually composed of homogeneous objects that have a high correlation. Besides, WDLRD is more robust for defect detection in various situations by adding a noise term to avoid noise or other interference on the fabric surface. In addition, a defect prior is incorporated into the objective function of WDLRD to guide locating the defective regions. The proposed optimization problem can be easily solved by an iterative algorithm based on augmented Lagrange multipliers. Experimental results on TILDA, periodically patterned fabric, and Textile & Apparel Artificial Intelligence databases show that the proposed WDLRD obtains better performance than state-of-the-art methods in locating the defective regions on fabric images.
Note to Practitioners
—This article is motivated by the problem that the performance of fabric defect detection in the textile industry is poor. It is necessary to develop an effective method to improve the defect detection accuracy and reduce overall manufacturing cost. Existing automatic defect detection approaches usually contain two stages: first, capture fabric images from the weaving machine and then use a defect detection algorithm in a host computer to conduct a real-time inspection and give an alarm if defects occur. This article focuses on locating defects for given defective images after the procedure of binary classification (which determines an image as defective or defect free). The article proposes an objective function to mathematically interpret the optimization problem between fabric images and predictive defects. The optimal solution can be obtained by employing the alternating direction method of multipliers (ADMMs). The proposed method is described as a new defect detection algorithm. Extensive experiments were conducted to evaluate the algorithm, and the experimental results indicate that the proposed method is superior to many existing fabric defect detection methods. Preliminary experiments suggest that this method is feasible but has not yet been really used in production. In future research, we will collect more fabric images from the textile industry and develop large-scale databases for verifying the proposed method for real-life applications.
High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions
in series
(e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams
in parallel
and (ii) repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems. All the codes are available at
https://github.com/HRNet
.
Fabric defect detection is an intriguing but challenging topic. Many methods have been proposed for fabric defect detection, but these methods are still suboptimal due to the complex diversity of both fabric textures and defects. In this paper, we propose a generative adversarial network (GAN)-based framework for fabric defect detection. Considering existing challenges in real-world applications, the proposed fabric defect detection system is capable of learning existing fabric defect samples and automatically adapting to different fabric textures during different application periods. Specifically, we customize a deep semantic segmentation network for fabric defect detection that can detect different defect types. Furthermore, we attempted to train a multistage GAN to synthesize reasonable defects in new defect-free samples. First, a texture-conditioned GAN is trained to explore the conditional distribution of defects given different texture backgrounds. Given a novel fabric, we aim to generate reasonable defective patches. Then, a GAN-based fusion network fuses the generated defects to specific locations. Finally, the well-trained multistage GAN continuously updates the existing fabric defect datasets and contributes to the fine-tuning of the semantic segmentation network to better detect defects under different conditions. Comprehensive experiments on various representative fabric samples are conducted to verify the detection performance of our proposed method.
Surface defect detection is a critical task in industrial production process. Nowadays, there are lots of detection methods based on computer vision and have been successfully applied in industry, they also achieved good results. However, achieving full automation of surface defect detection remains a challenge, due to the complexity of surface defect, in intra-class, while the defects between inter-class contain similar parts, there are large differences in appearance of the defects. To address these issues, this paper proposes a pyramid feature fusion and global context attention network for pixel-wise detection of surface defect, called PGA-Net. In the framework, the multi-scale features are extracted at first from backbone network. Then the pyramid feature fusion module is used to fuse these features into five resolutions through some efficient dense skip connections. Finally, the global context attention module is applied to the fusion feature maps of adjacent resolution, which allows effective information propagate from low-resolution fusion feature maps to high-resolution fusion ones. In addition, the boundary refinement block is added to the framework to refine the boundary of defect and improve the result of predict. The final prediction is the fusion of the five resolutions fusion feature maps. The results of evaluation on four real-world defect datasets demonstrate that the proposed method outperforms the state-of-the-art methods on mean Intersection of Union and mean Pixel Accuracy (NEU-Seg: 82.15%, DAGM 2007: 74.78%, MT_defect: 71.31%, Road_defect: 79.54%).
Vision-based defect recognition is an important technology to guarantee quality in modern manufacturing systems. And deep learning (DL) becomes a research hotspot in vision-based defect recognition due to the outstanding performances. However, most of the DL methods require a large sample to learn the defect information. While in some real-world cases, it is difficult and costly for data collecting, and only a small sample is available. Generally, a small sample contains less information, which may mislead the DL models, so that they cannot work as expected. Therefore, this requirement impedes the wide applications of DL in vision-based defect recognition. To overcome this problem, this paper proposes a multi-level information fusion-based DL method for vision-based defect recognition. In the proposed method, a three-level Gaussian pyramid is introduced to generate multi-level information of the defect, so that more information is available for model training. After the Gaussian pyramid, three VGG16 networks are built to learn the information and the outputs are fused for the final recognition result. The experimental results show that the proposed method can extract more useful information and achieve better performances on small-sample tasks, compared with the conventional DL methods and defect recognition methods. Furthermore, the analysis results of the robustness and response time also indicate that the proposed method is robust for the noise input, and it is fast for defect recognition, which takes 13.74ms to handle a defect image.
A complete defect detection task aims to achieve the specific class and precise location of each defect in an image, which makes it still challenging for applying this task in practice. The defect detection is a composite task of classification and location, leading to related methods are often hard to take into account the accuracy of both. And the implementation of defect detection depends on a special detection dataset which contains expensive manual annotations. In this paper, we proposed a novel defect detection system based on deep learning and focused on a practical industrial application: steel plate defect inspection. In order to achieve strong classification-ability, this system employs a baseline convolution neural network (CNN) to generate feature maps at each stage. And then the proposed multilevel-feature fusion network (MFN) combines multiple hierarchical features into one feature, which can include more location details of defects. Based on these multilevel features, a region proposal network (RPN) is adopted to generate regions of interest (ROIs). For each ROI, a detector, consisting of a classifier and a bounding box regressor, produces the final detection results. Finally, we set up a defect detection dataset NEU-DET for training and evaluating our method. On the NEU-DET, our method achieves 74.8/82.3 mAP with baseline networks ResNet34/50 by using 300 proposals. In addition, by using only 50 proposals, our method can detect at 20 fps on a single GPU and reach 92% of the above performance, hence the potential for real-time detection.
We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to 1/2 everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.
State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet [7] and Fast R-CNN [5] have reduced the running time of these detection networks, exposing region pro-posal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully-convolutional network that simultaneously predicts object bounds and objectness scores at each position. RPNs are trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. With a simple alternating optimization, RPN and Fast R-CNN can be trained to share convolu-tional features. For the very deep VGG-16 model [18], our detection system has a frame rate of 5fps (including all steps) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007 (73.2% mAP) and 2012 (70.4% mAP) using 300 proposals per image. The code will be released.
Nanoproducts represent a potential growing sector and nanofibrous materials are widely requested in industrial, medical and environmental applications. Unfortunately, the production processes at the nanoscale are difficult to control, and nanoproducts often exhibit localized defects that impair their functional properties. Therefore, defect detection is a particularly important feature in smart-manufacturing systems, to raise alerts as soon as defects exceed a given tolerance level and to design production processes that both optimize the physical properties and control the defectiveness of the produced materials. Here we present a novel solution to detect defects in nanofibrous materials by analyzing Scanning Electron Microscope (SEM) images. We employ an algorithm that learns, during a training phase, a model yielding sparse representations of the structures that characterize correctly produced nanofiborus materials. Defects are then detected by analyzing each patch of an input image and extracting features that quantitatively assess whether the patch conforms or not to the learned model. The proposed solution has been successfully validated over 45 images acquired from samples produced by a prototype electrospinning machine. The low computational times indicate that the proposed solution can be effectively adopted in a monitoring system for industrial production.
Deeper neural networks are more difficult to train. We present a residual
learning framework to ease the training of networks that are substantially
deeper than those used previously. We explicitly reformulate the layers as
learning residual functions with reference to the layer inputs, instead of
learning unreferenced functions. We provide comprehensive empirical evidence
showing that these residual networks are easier to optimize, and can gain
accuracy from considerably increased depth. On the ImageNet dataset we evaluate
residual nets with a depth of up to 152 layers---8x deeper than VGG nets but
still having lower complexity. An ensemble of these residual nets achieves
3.57% error on the ImageNet test set. This result won the 1st place on the
ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100
and 1000 layers.
The depth of representations is of central importance for many visual
recognition tasks. Solely due to our extremely deep representations, we obtain
a 28% relative improvement on the COCO object detection dataset. Deep residual
nets are foundations of our submissions to ILSVRC & COCO 2015 competitions,
where we also won the 1st places on the tasks of ImageNet detection, ImageNet
localization, COCO detection, and COCO segmentation.
State-of-the-art object detection networks depend on region proposal
algorithms to hypothesize object locations. Advances like SPPnet and Fast R-CNN
have reduced the running time of these detection networks, exposing region
proposal computation as a bottleneck. In this work, we introduce a Region
Proposal Network (RPN) that shares full-image convolutional features with the
detection network, thus enabling nearly cost-free region proposals. An RPN is a
fully-convolutional network that simultaneously predicts object bounds and
objectness scores at each position. RPNs are trained end-to-end to generate
high-quality region proposals, which are used by Fast R-CNN for detection. With
a simple alternating optimization, RPN and Fast R-CNN can be trained to share
convolutional features. For the very deep VGG-16 model, our detection system
has a frame rate of 5fps (including all steps) on a GPU, while achieving
state-of-the-art object detection accuracy on PASCAL VOC 2007 (73.2% mAP) and
2012 (70.4% mAP) using 300 proposals per image. The code will be released.
This paper proposes a Fast Region-based Convolutional Network method (Fast
R-CNN) for object detection. Fast R-CNN builds on previous work to efficiently
classify object proposals using deep convolutional networks. Compared to
previous work, Fast R-CNN employs several innovations to improve training and
testing speed while also increasing detection accuracy. Fast R-CNN trains the
very deep VGG16 network 9x faster than R-CNN, is 213x faster at test-time, and
achieves a higher mAP on PASCAL VOC 2012. Compared to SPPnet, Fast R-CNN trains
VGG16 3x faster, tests 10x faster, and is more accurate. Fast R-CNN is
implemented in Python and C++ (using Caffe) and is available under the
open-source MIT License at https://github.com/rbgirshick/fast-rcnn.
- K He
- G Gkioxari
- P Dollar
- R Girshick
K. He, G. Gkioxari, P. Dollar, and R. Girshick, "Mask R-CNN,"
2017 IEEE International Conference on Computer Vision (ICCV).
IEEE, 2017.
MMDetection: Open mmlab detection toolbox and benchmark
- Chen
K. Chen et al., "MMDetection: Open mmlab detection toolbox and
benchmark," ArXiv Prepr. ArXiv190607155, 2019.