Content uploaded by Zhenyu James Kong
Author content
All content in this area was uploaded by Zhenyu James Kong on Aug 03, 2017
Content may be subject to copyright.
Proceedings of the 2017 Industrial and Systems Engineering Conference
K. Coperich, E. Cudney, H. Nembhard, eds.
Textural Analysis-based Online Closed-Loop Quality Control for
Additive Manufacturing Processes
Chenang Liu, David Roberson, Zhenyu Kong
Grado Department of Industrial and Systems Engineering, Virginia Tech.,
Blacksburg, VA 24061, USA
Abstract
Additive manufacturing (AM) is a powerful technology for fabrication of components with complex geometries using
a variety of materials. One of the major challenges in the AM industry is how to guarantee product quality and
consistency by minimizing the defects. Although AM quality improvement can be achieved by optimizing machine
parameter settings offline, and/or post-processing of AM components, the effects may be limited, particularly for parts
with complex internal structure. Various defects severely deteriorate layer surface quality of AM components. The
objective of this work is to develop an image textural analysis-based real-time diagnosis and closed-loop feedback
control system for the fused deposition modeling (FDM) process. This system consists of a real-time image acquisition
device (digital microscope), a high accuracy image classification algorithm to monitor the status of the printing
process, and a PID controller for closed-loop control. The case study shows that this system is able to identify the
types and severity of defects effectively, and the adjustment of process parameters can be implemented to mitigate the
defects using a feedback control strategy.
Keywords
Additive Manufacturing, Image Textural Analysis, Online Classification, Closed Loop Control
1. Introduction
As an emerging technology, additive manufacturing (AM) has great potential in a large variety of applications, such
as aerospace, the automotive industry, healthcare, etc. [1]. AM, also known as 3D printing, is the process of joining
materials to make objects from 3D models, usually in a layer by layer fashion [2]. Nowadays there are various
technologies which can implement AM using different kinds of materials, such as fused deposition modeling (FDM),
selective laser sintering (SLS), stereolithography (SLA), etc. [3]. With the rapid development of AM, more complex
structures with more materials can be fabricated. However, ensuring AM part quality still remains challenging.
Considering the FDM process, which is a commonly applied technique in AM, the defects have various types, such
as voids, overfill, underfill, etc. [4, 5]. Some of these defects can be avoided by optimizing the machine parameter
settings before printing, or can be eliminated by post-processing after printing. However, due to the highly complex
interactions in consecutive layers, especially with varying cross-sectional geometries, some large process variations
may occur, which cannot be resolved by offline optimal parameter settings. Post-processing, such as machining or
polishing, is only effective for AM part external surface quality. However, some other defects, for instance, overfill
or underfill, occur in between layers, which exist inside of the printed parts, and consequently deteriorate the quality
of AM parts severely, such as strength, internal structure precision, and surface quality [5, 6]. To ensure the quality of
AM parts, it is necessary to develop an effective real-time process monitoring and closed-loop control system for AM
processes, which can detect the defects and adjust the machine parameters automatically to compensate for these
defects.
In this work, our investigation includes the following tasks: (1) Build a real-time acquisition system, which can capture
the high-resolution images for the material being extruded below the extruder; (2) Develop a high accuracy texture
analysis-based classification method to monitor the status of the printing; (3) Apply a PID controller to implement
automatic adjustment when defects occur during the printing process.
The rest of this paper is structured as follows: a brief review of related research work is provided in Sec. 2; the proposed
research approach is discussed in detail in Sec. 3; Sec. 4 presents the case studies, including experimental settings,
analytics, and the performance of online testing; finally, the conclusions and future direction are discussed in Sec. 5.
Liu, Roberson, Kong
2. Review of Related Work
Rao et al. [7] used heterogeneous sensors with effective classification algorithms to achieve real-time quality
monitoring for the FDM process. Similarly, for the same sensor configuration, Bastani et al. [8] developed another
online sparse estimation-based classification approach to effectively handle the real-time monitoring problem. Besides
algorithm development, more types of sensors are also investigated in a number of recent work, Wu et al. [9] proposed
a new in situ monitoring framework for FDM, based on an acoustic emission sensing system, which can identify both
normal and abnormal states of the machine conditions. Kousiatza et al. [10] investigated an approach to monitor the
strain and temperature distributions of the FDM process by an integrated fiber Bragg grating (FBG) sensing system.
For the image-based online monitoring techniques in FDM process investigation (or processes resembling FDM),
Fang et al. [11] applied machine vision techniques to detect defects based on optical imaging of each layer, which can
evaluate the geometrical integrity of the build via comparing the optical imaging result and its corresponding original
CAD design. Fang et al. [12] further developed a related online signature analysis-based monitoring approach to detect
the process anomalies. Cheng et al. [13] proposed an online approach to monitor the surface pattern by using image
intensity information, and were able to classify the randomly occurring defects and anomalies from assignable causes.
Although these research efforts have provided effective methods to implement process quality monitoring for FDM,
a key shortcoming evident is the lack of online detailed diagnosis, and strategies for process correction by parameter
adjustment. Prior experimental studies have already investigated the effects of important adjustable process variables
in FDM [4, 14]. Therefore, this paper seeks to address this gap by an image texture analysis-based online diagnosis
framework and real-time feedback closed-loop control system for FDM process.
3. Research Approach
3.1 Research Framework
The overall research approach of this work is summarized in Figure 1. It has the following five components: (1) high
speed data acquisition system design (Sec. 3.2); (2) textural feature extraction (Sec. 3.3); (3) online quality diagnosis
and evaluation, i.e. classification (Sec. 3.4); (4) closed-loop control and online adjustment strategy design (Sec. 3.5);
and (5) experimental testing and validation (Sec. 4).
Figure 1: Research framework
3.2 High Speed Data Acquisition System for AM Processes
The data acquisition system consists of multiple high resolution digital microscopes with high sampling frequency. In
order to avoid blind spots during data collection, the microscopes are installed around the extruder at different angles.
In our experimental setup, two digital microscopes are mounted in close proximity to the extruder of the FDM machine
as shown in Figure 2, and are capable of effectively monitoring the entire process.
3.3 Feature Extraction
Intuitively, for the investigated FDM process, the different types of surface defects can be identified by different
surface image textures. Since high resolution image data is able to keep much of the surface textural information, the
image texture analysis-based approach was applied in this work to implement the effective feature selection.
Liu, Roberson, Kong
3.3.1 Gray level co-occurrence matrix (GLCM)
Co-occurrence matrix, which is also referred to as co-occurrence distribution, is one means to represent the distance
and angular spatial relationship over an image sub-region of specific size [15]. In real-world applications, in order to
reduce the computational complexity, the gray level co-occurrence matrix (GLCM) [16] is usually applied, which is
transformed from gray-scale image. For a gray-scale image with dimension , the GLCM of , is a matrix
where the number of rows and columns is equal to the number of the gray levels in . The element of , ,
which is defined by the occurred frequency of a two-pixel combination with intensity and respectively, and the
spatial distance of these two pixel should be equal to . Suppose the number of gray level in is , then
(1)
where
(2)
Obviously, has multiple GLCMs by the number of gray level used and different setting of the spatial distance
. Equivalently, also can be represented as , where is the angle of direction and is the
distance. For example, is equal to
. In order to guarantee rotationally invariant in
feature extraction,
with a constant will be calculated as a group in this work, due to
.
Figure 2: Location of the digital microscopes
3.3.2 Feature vector construction
From a specific GLCM, a variety of statistics with textural interpretation can be obtained. Currently more than ten
types of statistical features have been proposed in literature [16, 17], and the following features are used in this work.
Contrast: Quantify the intensity contrast between pixel and the neighbor. Obviously, contrast is zero for a
constant image.
(3)
Correlation: Measurement for the correlation between a pixel and its neighbor. The range of correlation is
, and the boundary can be achieved only when the image is perfectly correlated. In particular, the
correlation measurement is meaningless for a constant image.
(4)
Energy: Energy is calculated by the sum of squares from GLCM, which is also known as angular second
moment (ASM). The range of energy is , and the value is equal to 1 for a constant image.
(5)
Homogeneity: The measurement for the closeness of the distribution of elements to the diagonal of GLCM,
and its range is also . When GLCM is diagonal, the value of homogeneity can achieve 1.
(6)
Liu, Roberson, Kong
3.4 Online Classification with Two Stages
In order to achieve the online adjustment, merely detecting the existence of defects is not enough, it is also necessary
to obtain the information about the type and severity of defects in real time, i.e. classification by image features.
However, training the classifier to identify the defect type and severity simultaneously may affect the reliability, due
to the unbalanced data, and also the optimal classification for different tasks may vary. Therefore, based on this
demand, due to the hierarchical relationship, this paper proposes a two-stage online classification framework to reduce
the rate of diagnosis error. At stage 1, by classifier 1, the aim is to distinguish the normal status or different types of
defects based on the real-time collected image. After that, to evaluate the severity of the defects will be the task of the
second stage via classifier 2 (see Figure 3).
Figure 3: The proposed two stage online classification framework
3.5 Closed-Loop Control System Feedback and Online Update
3.5.1 PID controller
Proportional–integral–derivative (PID) controller is a common kind of control-loop feedback mechanism widely
applied in industrial control systems. Nowadays it still can provide the simplest and most efficient solution to many
real-world control problems [18]. The working principle of the PID controller is to continuously update the control
variable , i.e., the adjustment value of the corresponding process parameter such as material feed rate from the
extruder in this study, including proportional, integral and derivative three components constructed by the error term
, which is the difference between the desired set-signal and the corresponding process variable measurement.
Mathematically it can be represented in the time domain as follows:
(7)
where , and are the tuning parameter corresponding to the proportional, integral gain and derivative terms
[19]. For the discrete-time system, the integral and derivative terms will be replaced by summation and difference
respectively. In practice, loop tuning for PID control usually is the most important problem, although it looks
conceptually intuitive. Manual tuning is a common option, and also there are some other well-developed tuning
methods which are good for some specific cases, such as Ziegler–Nichols method, Tyreus-Luyben method and Cohen-
Coon method, etc. [20]. Since this work will not consider complex control loops in the system, and the defects also
can be simulated by different parameter settings, the tuning parameters can be trained offline before online application.
3.5.2 Online parameter adjustment
Since the desired set-parameter can be assumed as a known constant and a rough real-time process parameter can be
estimated by online classification, the error term is able to be updated in real time. Also, the tuning parameters
of the controller can be trained by offline data. Therefore, the adjustment decision at time from controller, i.e. )
can be determined. In a real world application, in order to make the controller more robust, based on the property of
the process, an empirical upper/low bound of one-time adjustment can be specified. Using this bound as a
reference, the absolute value of final adjustment should be:
}
(8)
Figure 4: (a) Image acquisition system, (b) surface showing normal condition, (c) surface showing overfill defect
Liu, Roberson, Kong
4. Case Study
4.1 Experimental Setting and System Framework
In the case study, the experimental device (Figure 4) consists of the Hyrel System 30M printer and two digital
microscopes with illumination, which are fixed on two sides of the extruder with a ~45 degree inclination (see Figure
4a). The collected images have resolution and 15 times magnification. For online image acquisition, the
maximum frequency can achieve 1 Hz.
4.2 Quality Diagnosis
For the collected images, the quality diagnosis is based on image classification by textural feature vectors. In order to
evaluate the real time status, only the region of interest (ROI) is used, which is cropped from the small region below
the nozzle (see Figure 4b, 4c). In this study, two frequently occurred defects, i.e. overfill and underfill, with multiple
levels, were purposely generated using different feed rates, and then the training dataset can be collected. After testing
different classifier combinations at stage 1 and 2, using support vector machine (SVM) with RBF kernel [21] for both
classifier 1 and 2 presents the best performance. The 3-fold cross validation shows that this classifier can provide 80%
accuracy rate for stage 1 and 90% accuracy rate for stage 2. Since the mislabeled images may affect the decision of
controller, grouping the images which are captured from the same layer, and voting for the diagnosis result based on
individual classification, i.e. window-based diagnosis, is one of the solutions to further reduce the error rate in online
application. Also, the offline testing using the same classifier shows that the final accuracy rate can achieve 95%.
4.3 Online Feedback Control
For the online feedback control case study, we consider overfill and underfill as two separate cases, and the feed rate
parameter is considered to be the adjustable variable (with 100% feed rate being the manufacturer specified default
material extrusion amount). In a real process, small steady-state error may be allowed, for example, the surface quality
is also good in the case of 95% feed rate, i.e. slight underfill defect. Also, since there is a lag between parameter
change and machine response, the controller should have a fast rise time but small overshoot. Therefore, the
proportional and derivative parts are applied. Since adjustment is not continuous, the final controller can be simply
discretized as:
(9)
To verify the performance of the control system, we first allow normal printing of three layers, and then set the feed
rate as 50%, i.e. 50% underfill defect (see Figure 5b). We then run the feedback control system, with the minimal unit
for adjusting feed rate as 5%. The surface quality appears to return to normal after six additional layers of printing
(see Figure 5). Although steady-state error exists due to resolution (5%) of feed rate adjustment, the layer surface after
adjustment looks almost same as normal. Therefore, the result is acceptable and very promising.
Figure 5: (a) Controller performance, (b) initial surface (underfill defect), (c) surface after feedback adjustment
5. Conclusion and Future Direction
In this paper, we develop an image-based real-time process diagnosis with closed-loop control system for the FDM
process. The case study shows that this framework is effective for combating two very common defects in FDM, i.e.,
overfill and underfill, allowing the process to return to normal in a short time.
This paper is a preliminary work for online closed-loop control in AM, and the results show that proposed framework
is very promising. Therefore, future work and investigation along these lines is highly valuable to pursue, mainly in
three directions. First, since the current sensing system only has two microscopes, the coverage may possibly be
improved, as blind spots could exist which may affect the diagnosis results. Embedding a new image sensing device,
such as infrared thermal camera, may also improve the system performance. Second, additional image features and
Liu, Roberson, Kong
defect types should be investigated. As more types of defects are considered, the currently used features may not be
sufficient. Third, for the current feedback control system, PID controller still has a number of limitations. For example,
the optimal tuning parameter settings may vary at different layers if the part is complex, or the relationship between
defects and machine parameters may not be very clear. The response time is still not very fast. So, a further goal of
the ongoing work is to design a fuzzy logic controller [22] to overcome the limitations of the current control system.
On the other hand, the process optimization tools may be applied to the process output optimization by providing the
optimal setting of the input parameter [23].
References
1. Beyer, C., 2014, “Strategic implications of current trends in additive manufacturing,” Journal of
Manufacturing Science and Engineering, 136(6): 064701.
2. Standard, A., F2792. 2012. Standard Terminology for Additive Manufacturing Technologies, ASTM
F2792-10e1, 2012.
3. Gibson, I., D.W. Rosen, and B. 2010, Stucker, Additive manufacturing technologies, Vol. 238, Springer.
4. Peng, A. and X. Xiao, 2012, “Investigation on reasons inducing error and measures improving accuracy in
fused deposition modeling,” Advances in Information Sciences and Service Sciences, 4(5).
5. Agarwala, M.K., et al., 1996, “Structural quality of parts processed by fused deposition,” Rapid
Prototyping Journal, 2(4): 4-19.
6. Villalpando, L., H. Eiliat, and R. Urbanic, 2014, “An optimization approach for components built by fused
deposition modeling with parametric internal structures, ” Procedia CIRP, 17: 800-805.
7. Rao, P.K., et al., 2015, “Online real-time quality monitoring in additive manufacturing processes using
heterogeneous sensors,” Journal of Manufacturing Science and Engineering, 137(6): 061007.
8. Bastani, K., P.K. Rao, and Z. Kong, 2016, “An online sparse estimation-based classification approach for
real-time monitoring in advanced manufacturing processes from heterogeneous sensor data,” IIE
Transactions, 48(7): 579-598.
9. Wu, H., Y. Wang, and Z. Yu, 2016, “In situ monitoring of FDM machine condition via acoustic emission,”
The International Journal of Advanced Manufacturing Technology, 84(5-8): 1483-1495.
10. Kousiatza, C. and D. Karalekas, 2016, “In-situ monitoring of strain and temperature distributions during
fused deposition modeling process,” Materials & Design, 97: 400-406.
11. Fang, T., et al., 1998, “Online detection of defects in layered manufacturing. in Robotics and Automation,”
Proc. of the IEEE International Conference.
12. Fang, T., et al., 2003, “Signature analysis and defect detection in layered manufacturing of ceramic sensors
and actuators,” Machine Vision and Applications, 15(2): 63-75.
13. Cheng, Y. and M.A. Jafari, 2008, “Vision-based online process control in manufacturing applications,”
IEEE Transactions on Automation Science and Engineering, 5(1): 140-153.
14. Peng, A.H. and Z.M. Wang., 2010, “Researches into Influence of Process Parameters on FDM Parts
Precision,” Applied Mechanics and Materials, Trans Tech Publ.
15. Gotlieb, C.C. and H.E. Kreyszig, 1990, “Texture descriptors based on co-occurrence matrices,” Computer
Vision, Graphics, and Image Processing, 51(1): 70-86.
16. Peckinpaugh, S.H., 1991, “An improved method for computing gray-level cooccurrence matrix based
texture measures,” CVGIP: Graphical models and image processing, 53(6): 574-580.
17. Haralick, R.M. and K. Shanmugam, 1973, “Textural features for image classification,” IEEE Transactions
on systems, man, and cybernetics, 3(6): 610-621.
18. Ang, K.H., G. Chong, and Y. Li, 2005, “PID control system analysis, design, and technology,” IEEE
transactions on control systems technology, 13(4): 559-576.
19. Visioli, A., 2006, Practical PID control, Springer Science & Business Media.
20. Shahrokhi, M. and A. Zomorrodi, 2013, “Comparison of PID controller tuning methods,” Department of
Chemical & Petroleum Engineering Sharif University of Technology.
21. Cristianini, N. and J. Shawe-Taylor, 2000, “An introduction to support vector machines and other kernel-
based learning methods,” Cambridge university press.
22. Lee, C.-C., 1990, “Fuzzy logic in control systems: fuzzy logic controller,” IEEE Transactions on systems,
man, and cybernetics, 20(2): 419-435.
23. Mobin, M., M. Mobin, and Z. Li, 2016, “Multi-response optimisation of cavitation peening parameters for
improving fatigue performance using the desirability function approach,” International Journal of Applied
Decision Sciences, 9(2): 156-181.
