Automated Surface Inspection of Micro Parts
, Hendrik Thamer
, Michael Lütjen
BIBA – Bremer Institut für Produktion und Logistik GmbH at the University of Bremen
Hochschulring 2, D-28359 Bremen, Germany
Abstract. This chapter presents a machine vision system for detecting surface imperfections on
micro parts. It is part of a quality control concept for micro production. Because of increasing
product miniaturization, the mechanical manufacturing of micro components is becoming more
and more important. The combination of high manufacturing rates and low tolerances in
manufacturing processes enables the economical production of micro components. Due to the
small component sizes and the difficulties associated with the handling process, the manual
visual inspection retires as testing procedure. A customized surface inspection technology with
an efficient image processing and classification system is needed. The objective of our concept
is to identify surface imperfections such as raisings, laps and bulges on micro parts. The
implementation of the system is explained by reference to a micro deep-drawn component,
which is manufactured within the German Collaborative Research Center (CRC) 747.
Keywords: Surface Inspection, Micro Production, Image Processing, Machine Vision, Quality
PACS: 07.05.Pj, 06.30.Bp
Micro technology is one of the most important cross-sectional technologies and the
trend of miniaturization will outlast the next decades . Therefore, the importance of
automation of micro-manufacturing processes is still growing to produce mechanical
components for micro systems. The big challenge in the development of high-
performance micro-manufacturing processes is the balance between accuracy and
efficiency. Because of this conflict, the process configuration is extremely complex
and time consuming. Building and setting-up manufacturing stations takes much more
time than the actual production period of a few weeks . Due to the high
manufacturing rates of deep drawing micro processes with more than 300 parts per
minute, quality control can only be realized by industrial machine vision techniques.
The purpose of quality control includes the identification of faulty workpieces and the
manufacturing process reconfiguration by use of the quality control data.
The objective of this chapter is to present an automated surface inspection system
for micro parts, which is part of a quality control concept. It focuses on the
development of the image processing and classification system, which is based on the
prototypical software implementation “SInsMicro – Surface Inspection of Micro
First of all, the methods and principles for quality control in micro production are
described. In the next section, the developed quality control concept is demonstrated
Parts” from Scholz-Reiter et al. .
by reference to a micro cup, which is manufactured within the CRC 747. In order to
obtain 3D information about the surface structure of the micro part, the images of the
micro parts are acquired by a confocal laser microscope. These images are called
height maps. In the following section, the surface inspection system “SInsMicro” is
presented which uses a sequence of several classic image processing techniques for
identifying regions of surface imperfections in the height map. After determining these
so-called error regions, we specify the kind of imperfection by the use of a
classification system. The image processing and the classification system are
evaluated in the experiment section. The chapter ends with a conclusion and an
outlook on further research activities.
QUALITY CONTROL IN MICRO PRODUCTION
To achieve the efficient production of micro parts, the application and development
of suitable quality control methods is required. Micro parts are used in many different
areas of application such as electronic devices, medical equipment, sensor technology
and optoelectronics , .
According to the definition of the CRC 747, a micro part has more than two
dimensions less than one millimeter and tolerances of a few microns. Such micro parts
are components of micro systems like e.g. valves, gears and manipulators. Micro
manufacturing shows different and new problems in comparison to macro
manufacturing. However, it is not possible to apply methods used for macro products
to micro products . The main characteristic of micro manufacturing is based on so-
called “size effects” , . These effects prevent the transfer of quality control
methods known from the macro level to the micro level. Vollertsen described in 
eight different types of size effects, which can be divided into three causal categories
of “density”, “geometric shape” and “micro structure”. Due to the use of such thin
materials the grain orientation dominates the material behavior locally and causes
random deviations of material parameters. As a consequence for micro manufacturing,
surface defects also occur in stable processes because they are not predictable.
Therefore, methods of process planning, control and quality management have to be
adapted to the specific characteristics of micro manufacturing.
According to DIN EN ISO 9000, the term “quality management” is defined as the
result of coordinated activities which implement the quality policy of the company.
Quality management includes the activities of quality planning, quality control, quality
assurance and quality improvement. During the quality planning of micro components
the establishment of manufacturing tolerances for shape deviations is of special
importance. These tolerances are in the sub-micron range. Due to the dimension
metrology plays an important role as it is the only access to the micro world. For the
geometrical inspection of micro parts many different measuring systems exist which
are mostly tailored to one specific task. Additionally, the demands for an inspection
system in micro technology are various . For instance, the ratio of manufacturing
tolerance to measurement uncertainty is significantly reduced compared to the macro
production. In addition, the ability to scan micro parts entirely three-dimensionally is
essential. Due to its flexibility the confocal laser microscope is an inspection system
which is often used in quality control. It permits not only the three-dimensional shape
acquisition but also the characterization of micro structures . Conventional
confocal microscopes are suitable for visualizing and analyzing surfaces. In order to
inspect complex micro parts, the component is scanned from different views. A
confocal laser microscope measures surfaces by means of the time-of-flight principle.
By drawing on several cuts in different focal planes, different layers of the micro
component are obtained and a three-dimensional computer reconstruction of the
investigated object can be created. This means that the same micro component can be
subjected to multiple investigations in a time series from a number of orientations
Another suitable measurement method for micro system inspection is the Digital
Holographic Microscopy (DHM). Its main application is the three-dimensional
interferometric deformation measurement. The main advantage of the DHM over the
confocal laser microscopy is the easy and fast processing from recording to
reconstruction and evaluation .
QUALITY INSPECTION OF A MICRO CUP
In the following section, the development of a surface inspection system is
demonstrated. It is based on micro parts which are manufactured within the CRC 747
. One of the manufactured micro parts is the so-called micro cup. The micro cup
has a diameter of approximately 1 millimeter with a sheet thickness of about 20
microns . The micro cup is the result of a micro deep-drawing process. Micro
deep-drawing is defined as the forming of a sheet metal part using pressure, so that a
hollow is created, which is open on one side. A stamp is mostly used for this purpose.
Micro cups or sleeves with a diameter of one millimeter or smaller can be produced by
micro deep-drawing. In comparison to macro deep-drawing, there are still small
crinkles on the flange of the micro part. Figure 1 compares the results of a
manufacturing process between a macro- and a micro cup.
FIGURE 1. Comparison of micro and macro deep drawing cups 
The objective of the sub project B5 of the CRC 747 is to provide quality control
methods for the manufacturing of deep drawn micro parts. One constituent of quality
control is the surface inspection. According to VDI/VDE 2601, it includes in principle
the determination of the condition of the surface. DIN EN ISO 8785 describes and
classifies over 20 different types of surface imperfections such as grooves, scratches,
cracks, pores and outgrowths. Surface inspection is one of the most complex and
challenging problems in micro quality inspection. Besides the large number of surface
characteristics also brightness variations, reflections etc. can occur .
In particular, deep-drawn parts cause many challenges for optical 3D-metrology. It
is difficult to explore the whole angled geometry because the metal surface shines and
the light waves are reflected. Another problem is the occurrence of tiny impurities of
the material or the tool, which can lead to fractures, blisters and cracks on the micro-
cup during manufacture. In order to cope with these challenges, a comprehensive
quality control concept is required for micro deep-drawing. A particular challenge for
the metrology and image processing system, which is investigated by the CRC 747, is
the realization of surface inspection with planned production rates of more than 300
parts per minute and throatiness characteristics up to 0.4 microns. For this reason,
extremely fast testing routines are required. One suitable optical 3D-measurement
technique is the DHM which is also developed in the CRC 747. In order to develop the
image processing system, the 3D-height maps acquisition is realized by the laser
scanning microscope Keyence VK-9700. It also generates 3D-height maps, but takes
much more time and the images have to be stitched together. The wavelength of the
VK-9700 laser unit is at 408 nm and in the violet range. The other technical data of the
microscope is shown in table 1.
TABLE 1. Technical Data Keyence VK-9700 
Measurement Range (vertical)
Complete Measurement Range
67 μm (highest enlargement)
1x to 6x
Photoelectron Multiplier Tube
The 3D surface information is stored in a height map. In order to identify surface
imperfections in the image, a sequence of image processing techniques is applied.
After detecting such regions, the type of surface imperfection is classified by a
machine learning procedure. Then it can be decided whether the micro cup is defect or
not. Additionally, the results of the 3D-geometry measurement have to be considered.
A corresponding analysis is also done in the CRC 747. Figure 2 shows the complete
quality control loop which involves the loop from the manufacturing process of the
micro cup to image acquisition and processing to the reconfiguration of the
manufacturing process in the case of unacceptable defect rates.
FIGURE 2. Quality control loop
In the following sections, the image processing sequence and the classification of
surface imperfections are described in more detail. For information about the
reconfiguration of the manufacturing process see Scholz-Reiter et al. . The
developed software prototype is based on the Image Processing Toolbox of MATLAB
. The main graphical user interface of the software SInsMicro is illustrated in
FIGURE 3. Screenshot of the main GUI of SInsMicro
IMAGE PROCESSING SEQUENCE
Industrial quality inspection is an important application domain of 3D computer
vision methods. Traditional vision-based industrial quality inspection systems
primarily rely on two-dimensional detection and pose estimation algorithms like the
detection of point and line features . One of the most common image processing
methods used to recognize a specific object type on a surface is structural analysis
. These textural features are calculated directly from the image information or on
the basis of a histogram captured from the source image . To detect surface
imperfections on the micro cup these image processing techniques are applied on the
height maps. Because of the angled geometry of the micro cup, the laser microscope
has to record multiple images to reconstruct the whole cup in three dimensional height
maps. The pixel intensity of the height maps describes the distance from the laser unit
to the micro object. Figure 4 illustrates the micro cup and the corresponding height
map. The black highlighted area shows a manufacturing error. In this case, a lap is
visible on the surface. The image acquisition process is influenced by measurement
noise, which is caused by reflections.
FIGURE 4. (a) Image of the micro cup (b) Corresponding height map of the micro cup
Based on these height maps acquired by the confocal laser microscope, we
developed an automated surface inspection system. For the automatic detection of
manufacturing defects several image processing methods are used, which are
presented in detail in the following. The image processing is demonstrated by the
usage of the height map from figure 4. The different steps of image processing are
illustrated in figure 5 and will be illustrated in the following.
FIGURE 5. Steps of image processing
The objective of the segmentation step is to distinguish between regions in the
height map, which represents the background and regions which belong to the micro
cup. The background is characterized by a high noise ratio. Therefore, the
segmentation is realized by applying a neighborhood operation to the height map. This
operation uses a pattern that has a fixed size and is normally a square with a centre
point. The value of this point depends on its neighborhood and is computed by a
predefined function. In this case, the standard deviation of the neighborhood is
computed and stored in the point. Image areas containing noisy background
information obtain high values for the standard deviation. Regions which represent the
object are characterized by a small standard deviation. The results of the neighborhood
operation are stored in a new matrix (figure 5b), which has the same dimension as the
source height map.
Afterwards, a threshold value is defined to distinguish between object and
background. The value of the threshold is defined by using Otsu´s method . Every
pixel that has a value higher than the threshold is defined as background and
represented by the value 0 in the image. In the other case, pixel values which are
smaller than the threshold belong to the object and are represented by the value 1.
Thereby, a segmentation of the image is realized (figure 5c).
Before surface defects can be properly detected, the influence of noise that affects
the micro cup part of the image has to be reduced. For this reason, smoothing filters
from conventional two-dimensional image processing techniques are applied. These
filters can be easily adapted to three-dimensional images . To reduce measurement
noise a median filter is applied to those areas, which are defined as part of the object
in the previous segmentation step. The median filter belongs to the class of ranking
filters. The values of the pixels in a defined area of a pixel are collected, sorted and
placed in a hierarchy according to their size. Afterwards, the median in the sorted list
is selected and replaces the value of the pixel.
Surface imperfections are characterized by abrupt changes in height information.
These features can be detected by edge detection algorithms. The aim of the edge
detection step is to detect continuous edges which are not identical to the edge
between object and background. This is performed by applying edge enhancing
operators. These operators are also called high-pass filter. They suppress low spatial
frequencies, high frequencies are enforced. Thereby, image areas where adjacent
pixels have different pixel values are emphasized . For this purpose, many
different filter algorithms exist. The software SInsMicro includes the possibility to
change the detection algorithm. Supported algorithms or operators are
• Sobel Operator
• Canny Algorithm
• Prewitt Operator
• Roberts Operator
Details about the working principle of the different algorithms can be found in .
Another possibility to influence the resulting edges is adjusting the threshold value of
the edge detection algorithm. The threshold value defines the required difference
between the pixel values from the edge and the environment. The optimal threshold is
defined in the learning process, which is described below in the parameter adjustment
The resulting edges have to be classified as surface imperfection when they satisfy
specified requirements. The result of the application of these requirements is presented
in figure 6. The rectangle defines an error region. In our implementation, an error
region is defined as a region where the edge describes a continuous pathway and the
pathway borders a region with a specific size. The required size of the rectangle can be
defined in SInsMicro. In the experiment runs, the region has to include 120 pixels.
FIGURE 6. Region with a surface imperfection.
For simplification, only edges which are inside the object area are concerned. Edges
from the object to the background are not considered. However, applying the edge
detection algorithm involves the detection of these edges. Therefore, morphological
operators are applied on the segmented and binary image (figure 5c). To remove all
edges which represent the border to the background, the erosion operation is executed.
Eq.1 describes the application of the morphological operator.
In contrast to a common matrix multiplication, the multiplication is performed
element by element. Thereby, the size of the object is reduced and possible surface
imperfections situated in these edges are also cleared away. The detection of surface
imperfections near by the edges between object and background is one of the further
The choice of the edge detection algorithm and the edge threshold are the two
parameters for affecting the result of the image processing step. The different tilting of
the micro cups in the image acquisition process causes a different threshold value.
Additionally, each edge detection algorithm supplies different results. The values of
the parameters are determined experimentally. Therefore, height maps with obvious
surface imperfections are used and the suitable error regions are located. SInsMicro
starts the image processing sequence with default parameter settings. After the
sequence, the result of the computation is compared with the predefined surface
imperfection regions. When they do not match, the image processing sequence is
started once again with adjusted parameters. The parameter adjustment stops when the
resulting surface region matches the predefined region. The computation is executed
for each edge detection algorithm. Finally, the best parameter settings and edge
detection algorithm are taken. The resulting parameter settings are described in the
following experiment section.
The experiment section is divided into three parts. At first, the results of the
parameter adjusting are described. Afterwards, the results of the image processing step
by use of the experimentally defined parameters are presented. Finally, the results of
the classification process are described. The experiments are performed by the use of
100 manufactured micro cups. The corresponding height maps are generated by a
confocal laser microscope. Each micro cup is scanned from three different positions.
Therefore, 300 height maps are analyzed by the software SInsMicro.
Adjusting the parameters
The threshold adjustment is computed for every edge detection algorithm by the
means of predefined surface imperfection regions. The best solution is taken as
threshold and detection algorithm choice. Figure 7 illustrates the adjusting principle. A
height map containing a surface imperfection is used for adjusting the threshold. The
region of the imperfection is defined and then the threshold is adjusted until the
computed region matches the predefined region.
FIGURE 7. Comparison test of predefined and computed error regions
Obvious surface imperfections are visible in 8 of 300 height maps. For each height
map the best algorithm and threshold should be computed. The most frequent edge
detection algorithm and the corresponding average threshold value are taken over in
the settings of SInsMicro.
A problem which can affect the correctness of the computation is the position of
the micro cup. As mentioned before, the different tilting of the micro cups during the
laser scanning process causes different gradients in the height maps. Therefore, an
edge which represents a surface imperfection requires a higher threshold value. The
positions of 3 of the 8 height maps are similar to each other. Hence, the parameter
adjusting is only performed on these height maps. Table 2 shows the results for the
threshold definition and the corresponding matching rate.
TABLE 2. Results of parameter adjusting.
The canny algorithm shows the highest matching rate and is set as edge detection
The evaluation of the surface imperfection detection system is performed by
applying the image processing sequence to the 300 height maps. Afterwards, the
average time needed for complete region detection, the number of detected
imperfection regions and the number of correctly detected imperfection regions is
recorded. Therefore, the predefined imperfection regions from the parameter adjusting
experiment are taken for comparison with the detected regions. A surface imperfection
region is detected correctly when it matches the predefined region to 80%. Table 3
shows the results of the image processing evaluation.
TABLE 3. Results of the image processing experiments
Average number of detected
regions per height map
Correctly identified regions
Average time [s]
The reason for the very low imperfection detection rate is the different position and
tilting of the micro cup in all height maps. Therefore, the threshold value is not
suitable for each image. The detected regions which do not represent a surface
imperfection should be rejected in the classification step.
The correct classification of surface imperfections is an important step in the
process of quality control. In order to adjust suitable process and machine settings, the
process quality has to be measured in terms of occurring surface imperfections.
Typical process failures exist which lead to characteristic surface imperfections.
Therefore, the quick and correct classification enables the implementation of a faster
quality control. Using the example of the micro cup, we developed an image
processing sequence for the classification of surface imperfections corresponding to
DIN EN 8785. For that reason we tried out two different image sizes and several
In order to reduce the set of images for the learning sequence and obtain a better
classification rate, the number of surveyed characteristics has to be strictly limited.
Therefore, we focused on the shape of the surface imperfection and left out the height
information. Based on the detection of a surface imperfection, we created so called
standard failure images (SFI). Thus, we cut out the concerning area of the surface
imperfection, colored it white and filled in the background with black. Then we
rotated the image so that the longest failure side reaches from left below to right above
and the image becomes square. Finally, we resized the image to 7x7 or 14x14 Pixels.
The image processing sequence is illustrated in figure 8.
FIGURE 8. Creation of standard failure images: a) height map, b) rotated binary image, c) 7x7 pixels
standard failure image
The feasibility of classification was examined using a test set of 25 real world
failure images which were detected on the micro cups. These images were classified
manually in the main four characteristic failure classes which occurred on the micro
cups: blister, bulge, lap and raising. As we required more images, we used a training
set of 100 synthetic standard failure images which contained 25 images for each
failure class. The objective of the classification was to obtain a high classification rate
for the real world test set. We tested it on WEKA software with sizes of 7x7 and
14x14 pixels for the SFI and 4 different classifiers. WEKA is a collection of machine
learning algorithms for data mining tasks. The machine learning algorithms are also
called classifiers. The results of classification are illustrated in Table 4. The
classification results show that a classification rate of 76% can be achieved, which is
an acceptable classification rate for the detection of systematic failures. The leave-
one-out cross-validation of the training set shows a problem with the low quantity of
training data. Further analysis shows that especially the correct classification of
raising-failures is very difficult. This refers to the definition of raising-failures and to
the correct manual classification of it. The shapes of raising-failures differ immensely.
Also, the enlargement of the SFI-size was no improvement for the classification.
Especially the fourfold amount of the pixels in relation to the small training set were
too many characteristics for the classifiers.
TABLE 4. Results of classification.
Pixel size of SFI
Type of classifier
LOOCV for training
Evaluation on test set
Ultimately, the classification concept should become better when the implementation
and integration of the surface inspection process into the demonstrator platform is
accomplished. The amount of data will increase significantly and on consequence the
classifiers will provide better results. Moreover, the results show that the classification
process is also a big option for the early development phases of the manufacturing
process when only few data exists and the process is unstable.
CONCLUSION AND OUTLOOK
The mechanical mass production of micro components requires efficient image
processing systems in order to meet the stringent quality requirements. To draw
conclusions about the surface inspection of micro components, many different
standards and applications were presented. Afterwards, the developed software
SInsMicro for the surface inspection of the micro-cup was described. It uses
conventional image processing techniques and machine vision operations which are
applied on high-resolution 3D-height maps in order to detect surface imperfections.
The implementation of this approach shows that defective surface regions can be
successfully identified. The correct identification depends on the position of the micro
cup, when the height map is acquired by the confocal laser microscope. If the position
of the micro cup differs too much, the surface imperfection cannot be detected
correctly. The thresholds for the image processing parameter do not fit. This problem
can be solved in two ways. At first, the software can be extended by an image
registration step. Thereby, the height map is mapped to a reference height map which
represents the micro cup in a defined position. Therefore, the detection rate of the
software system increases. The second possible way refers to the handling of micro
parts, which is one big challenge in micro production. By applying suitable micro
handling techniques, the micro cup could be placed in similar position and the defined
threshold value for edge detection could be used for all height maps in the same way.
The two-stage classification of the surface imperfection allows a further statistical
analysis of manufacturing processes, as systematic failures can be identified easily and
appropriate quality improvement measures can be initiated. The objective of the
subsequent project steps is the application of image registration techniques in order to
enhance the detection rate of surface imperfections. Also 3D-geometrical shape
measurement will be integrated in the evaluation of the micro parts.
The authors gratefully acknowledge the financial support by DFG (German
Research Foundation) for Subproject B5 "Sichere Prozesse" within the SFB 747
(Collaborative Research Center) „Mikrokaltumformen - Prozesse, Charakterisierung,
1. H. Reimer, “BMBF: Die Hightech-Strategie für Deutschland” in Datenschutz und Datensicherheit -
DuD, 2006, pp. 665-666.
2. B. Scholz-Reiter, M. Lütjen and N. Brenner, “ Technologieinduzierte Wirkungszusammenhänge in
der Mikroproduktion – Entwicklung eines Modellierungskonzepts” in 22. HAB-Forschungsseminar
Digital Engineering, Magdeburg Germany, 2009.
3. B. Scholz-Reiter, H. Thamer and M. Lütjen, “Optical Quality Assurance in Micro Production” in
Lecture Notes in Engineering and Computer Science: Proceedings of The International
MultiConference of Engineers and Computer Scientists 2010, IMECS 2010, 17-19 March, 2010,
Hong Kong, pp. 1521-1525.
4. A.Gillner, D. Hellrung, A. Bayer, F. Schepp and R. Erhardt, “Miniaturisierung von Bauteilen und
Komponenten“ in 7. Umformtechnisches Kolloquium Darmstadt, 2000, pp. 107-116.
M. Geiger, M. Kleiner, R. Eckstein, N. Tiesler and U. Engel, “Microforming“ in 51st General
Assembly of CIRP, Nancy, 50/2, 2001, pp. 445-462.
6. J. Fleischer, G. Lanza, M. Schlipf and I. Behrens, “Quality assurance in micro production“ in
Microsystem Technologies, 2006, pp.707-712.
7. K. Zhang and L. Kun,”Classification of size effects and similarity evaluating method in micro
forming” in Journal of Materials Processing Technology 209 (2009) 11, 2009, pp. 4949-4953.
8. X. Lai, L. Peng, P. Hu, S. Lan and J. Ni, “Material behavior modeling in micro/meso-scale forming
process with considering size/scale effects” in Computational Materials Science 43 (2008) 4, pp.
9. F. Vollertsen, “Categories of size effects” in Production Engineering 2 (2008) 4, pp. 377-383.
10. T. Pfeifer, S. Driessen and G. Dussler, “Process observation for the assembly of hybrid micro
systems” in Microsystems Technologies 10 (2004), pp. 211-218.
11. T. Wilson, Confocal Microscopy. London: Academic Press, 1996.
12. B. Tata and B. Ray, Confocal laser scanning microscopy: Applications in Material Science and
technology in Material Science, vol. 21, 1998, pp. 263-278.
13. W. Jüptner, C. von Kopylow and C. Falldorf, “Digital Holography and its application for
microsystems inspection“ in Optoelectronics letters vol. 4, 2008.
14. Collaborative Research Center 747. Available: http://www.sfb747.uni-bremen.de/welcome-to-the-
15. F. Vollertsen, Z.Hu, H. Schulze Niehoff and C. Theiler, “State of the art in micro forming and
investigations in micro deep drawing“ in Journal of Materials Processing Technology 151, 2004,
16. R. Louban, Image Processing of Edge and Surface Defects, Berlin-Heidelberg: Springer, 2009.
17. Keyence 3D Laser Scanning Microscope (VK-9700 SERIES) Specifications. Available:
18. B. Scholz-Reiter, M. Lütjen, D. Lappe, H. Thamer and N. Brenner, “Logistische Qualitätslenkung in
der Mikrokaltumformung“ in Industrie Management 4-2010.
19. MATLAB - Image Processing Toolbox, Available: http://www.mathworks.de/products/image/
20. C. Wöhler, 3D Computer Vision – Efficient Methods and Applications, Berlin-Heidelberg: Springer-
21. B.Jähne, Digitale Bildverarbeitung, Berlin-Heidelberg: Springer-Verlag, 1997
22. N. Otsu, “A threshold selection method from grey level histograms” in IEEE Transactions on
Systems, Man, and Cybernetics, New York, 1979, pp.62–66.
23. J. Toriwaki and H. Yoshida, Fundamentals of Three-Dimensional Image Processing, London:
24. J. Steinmüller, Bildanalyse, Berlin Heidelberg: Springer Verlag, 2006.