Intelligent systems for optical form measurement: automated assessment of pose and coverage

Full text

INTELLIGENT SYSTEMS FOR OPTICAL FORM MEASUREMENT:
AUTOMATED ASSESSMENT OF POSE AND COVERAGE
Sofia Catalucci1, Nicola Senin1,2, Samanta Piano1, Richard Leach1
1Manufacturing Metrology Team, Faculty of Engineering
University of Nottingham
Nottingham, United Kingdom
2Department of Engineering
University of Perugia
Perugia, Italy
INTRODUCTION
This work addresses the development of
intelligent and adaptive optical form
measurement systems for quality inspection of
additively manufactured complex parts. The
ultimate objective is to obtain smart optical
measurement systems capable of automatically
reconfiguring themselves while inspecting new
geometries, and capable of assessing whether
completed measurements are sufficient, or
further measurements should be performed.
Intelligent behaviour is achieved through
automated self-assessment of measurement
performance, while the measurement itself is
being executed [1]. The decisional process is
supported by multiple sources of information [2],
namely: knowledge of part specifications (CAD
model, dimensional and geometric tolerances,
materials); knowledge of the manufacturing
process and the material, leading to predictability
of likely types of form error; knowledge of the
measurement instrument itself (metrological
performance and behaviour), and how it is
expected to interact with any specific material and
part geometry. The optical measurement
technologies covered by the project produce
point clouds: the work presented in this paper
focuses on algorithmic processing of point
clouds, and deals with the following, specific
challenges: a) automated point cloud localisation
within the part geometry, i.e. identifying what
surfaces have been captured by any given point
cloud, acquired from a part of unknown position
and orientation; b) automated assessment of
coverage and sampling density for the exposed
surfaces, including recognition of critical regions
(i.e. poorly represented by the point cloud), in
order to support automated planning for further
measurements.
TEST SET UP
The experimental set-up is based on a
combination of a commercial measurement fringe
projection system (blue-light technology GOM
Atos Core 300), shown in Figure 1, and the point
cloud processing commercial software Polyworks
Inspector by Innovmetric. Automation is achieved
by interfacing Polyworks with MATLAB, via
scripting.
FIGURE 1. The optical measurement system
while measuring one of the test parts.
Test cases
The selected test measurement parts are shown
in Figure 2. Sample A (Figure 2a) was fabricated
by selective laser sintering (SLS) using Nylon 12,
with size of a rectangular enclosing envelope (50
× 50 × 28) mm; sample B (Figure 2b) was
fabricated by laser powder bed fusion (LPBF)
using stainless steel 316L, with dimensions of
(125 × 45 × 8) mm.

FIGURE 2. Test parts; a) Nylon 12 pyramid
sample (50 × 50 × 28) mm fabricated by SLS; b)
stainless steel 316L automotive sample (125 × 45
× 8) mm fabricated by LPBF.
The nominal geometries of the test parts are
available as triangle meshes. Example results of
single measurements on the test parts with
unknown pose are shown in Figure 3a for sample
A and Figure 3b for sample B.
FIGURE 3. Example measurements: a) sample
A; b) sample B.
As sample A has four nominally identical sides,
pose estimation only pertains to the accurate
identification of the angular orientation of the
visible corner in the point cloud.
DATA PROCESSING METHOD
The first data processing step consists of
detecting the pose by identification and best-
matching of landmark features present on both
the measured point cloud and the nominal
reference geometry (triangle mesh). In the
second step, once the point cloud has been
aligned to the mesh, the degree of coverage can
be assessed by identifying the surfaces that have
not been reached by the measurement
instrument. For the covered surfaces, the density
and spatial distribution of the measured points
can be computed by inspecting the positions of
the points falling within each triangle of the mesh.
STEP 1: ALIGNMENT
Alignment, also referred to as registration,
consists of a coarse phase and a fine phase.
Coarse registration
Coarse registration is based on the identification
and matching of common landmarks both in the
measured point cloud and in the triangle mesh.
Landmarks can be identified through computation
of local feature descriptors [3-5]. In this work,
local curvatures are used.
Surface normal vectors are identified both on the
point cloud and in the triangle mesh, by using
principal component analysis [6] on local subsets
of neighbouring points selected via the k-nearest
neighbour algorithm [7]. The principal curvatures
𝑘1 and 𝑘2 are then computed [8]. From the
principal curvatures, the Gaussian curvature K
and mean curvature H are computed as follows:
𝐾 = 𝑘1 ∙ 𝑘2,
(1)
𝐻 = (𝑘1 + 𝑘2)
2.
(2)
Example results for curvature are shown in
Figures 4 to 7.
The next step involves the identification of
clusters of points with similar curvature values: a
first k-means clustering process [9] was used to
identify k-classes of curvature values (k = 5). The
highest-curvature class was then isolated; the
resulting points were subjected to another
clustering process, this time aimed at isolating
spatially distant subsets of points with high-
curvature values. The second clustering was,
therefore, hierarchical and based on Euclidean
distances between points (Figures 8 to 11).
a) b)
a)
b)

FIGURE 4. Gaussian curvature K estimation on
extracted vertices of the triangle mesh (sample
B).
FIGURE 5. Mean curvature H estimation on
extracted vertices of the triangle mesh (sample
B).
FIGURE 6. Gaussian curvature K estimation on
point cloud dataset (sample B).
FIGURE 7. Mean curvature H estimation on point
cloud dataset (sample B).

FIGURE 8. k-means clustering on K curvature.
Cluster 2 refers to the extracted vertices of the
triangle mesh with the highest curvature values
(sample B).
FIGURE 9. Hierarchical clustering and centroids
computation of clustered extracted vertices of the
triangle mesh (sample B). The points taken into
account are the ones with the highest curvature
values.
FIGURE 10. k-means clustering on K curvature.
Cluster 2 refers to the points with the highest
curvature values (sample B).
FIGURE 11. Hierarchical clustering and centroids
computation of clustered point cloud (sample B).
The points taken into account are the ones with
the highest curvature values.

The identified common landmarks in both
datasets, described by high curvature values, are
then best-matched, using Random sample
consensus (RANSAC) [10,11]: at each iteration,
good matches were considered those resulting in
a spatial alignment which minimises the sum of
squared distances between matched points,
using the Procrustes algorithm [12].
Fine registration
Fine registration is based on a best-fit algorithm
[15], which iteratively minimises the distances
from the measured dataset to the reference
entity, revising the transformation based on a
rigid transformation (translation and rotation) until
the variation of the squared error is minimised.
The “registration error function” is defined as the
sum of squared Euclidean distances between
each point in the cloud and its closest neighbour
located on the triangular facets [13].
COVERAGE ASSESSMENT
After the fine registration process is completed,
each triangular facet belonging to the original
mesh will have a certain number of measured
points associated with it. Coverage expresses
how comprehensively each triangle is
represented by the associated measured points.
To assess coverage, the number of points falling
within each triangle is considered in relation to the
area of the triangle with the purpose to obtain a
measure of spatial sampling density, i.e. number
of points per unit area. Sampling density is
computed on all the triangles (Figure 12). Then, a
percentage of the maximum density is set as
threshold to discriminate between adequately
and inadequately covered triangles (simply
referred to as "uncovered"). Finally, a coverage
ratio can be defined as the percentage of
triangles with adequate coverage over the total
number of triangles in the mesh. Additionally, the
ratio between the total area occupied by triangles
classified as covered, and the total area of all the
triangles in the mesh, can be computed, and is
referred to as "covered area ratio".
Example results of coverage computation are
shown in Figures 13 to 14, where the threshold
has been set to 75% of the maximum sampling
density per triangle. The areal coverage is either
estimated based on the number of triangular
facets associated with measured points over the
total number of triangles, and the sum of the
covered area over the total area of the object
(Table 1).
FIGURE 12. Triangle facets; colouring
proportional to sampling density (sample B).
FIGURE 13. Covered and uncovered triangles for
sample A (threshold on sampling density at 75%).
FIGURE 14. Covered and uncovered triangles for
sample B (threshold on sampling density at 75%).

TABLE 1. Coverage ratio results.
No. of
triangles
in the
mesh
Coverage
ratio (%
covered
triangles)
Covered
area ratio
(%
covered
area)
Sample
A
1344
22%
32%
Sample
B
1785
39%
42%
CONCLUSIONS AND FUTURE WORK
In this paper, preliminary results from the early
stage development of an intelligent system for
complex shape measuring have been presented.
Methods and algorithms for the automatic
assessment of part pose and measurement
coverage have been introduced and discussed
with the support of two test cases. The prototype
implementation is realised using a combination of
commercial measurement hardware and
software, and custom software modules
developed in-house.
Future work will address: 1) the estimation of
uncertainty associated with alignment and
assessment of coverage. Alignment in particular
may be affected by problems of geometric
stability (e.g. see [14] for ICP); 2) the
differentiation of part surfaces depending on
functional relevance, so that assessment of
coverage quality can be weighed; 3) the
implementation of feedback mechanisms based
on the results of pose and coverage estimation,
to automate planning for further measurement
actions.
ACKNOWLEDGEMENTS
The authors would like to acknowledge Patrick
Bointon of the Manufacturing Metrology Team,
Leonidas Gargalis and Joe White of the Centre
for Additive Manufacturing, University of
Nottingham, for their assistance in designing and
producing the test cases. We also acknowledge
funding from EPRSC project EP/M008983/1.
REFERENCES
[1] Stavroulakis P, Chen S, Derlome C, Bointon
P, Tzimiropoulos G, Leach R K. Rapid
calibration tracking of extrinsic projector
parameters in fringe projection using
machine learning. Opt. Lasers Eng. 2019;
114: 7-14.
[2] Senin N, Leach R K. Information-rich
surface metrology. Proc. CIRP. 2018; 75:
19-26.
[3] Hana X F, Jin J S, Xie J, Wang M J, Jiang
W. A comprehensive review of 3D point
cloud descriptors. 2018; arXiv:1802.02297.
[4] Tombari F, Salti S, Di Stefano L. Unique
signatures of histograms for local surface
description. European conference on
computer vision. Springer; 2010. 356-369.
[5] Bellekens B, Spruyt V, Berkvens R, Penne
R, Weyn M. A benchmark survey of rigid 3d
point cloud registration algorithms. Inter.
Journ. Adv. Intell. Sysm. 2015.
[6] Chung D, Lee Y DS. Registration of multiple-
range views using the reverse-calibration
technique. Pattern Recogn. 1998; 31: 457-
464.
[7] Friedman J H, Bentely J, Finkel R A. An
Algorithm for Finding Best Matches in
Logarithmic Expected Time. ACM
Transactions on Mathematical Software.
1977; 3: 209-226.
[8] Merigot Q, Ovsjanikov M, Guibas L
J. Voronoi-based curvature and feature
estimation from point clouds. Visualization
and Computer Graphics, IEEE Transactions
on. 2011; 17: 743-756.
[9] Ding C, He X. K-means clustering via
Principal Component Analysis. Proceedings
of International Conference on Machine
Learning. 2004. 225-232.
[10] Moretti M, Gambucci G, Leach R K, Senin
N. Assessment of surface topography
modifications through feature-based
registration of areal topography data. Surf.
Topogr. Metrol. Prop. 2019; 7.
[11] Fishler M A, Bolles R C. Random sample
consensus: A paradigm for model fitting with
applications to image analysis and
automated cartography. Commun. ACM.
1981; 24: 381-95.
[12] Kendall D G. A survey of the statistical
theory of shape. Statistical Science. 1989; 4:
87-99.
[13] Besl P, McKay N. A method for registration
of 3-D shapes. IEEE TPAMI. 1992; 14: 239-
256.
[14] Gelfand N, Ikemoto L, Rusinkiewicz S,
Levoy M. Geometrically stable sampling for
the ICP algorithm. Proceedings of
International Conference on 3-D Digital
Imaging and Modeling, 3DIM. 2003.