Content uploaded by Sofia Catalucci

Author content

All content in this area was uploaded by Sofia Catalucci on Nov 15, 2019

Content may be subject to copyright.

INTELLIGENT SYSTEMS FOR OPTICAL FORM MEASUREMENT:

AUTOMATED ASSESSMENT OF POSE AND COVERAGE

Sofia Catalucci1, Nicola Senin1,2, Samanta Piano1, Richard Leach1

1Manufacturing Metrology Team, Faculty of Engineering

University of Nottingham

Nottingham, United Kingdom

2Department of Engineering

University of Perugia

Perugia, Italy

INTRODUCTION

This work addresses the development of

intelligent and adaptive optical form

measurement systems for quality inspection of

additively manufactured complex parts. The

ultimate objective is to obtain smart optical

measurement systems capable of automatically

reconfiguring themselves while inspecting new

geometries, and capable of assessing whether

completed measurements are sufficient, or

further measurements should be performed.

Intelligent behaviour is achieved through

automated self-assessment of measurement

performance, while the measurement itself is

being executed [1]. The decisional process is

supported by multiple sources of information [2],

namely: knowledge of part specifications (CAD

model, dimensional and geometric tolerances,

materials); knowledge of the manufacturing

process and the material, leading to predictability

of likely types of form error; knowledge of the

measurement instrument itself (metrological

performance and behaviour), and how it is

expected to interact with any specific material and

part geometry. The optical measurement

technologies covered by the project produce

point clouds: the work presented in this paper

focuses on algorithmic processing of point

clouds, and deals with the following, specific

challenges: a) automated point cloud localisation

within the part geometry, i.e. identifying what

surfaces have been captured by any given point

cloud, acquired from a part of unknown position

and orientation; b) automated assessment of

coverage and sampling density for the exposed

surfaces, including recognition of critical regions

(i.e. poorly represented by the point cloud), in

order to support automated planning for further

measurements.

TEST SET UP

The experimental set-up is based on a

combination of a commercial measurement fringe

projection system (blue-light technology GOM

Atos Core 300), shown in Figure 1, and the point

cloud processing commercial software Polyworks

Inspector by Innovmetric. Automation is achieved

by interfacing Polyworks with MATLAB, via

scripting.

FIGURE 1. The optical measurement system

while measuring one of the test parts.

Test cases

The selected test measurement parts are shown

in Figure 2. Sample A (Figure 2a) was fabricated

by selective laser sintering (SLS) using Nylon 12,

with size of a rectangular enclosing envelope (50

× 50 × 28) mm; sample B (Figure 2b) was

fabricated by laser powder bed fusion (LPBF)

using stainless steel 316L, with dimensions of

(125 × 45 × 8) mm.

FIGURE 2. Test parts; a) Nylon 12 pyramid

sample (50 × 50 × 28) mm fabricated by SLS; b)

stainless steel 316L automotive sample (125 × 45

× 8) mm fabricated by LPBF.

The nominal geometries of the test parts are

available as triangle meshes. Example results of

single measurements on the test parts with

unknown pose are shown in Figure 3a for sample

A and Figure 3b for sample B.

FIGURE 3. Example measurements: a) sample

A; b) sample B.

As sample A has four nominally identical sides,

pose estimation only pertains to the accurate

identification of the angular orientation of the

visible corner in the point cloud.

DATA PROCESSING METHOD

The first data processing step consists of

detecting the pose by identification and best-

matching of landmark features present on both

the measured point cloud and the nominal

reference geometry (triangle mesh). In the

second step, once the point cloud has been

aligned to the mesh, the degree of coverage can

be assessed by identifying the surfaces that have

not been reached by the measurement

instrument. For the covered surfaces, the density

and spatial distribution of the measured points

can be computed by inspecting the positions of

the points falling within each triangle of the mesh.

STEP 1: ALIGNMENT

Alignment, also referred to as registration,

consists of a coarse phase and a fine phase.

Coarse registration

Coarse registration is based on the identification

and matching of common landmarks both in the

measured point cloud and in the triangle mesh.

Landmarks can be identified through computation

of local feature descriptors [3-5]. In this work,

local curvatures are used.

Surface normal vectors are identified both on the

point cloud and in the triangle mesh, by using

principal component analysis [6] on local subsets

of neighbouring points selected via the k-nearest

neighbour algorithm [7]. The principal curvatures

𝑘1 and 𝑘2 are then computed [8]. From the

principal curvatures, the Gaussian curvature K

and mean curvature H are computed as follows:

𝐾 = 𝑘1 ∙ 𝑘2,

(1)

𝐻 = (𝑘1 + 𝑘2)

2.

(2)

Example results for curvature are shown in

Figures 4 to 7.

The next step involves the identification of

clusters of points with similar curvature values: a

first k-means clustering process [9] was used to

identify k-classes of curvature values (k = 5). The

highest-curvature class was then isolated; the

resulting points were subjected to another

clustering process, this time aimed at isolating

spatially distant subsets of points with high-

curvature values. The second clustering was,

therefore, hierarchical and based on Euclidean

distances between points (Figures 8 to 11).

a) b)

a)

b)

FIGURE 4. Gaussian curvature K estimation on

extracted vertices of the triangle mesh (sample

B).

FIGURE 5. Mean curvature H estimation on

extracted vertices of the triangle mesh (sample

B).

FIGURE 6. Gaussian curvature K estimation on

point cloud dataset (sample B).

FIGURE 7. Mean curvature H estimation on point

cloud dataset (sample B).

FIGURE 8. k-means clustering on K curvature.

Cluster 2 refers to the extracted vertices of the

triangle mesh with the highest curvature values

(sample B).

FIGURE 9. Hierarchical clustering and centroids

computation of clustered extracted vertices of the

triangle mesh (sample B). The points taken into

account are the ones with the highest curvature

values.

FIGURE 10. k-means clustering on K curvature.

Cluster 2 refers to the points with the highest

curvature values (sample B).

FIGURE 11. Hierarchical clustering and centroids

computation of clustered point cloud (sample B).

The points taken into account are the ones with

the highest curvature values.

The identified common landmarks in both

datasets, described by high curvature values, are

then best-matched, using Random sample

consensus (RANSAC) [10,11]: at each iteration,

good matches were considered those resulting in

a spatial alignment which minimises the sum of

squared distances between matched points,

using the Procrustes algorithm [12].

Fine registration

Fine registration is based on a best-fit algorithm

[15], which iteratively minimises the distances

from the measured dataset to the reference

entity, revising the transformation based on a

rigid transformation (translation and rotation) until

the variation of the squared error is minimised.

The “registration error function” is defined as the

sum of squared Euclidean distances between

each point in the cloud and its closest neighbour

located on the triangular facets [13].

COVERAGE ASSESSMENT

After the fine registration process is completed,

each triangular facet belonging to the original

mesh will have a certain number of measured

points associated with it. Coverage expresses

how comprehensively each triangle is

represented by the associated measured points.

To assess coverage, the number of points falling

within each triangle is considered in relation to the

area of the triangle with the purpose to obtain a

measure of spatial sampling density, i.e. number

of points per unit area. Sampling density is

computed on all the triangles (Figure 12). Then, a

percentage of the maximum density is set as

threshold to discriminate between adequately

and inadequately covered triangles (simply

referred to as "uncovered"). Finally, a coverage

ratio can be defined as the percentage of

triangles with adequate coverage over the total

number of triangles in the mesh. Additionally, the

ratio between the total area occupied by triangles

classified as covered, and the total area of all the

triangles in the mesh, can be computed, and is

referred to as "covered area ratio".

Example results of coverage computation are

shown in Figures 13 to 14, where the threshold

has been set to 75% of the maximum sampling

density per triangle. The areal coverage is either

estimated based on the number of triangular

facets associated with measured points over the

total number of triangles, and the sum of the

covered area over the total area of the object

(Table 1).

FIGURE 12. Triangle facets; colouring

proportional to sampling density (sample B).

FIGURE 13. Covered and uncovered triangles for

sample A (threshold on sampling density at 75%).

FIGURE 14. Covered and uncovered triangles for

sample B (threshold on sampling density at 75%).

TABLE 1. Coverage ratio results.

No. of

triangles

in the

mesh

Coverage

ratio (%

covered

triangles)

Covered

area ratio

(%

covered

area)

Sample

A

1344

22%

32%

Sample

B

1785

39%

42%

CONCLUSIONS AND FUTURE WORK

In this paper, preliminary results from the early

stage development of an intelligent system for

complex shape measuring have been presented.

Methods and algorithms for the automatic

assessment of part pose and measurement

coverage have been introduced and discussed

with the support of two test cases. The prototype

implementation is realised using a combination of

commercial measurement hardware and

software, and custom software modules

developed in-house.

Future work will address: 1) the estimation of

uncertainty associated with alignment and

assessment of coverage. Alignment in particular

may be affected by problems of geometric

stability (e.g. see [14] for ICP); 2) the

differentiation of part surfaces depending on

functional relevance, so that assessment of

coverage quality can be weighed; 3) the

implementation of feedback mechanisms based

on the results of pose and coverage estimation,

to automate planning for further measurement

actions.

ACKNOWLEDGEMENTS

The authors would like to acknowledge Patrick

Bointon of the Manufacturing Metrology Team,

Leonidas Gargalis and Joe White of the Centre

for Additive Manufacturing, University of

Nottingham, for their assistance in designing and

producing the test cases. We also acknowledge

funding from EPRSC project EP/M008983/1.

REFERENCES

[1] Stavroulakis P, Chen S, Derlome C, Bointon

P, Tzimiropoulos G, Leach R K. Rapid

calibration tracking of extrinsic projector

parameters in fringe projection using

machine learning. Opt. Lasers Eng. 2019;

114: 7-14.

[2] Senin N, Leach R K. Information-rich

surface metrology. Proc. CIRP. 2018; 75:

19-26.

[3] Hana X F, Jin J S, Xie J, Wang M J, Jiang

W. A comprehensive review of 3D point

cloud descriptors. 2018; arXiv:1802.02297.

[4] Tombari F, Salti S, Di Stefano L. Unique

signatures of histograms for local surface

description. European conference on

computer vision. Springer; 2010. 356-369.

[5] Bellekens B, Spruyt V, Berkvens R, Penne

R, Weyn M. A benchmark survey of rigid 3d

point cloud registration algorithms. Inter.

Journ. Adv. Intell. Sysm. 2015.

[6] Chung D, Lee Y DS. Registration of multiple-

range views using the reverse-calibration

technique. Pattern Recogn. 1998; 31: 457-

464.

[7] Friedman J H, Bentely J, Finkel R A. An

Algorithm for Finding Best Matches in

Logarithmic Expected Time. ACM

Transactions on Mathematical Software.

1977; 3: 209-226.

[8] Merigot Q, Ovsjanikov M, Guibas L

J. Voronoi-based curvature and feature

estimation from point clouds. Visualization

and Computer Graphics, IEEE Transactions

on. 2011; 17: 743-756.

[9] Ding C, He X. K-means clustering via

Principal Component Analysis. Proceedings

of International Conference on Machine

Learning. 2004. 225-232.

[10] Moretti M, Gambucci G, Leach R K, Senin

N. Assessment of surface topography

modifications through feature-based

registration of areal topography data. Surf.

Topogr. Metrol. Prop. 2019; 7.

[11] Fishler M A, Bolles R C. Random sample

consensus: A paradigm for model ﬁtting with

applications to image analysis and

automated cartography. Commun. ACM.

1981; 24: 381-95.

[12] Kendall D G. A survey of the statistical

theory of shape. Statistical Science. 1989; 4:

87-99.

[13] Besl P, McKay N. A method for registration

of 3-D shapes. IEEE TPAMI. 1992; 14: 239-

256.

[14] Gelfand N, Ikemoto L, Rusinkiewicz S,

Levoy M. Geometrically stable sampling for

the ICP algorithm. Proceedings of

International Conference on 3-D Digital

Imaging and Modeling, 3DIM. 2003.