Conference PaperPDF Available

Automated assessment of measurement quality in optical coordinate metrology of complex freeform parts

euspen’s 21st International Conference &
Exhibition, Copenhagen, DK, June 2021
Automated assessment of measurement quality in optical coordinate metrology of
complex freeform parts
Sofia Catalucci1, Nicola Senin1,2, Samanta Piano1, Richard Leach1
1Manufacturing Metrology Team, Faculty of Engineering, University of Nottingham, Nottingham, NG8 1BB
2Department of Engineering, University of Perugia, Perugia, 06125, Italy
The measurement of complex freeform geometries represents a fundamental challenge for quality assurance in the production of
high value-added parts, in particular when additive manufacturing technologies are involved. In addition, the increasing advances
towards automation and integration in industrial production hint at the possibility of developing intelligent coordinate measuring
systems, capable of autonomously planning a measurement process and assessing measurement performance while the inspection
task is in progress. In this context, optical measurement technologies appear as ideal candidates, featuring high sampling densities,
relatively short measurement times and capablity to access complex surfaces. In this work, the ongoing development of algorithmic
solutions dedicated to the automated assessment of measurement quality is discussed. The solutions are designed to be embedded
in smart and autonomous coordinate measuring systems, and need only the acquired point clouds and knowledge of the nominal
geometry (CAD model) to operate. At the core of the algorithmic solutions, point cloud analysis and spatial statistics are used to
assess measurement uncertainty and part coverage, the latter referring to the capability to sample hidden surfaces and hollow
features, as typically found in additively manufactured parts. The algorithmic solutions are illustrated and validated through
application to a test case of industrial relevance, generated via additive manufacturing.
Optical measurement technologies, point cloud analysis, quality indicators, part coverage, measurement uncertainty
1. Introduction
In conventional manufacturing the fabrication of highly
complex and freeform components is often difficult, and
geometric complexity is achieved through assembly [1]. Additive
manufacturing (AM) technologies, on the contrary, offer
opportunities for increased freedom in the design and
fabrication of more complex parts, reducing the need for
assembly, but increasing the challenge of metrological
inspection [2,3]. The process of identifying optimal
measurement set-ups must be repeated for the inspection of
each new, geometrically complex, AM part, which implies the
selection of the most suitable measurement technology and
related measurement process parameters, maximisation of part
coverage, optimisation of fixturing and optimal pose selection.
Given the pressing need in several industrial sectors (including
automotive, aerospace and biomedical [4]) to adopt optimised
solutions for checking product quality whereas reduce the
manufacturing times and costs, novel means for the
optimisation of the measuring process are needed.
In this context, measurement systems can be defined as
“smart” when capable of merging the advantages of the
underlying measurement technologies with increased capability
of self-adaptation and flexibility [5] to the inspection of
manufactured components. In particular, flexibility is defined in
the roadmap “Manufacturing Metrology 2020” (VDI/VDE-GMA
[6,7]) as the “adaptation to changes in measurement tasks”, i.e.
being able to respond flexibly to changes in measurement
requirements, and being able to inspect different features and
new components in a fully-automated way. At the same time,
the assessment of measurement performance accomplished in
real-time is in high demand. Therefore, based on the resultant
feedbacks and outputs from the measuring procedure, smart
systems will be able to assess quality in-process (i.e. while the
measurement is being performed), correcting themselves and
streamlining the additional measurements required in order to
increase the quality of the results.
The work presented in this paper focuses on the preliminary
development of a set of algorithmic solutions for the automated
computation of measurement quality indicators. These solutions
are designed to be integrated into optical instruments, guiding
them towards future full automation of part inspection and
intelligent measurement planning.
2. Methodology
In this work measurement quality indicators are defined,
aiming at assessing the quality of high-density point clouds
produced by optical technologies and registered to the
underlying CAD model available in form of triangle mesh. As a
central aspect in the computation of the quality indicators,
measured points are analysed in relation to where they fall
within the triangle mesh (i.e. within which individual triangle
facet). Therefore, a cloud-to-mesh association strategy is
implemented at the core of the algorithmic solutions. In
addition, the use of a triangle mesh allows to define spatial maps
of local measurement requirements (for example coverage
specifications, and precision of critical areas) through triangle
tagging (see schematic triangle tagging example in Figure 1). This
in turn allows the evaluation of local measurement performance
of the point cloud.
Figure 1. Schematic representation of triangle tagging: a triangle mesh
with overlaid colour maps. a) Point-to-triangle associations (points
associated to each specific triangle are illustrated with different colours),
and b) coverage specifications (covered triangles are indicated in green,
while uncovered ones are indicated in red)
2.1. Selected test case and experimental set-up
In this work, a metal bracket featuring a complex hollow
geometry is selected as test case for computation of the
measurement quality indicators. The test case features
approximate dimensions of (125 × 45 × 8) mm (size of the
enclosing envelope). It was fabricated at the University of
Nottingham by laser powder bed fusion (LPBF) using stainless
steel 316L. The test case was selected as suitable to assess the
performance of the measurement quality indicators as it
presents regions of surface with different functional
importance; furthermore, the part contains some of the typical,
complex features found in AM components (e.g. high slope
angles, hollow features).
The experimental set-up for the measurement of the selected
part was based on the combination of a commercial
measurement fringe projection system (blue-light technology
GOM ATOS Core 300) and algorithms developed in MATLAB
(measurement set-up and test sample shown in Figure 2).
In this work, measurements were made in the Manufacturing
Metrology Team (MMT) laboratory at the University of
Nottingham, where temperature was set at (20 ± 0.5) °C. The
part was placed on a rotary table and scanned at 360° for a total
of five repeats (performed in repeatability and reproducibility
conditions). The acquired data were stitched in real-time into
five complete 3D point clouds using the GOM Scan software,
with the support of reference point markers of 0.4 mm sticked
onto the surfaces of the part.
Figure 2. The optical fringe projection measurement system GOM Atos
Core 300 while measuring the selected test part
Data processing for the measured datasets involved manual
removal of points belonging to the background surfaces
surrounding the part, application of a noise filter based on
outlier detection, and deletion of isolated points [8,9].
2.2. Registration and cloud-to-mesh association
The registration of the measured point clouds to the
underlying reference geometry representing the part (available
in form of STL triangle mesh) is a fundamental step of the
measurement pipeline. In this work, registration was performed
based on landmark matching (i.e. local curvature). A detailed
discussion of the registration approach adopted in this work is
illustrated in [10].
Once the registration of the datasets was completed, a cloud-
to-mesh association pipeline was applied to identify the triangle
facets associated to each point within each measurement repeat
(point-to-triangle association). The procedure is outlined as
computation of the normal vector for each point in the cloud,
via principal component analysis (PCA) [11];
ray casting of the normal onto the triangle mesh to identify the
intersection point, and thus the triangle facet associated to the
point [12];
assessment of the validity of the association based on point-
to-triangle distance [8], using either:
a) a hardcoded threshold on maximum permissible distance;
b) outlier rejection (
𝜇 ± 3𝜎
) with respect to the population
of distances.
A colour map representation of the associations can be used
to visually assess the point-to-triangle association (Figure 3).
Figure 3. Example rendering of point-to-triangle association. Points
mapped to each specific triangle are shown in different colour
3. Measurement quality indicators: definitions and results
Once all the points have been processed (i.e. associated to a
triangle), the number of points paired to each triangle can be
computed, which in turn leads to a first assessment of coverage.
In addition, the point-to-triangle distance related to each
individual association (signed, considering the orientation of the
triangle facet normal, assumed as pointing outwards) can be
recorded. From the stored information, measurement quality
indicators are derived addressing for example: sampling density
(e.g. number of points associated to each triangle), accessibility
of hidden regions (triangles with none or too few points), local
point scatter and bias with respect to the surface (signed
distances of the points associated to each triangle). Detailed
definitions of the indicators and computation methods are
presented in [13].
Example results of the computation of measurement quality
indicators on the test part are shown in Figure 4 (colour maps of
coverage ratio, sampling density, and point dispersion in panels
a), b) and c) respectively) as locally mapped to the underlying,
registered geometry. Mesh triangles were colour-coded
according to the indicator results. The threshold value for the
classification of covered and uncovered facets for each triangle
was set at 75% of the maximum computed sampling density. In
order to obtain a visually clearer distribution of the results
shown via colour map, the values recorded for the sampling
densities and point dispersion were normalised by division with
their respective maximum value recorded across the repeats.
Boxplots for the coverage ratio and the coverage area ratio
indicators are shown in Figure 5. From the five repeated
measurements, approximately 92% of the triangles resulted as
sufficiently covered (coverage ratio) and approximately 86.5% of
the part area resulted as sufficiently covered (coverage area
Figure 4. Example of indicators results in form of customised colour
maps: a) coverage ratio reporting covered and uncovered triangles
rendered using binary colouring, b) sampling density overlaid to triangle
mesh (shown in normalised form), c) mesh triangles coloured using the
dispersion of signed point-to-surface distances (shown in normalised
Figure 5. Indicators of part coverage (boxplots from five measurement
repeats): coverage ratio, and coverage area ratio. The circles are the
individual results for the indicator computed on each one of the five
point cloud repeats, the black circle is the median
An example of the probability distribution of dispersion of
signed distances is shown in Figure 6 (i.e. first measurement
repeat indicated as
). The statistics computed from the
probability distributions for the measurement repeats are
reported in Table 1 (probability distributions indicated as
𝑠", 𝑠#, 𝑠$
Figure 6. Example binned histogram of dispersion of signed point-to-
surface distance values for the first measurement repeat (see the
statistics reported in Table 1 indicated in column
). Dispersion is
expressed in millimetres; normalised frequency (vertical axes) is the
number of occurrences of the values in a bin, divided by the total
number of occurrences
Table 1 Statistics of the distribution of dispersion of signed distances
0.00 -
0.00 -
0.00 -
0.00 -
0.00 -
4. Conclusions and future work
The ability to capture hidden surfaces of the part geometry,
maximise measurement coverage, and produce high-density
point clouds represent increasingly challenging requirements
that must be fulfilled by measuring instruments, in particular
when called to inspect complex geometries such as those
produced by AM. The proposed work focuses on the preliminary
development of sets of algorithmic solutions to automatically
compute measurement quality indicators, designed to be
embedded in future, smart optical measurement systems for the
full automation of part inspection and intelligent measurement
The developed measurement quality indicators can also
support investigation on the performance and behaviour of
optical measurement technologies, providing the foundation to
further research towards the development of more advanced
optical measuring instruments.
The authors would like to acknowledge Leonidas Gargalis of
the Centre for Additive Manufacturing (CfAM) at the University
of Nottingham for his assistance in providing the test case. We
also acknowledge funding from EPRSC project EP/M008983/1.
[1] Gibson I, Rosen D W and Stucker B 2010 Additive Manufacturing
Technologies: 3D Printing, Rapid Prototyping, and Direct Digital
Manufacturing (Springer: New York, USA)
[2] Varadan V K 2005 Handbook of Smart Systems and Materials (IOP
Publishing: London)
[3] Pascual D G, Daponte P and Kumar U 2019 Handbook of Industry
4.0 and SMART Systems (CRC Press)
[4] Leach R K, Bourell D, Carmignato S, Donmez A, Senin N and Dewulf
W 2019 Geometrical metrology for metal additive manufacturing
CIRP Ann. 68 677-700
[5] Imkamp D, Berthold J, Heizmann M, Kniel K, Peterek M, Schmitt R,
Seidler J and Sommer K-D 2016 Challenges and trends in
manufacturing measurement technology the “Industrie 4.0”
concept J. Sens. Sens. Syst. 5 325335
[6] VDI/VDE-GMA 2011 Fertigungsmesstechnik 2020: Technologie-
Roadmap für die Messtechnik in der industriellen Produktion (VDI-
Verlag: Düsseldorf)
[7] Berthold J and Imkamp D 2013 Looking at the future of
manufacturing metrology: roadmap document of the German
VDI/VDE Society for Measurement and Automatic Control J. Sens.
Sens. Syst. 2 1–7
[8] Han X F, Jin J S, Wang M J, Jiang W, Gao L and Xiao L 2017 A review
of algorithms for filtering the 3D point cloud Signal Process. Image
Commun. 57 103-112
[9] Catalucci S and Senin N 2020 State of the art in point cloud
analysis. In: Leach R K Advances in Optical Form and Coordinate
Metrology (IOP Publishing), Chap. 2
[10] Catalucci S, Senin N, Piano S and Leach R 2019 Intelligent systems
for optical form measurement: Automated assessment of pose and
coverage Proc. 34th ASPE, Pittsburgh, USA 145-150
[11] Hoppe H, DeRose T, Duchamp T, McDonald J and Stuetzle W 1992
Surface reconstruction from unorganized points Comput. Graph.
(ACM) 26 71-78
[12] Woop S, Schmittler J and Slusallek P 2005 RPU: A programmable
ray processing unit for realtime ray tracing SIGGRAPH ‘05 24
[13] Catalucci S, Senin N, Sims-Waterhouse D, Ziegelmeier S, Piano S
and Leach R 2020 Measurement of complex freeform additively
manufactured parts by structured light and photogrammetry
Measurement 164 108081
... In their classification, they introduce numerous quality criteria, including coverage and part accessibility (see section 3.5). Prior to this work, Catalucci et al [85,86] developed a set of algorithmic solutions for the assessment of the quality of complex measured shapes, based on knowledge of a reference model in the form of a triangle mesh and the intrinsic properties of measured point clouds. Zhang et al [87] proposed a technique to optimise the number of cameras and their respective positions for the measurement of the geometry of a given object. ...
... Hoppe et al [193] suggested in their early work related to the reconstruction of surfaces from unorganised point clouds to qualify data according to indicators of noise and density. Both indicators appeared to be representative of the quality of each point, highlighting the performance of the point-to-surface [86]: (a) boxplots from five measurement repeats indicating coverage ratio, and coverage area ratio respectively (individual results for each of the five repeats, and median); and (b) covered and uncovered triangles rendered using binary colouring projected onto the triangle mesh surface. Reproduced with permission from [86]. ...
... Both indicators appeared to be representative of the quality of each point, highlighting the performance of the point-to-surface [86]: (a) boxplots from five measurement repeats indicating coverage ratio, and coverage area ratio respectively (individual results for each of the five repeats, and median); and (b) covered and uncovered triangles rendered using binary colouring projected onto the triangle mesh surface. Reproduced with permission from [86]. © EUSPEN. ...
Full-text available
Manufacturing has recently experienced increased adoption of optimised and fast solutions for checking product quality during fabrication, allowing for manufacturing times and costs to be significantly reduced. Due to the integration of machine learning algorithms, advanced sensors and faster processing systems, smart instruments can autonomously plan measurement pipelines, perform decisional tasks and trigger correctional actions as required. In this paper, we summarise the state of the art in smart optical metrology, covering the latest advances in integrated intelligent solutions in optical coordinate and surface metrology, respectively for the measurement of part geometry and surface texture. Within this field, we include the use of a priori knowledge and implementation of machine learning algorithms for measurement planning optimisation. We also cover the development of multi-sensor and multi-view instrument configurations to speed up the measurement process, as well as the design of novel feedback tools for measurement quality evaluation.
Full-text available
Strategic considerations and publications dealing with the future of industrial production are significantly influenced these days by the concept of “Industrie 4.0”. For this reason the field of measurement technology for industrial production must also tackle this concept when thinking about future trends and challenges in metrology. To this end, the Manufacturing Metrology Roadmap 2020 of the VDI/VDE Society for Measurement and Automatic Control (GMA) was published in 2011 (VDI/VDE-GMA, 2011; Imkamp et al., 2012). The content of this roadmap is reviewed and extended here, covering new developments in the field of the Industrie 4.0 concept and presented with expanded and updated content.
Full-text available
"Faster, safer, more accurately and more flexibly'' is the title of the "manufacturing metrology roadmap'' issued by the VDI/VDE Society for Measurement and Automatic Control ( The document presents a view of the development of metrology for industrial production over the next ten years and was drawn up by a German group of experts from research and industry. The following paper summarizes the content of the roadmap and explains the individual concepts of "Faster, safer, more accurately and more flexibly'' with the aid of examples.
The needs, requirements, and ongoing and future research issues in geometrical metrology for metal additive manufacturing are addressed. The infrastructure under development for specification standards in AM is presented, and the research on geometrical dimensioning and tolerancing for AM is reviewed. Post-process metrology is covered, including the measurement of surface form, texture and internal features. In-process requirements and developments in AM are discussed along with the materials metrology that is pertinent to geometrical measurement. Issues of traceability, including benchmarking artefacts, are presented. The information in the review sections is summarized in a synthesis of current requirements and future research topics.
In recent years, 3D point cloud has gained increasing attention as a new representation for objects. However, the raw point cloud is often noisy and contains outliers. Therefore, it is crucial to remove the noise and outliers from the point cloud while preserving the features, in particular, its fine details. This paper makes an attempt to present a comprehensive analysis of the state-of-the-art methods for filtering point cloud. The existing methods are categorized into seven classes, which concentrate on their common and obvious traits. An experimental evaluation is also performed to demonstrate robustness, effectiveness and computational efficiency of several methods used widely in practice.
This book covers in detail the various aspects of joining materials to form parts. A conceptual overview of rapid prototyping and layered manufacturing is given, beginning with the fundamentals so that readers can get up to speed quickly. Unusual and emerging applications such as micro-scale manufacturing, medical applications, aerospace, and rapid manufacturing are also discussed. This book provides a comprehensive overview of rapid prototyping technologies as well as support technologies such as software systems, vacuum casting, investment casting, plating, infiltration and other systems. This book also: Reflects recent developments and trends and adheres to the ASTM, SI, and other standards Includes chapters on automotive technology, aerospace technology and low-cost AM technologies Provides a broad range of technical questions to ensure comprehensive understanding of the concepts covered.
We describe and demonstrate an algorithm that takes as input an unorganized set of points fx 1 ;:::;x n g # IR on or near an unknown manifold M, and produces as output a simplicial surface that approximates M. Neither the topology, the presence of boundaries, nor the geometry of M are assumed to be known in advance --- all are inferred automatically from the data. This problem naturally arises in a variety of practical situations such as range scanning an object from multiple view points, recovery of biological shapes from two-dimensional slices, and interactive surface sketching.
  • V Varadan
Varadan V K 2005 Handbook of Smart Systems and Materials (IOP Publishing: London)
  • S Catalucci
  • N Senin
  • D Sims-Waterhouse
  • S Ziegelmeier
  • S Piano
  • R Leach
Catalucci S, Senin N, Sims-Waterhouse D, Ziegelmeier S, Piano S and Leach R 2020 Measurement of complex freeform additively manufactured parts by structured light and photogrammetry Measurement 164 108081