Content uploaded by Radomír Mendřický
Author content
All content in this area was uploaded by Radomír Mendřický on May 09, 2019
Content may be subject to copyright.
MM SCIENCE JOURNAL I 2016 I DECEMBER
1565
DETERMINATION OF
MEASUREMENT ACCURACY
OF OPTICAL 3D SCANNERS
RADOMIR MENDRICKY
Technical University of Liberec
Department of Manufacturing Systems and Automation,
Liberec, Czech Republic
DOI: 10.17973/MMSJ.2016_12_2016183
e-mail: radomir.mendricky@tul.cz
Nowadays, the use of optical 3D digitisation in metrology
becomes more frequent and desired. Unfortunately, there are
still no binding standards for determining measurement
uncertainty of these systems and manufacturers of 3D scanners
often use their own standards to define accuracy of their
device. This paper introduces a methodology to assess the
accuracy of digitisation using 3D optical scanners. The paper
deals with practical implementation of an acceptance test for
ATOS contact-less 3D scanners (from design and manufacturing
of own test etalon, through the determination of its nominal
dimensions, up to digitisation and evaluation) and publishes the
results of several experiments demonstrating the impact of
various factors on measurement accuracy.
KEYWORDS
Optical measurement, optical 3D scanner, 3D digitization,
accuracy, calibration, Acceptance test, Atos
1 INTRODUCTION
Currently, the measurement of dimensional and shape
precision in industrial practice is performed by conventional
methods such as a contact method using coordinate measuring
machines (hereinafter only CMM). Even though these machines
provide one of the most accurate results [Flack 2011], they
cannot be used in some cases. An example may be
measurement of surfaces with very complex shapes.
That is the reason why laser and optical measurement systems,
so called 3D scanners, are used more and more often. These
scanners digitise the part, and the inspection itself is performed
on a virtual model obtained by means of the digitisation
process (for example [Harding 2013, Zhang 2013]). Inspection
using these systems offer several crucial advantages such as
fast measurement of parts, even with complex shapes, high
data density and, above all, independence of results on part’s
rigidity. Due to the overall description of a measured part, it
also allows to perform complex and objective analysis.
However, the accuracy of these measurement methods is not
so apparent. Generally, there are no strictly determined
specifications for measuring uncertainty of optical 3D scanners.
Their accuracy is usually not clearly quantifiable and we have to
carry out various comparison tests.
Figure 1. GOM calibration etalon for so called Acceptance Test
Manufacturers of 3D scanners create their own standards and
verify the precision of their devices in special metrological
laboratories using etalons of ideal shapes such as spherical
systems – see calibration etalon for performance of so called
Acceptance Test (Fig. 1) for optical scanners manufactured by
GOM [GOM mbH 2014].
Currently, optical 3D scanners are often used as a universal
measurement and inspection device. Therefore, the user must
be sure that he uses an optical scanner working in a defined
range of accuracy. In long-term perspective the only way to
meet this requirement is to use comparable criteria and regular
inspection of the device (scanner). The manufacturer
recommends to perform the acceptance test approximately
once a year or more often for specific industries. Naturally, it is
necessary to keep these intervals in manufacturing companies
regarding quality and certification standards. However, these
inspections are often underestimated in research laboratories
and are not performed within the recommended intervals, or at
all. Performance of the test in an approved or manufacturer’s
laboratory is very costly, while the whole measurement system
is not available for several days or week after it is sent to the
manufacturer for inspection. However, the accuracy and
reliability of the system in development laboratories is crucial
as well. That is why we focused on research with the aim to
determine accuracy of optical 3D scanner measurement in
laboratory conditions. We were interested in the existence of
possibility to design and manufacture a calibration etalon that
would enable to reliably perform a test of measurement
accuracy what would be comparable with the acceptance test.
This would facilitate implementation of own inspection of
system’s reliability in shorter intervals and with significantly
reduced costs. Additionally, the approved etalon would be used
for further testing and experiments.
During an analysis of research papers dealing with a similar
issue, it was found out that these researches are often
addressing only partial analyses or unilaterally focused
experiments. One of the first tests of this type has already been
made in 2003 by [Keller 2003], who tried using contact-less
measurements to determine planar dimensions of machine
parts. The aim was to analyse the origin of each error of this
measurement method and to find a possibility to reduce this
error to minimum. Recently, [Dokoupil 2013] performed an
experimental identification of ATOS Triple Scan optical system
deviations related to application of matting chalk and titanium
coating. The aim of the research described in this literature was
merely to assess measurement uncertainty when applying
chalk and titanium powder, as well as to determine layer
thickness of matting powders. Another significant research
discussing the influence of matting coatings on the precision of
3D optical measurement was published by [Palousek 2015]. His
team performing the research found out that while the chalk
coating may reach the average thickness up to 44 µm, using
titanium-white-based anti-reflection coating reduces the
thickness approximately tenfold – down to 5 µm, offering a
highly positive effect on the accuracy of digitisation process. A
more detailed comparison of several scanning systems and
assessment of 3D scanner precision was published by [Barbero
2011]. In order to determine the measurement uncertainty, the
team performed measurement of several calibration elements
such as sphere, cylinder and gage block. An uncertainty of
25 μm was determined during the Atos system measurement
process.
MM SCIENCE JOURNAL I 2016 I DECEMBER
1566
Another interesting analysis concerning the evaluation of
accuracy 3D scanners and researches regarding the various
types of calibration artefacts or directly of calibration of optical
systems, can be also found for example, in publications [Acko
2012, Burchardt 2015, Campanelli 2016, Dury 2015, McCarthy
2011]. The first of these papers [Acko 2012] describes three
different types of artefacts for calibration, namely tetrahedron
artefacts for testing the basic measurement capability of optical
3D devices, freeform verification artefacts for testing the
capability of measuring complex geometry, and a large gear
artefact for task related calibration of different types of CMMs.
Other paper [Dury 2015] describes a verification facility that
has been developed with the aim of providing a 3D optical
scanner verification service to global industry ensuring greater
confidence in their measurement capability. The device allows
to simulate typical usage conditions where temperature and
lighting may vary. In addition, a range of test artefacts have
been specifically developed to identify scanners’ sensitivity to
colour, resolution, roughness, and laser scanning articulating
arm scan velocity. The paper [McCarthy] states that
documented standards for the verification of fixed CMMs fitted
with tactile probes are now widely available, whereas
verification procedures and more specifically verification
artefacts for optical-based systems are still in their infancy. The
paper further describes a freeform verification artefact that has
been developed, calibrated and used to support a
measurement comparison between a fixed CMM and a number
of optical systems (laser triangulation scanning,
photogrammetry and fringe projection). The research
concludes that the accuracy of the optical-based systems tested
is not as good as tactile probing systems.
Fairly extensive own analysis of measurement accuracy of
contact-less optical 3D scanners was performed in 2015 by
[Mendricky 2015]. This analysis focused primarily on inspecting
digitisation of objects with various shapes. Also, the capabilities
of 3D scanners to capture detailed elements on the measured
parts were examined. However, even in this study or any other
research available, the research was not in compliance with the
standards. There have not been used procedures defined in the
so-called acceptance test, which is decisive for checking the
accuracy of 3D optical measurement systems.
2 OPTICAL MEASUREMENT SYSTEM
In case of our research, the objective was to evaluate the
measurement accuracy and therefore design a calibration
etalon for ATOS 3D optical scanner (see Fig. 2). Atos is an
optical measurement system, whose measurement process is
based on principles of optical triangulation, photometry and
Fringe Projection method [Gorthi 2010]. This system is used in
various industrial branches such as construction, production,
quality control, design, etc.
Figure 2. ATOS optical 3D scanner with MV250 measuring volume
The most important part of the system is the optical 3D scanner
itself, consisting of a projector, two cameras and a control unit.
By choosing appropriate lens, we define the size of a 3D area in
which the measured object will be scanned – so called
measuring volume. Setting the volume is not only affecting the
size of the measured part, but also significantly influences the
density of measured points and the actual scanning accuracy.
When designing the etalon, we focused on three available
measuring volumes listed in Tab. 1.
Measuring volume
Resolution
Measurement
distance
55×44×30 (hereinafter 55
,
SO)
0.04 [mm]
300 [mm]
250×200×200 (hereinafter 250)
0.18 [mm]
730 [mm]
700×560×560 (hereinafter
700)
0.50 [mm]
1030 [mm]
Table 1. Overview of the ATOS system measuring volumes
3 ACCEPTANCE TEST
As indicated above, the acceptance test is performed to verify
measurement accuracy of optical systems. Based on
characteristic parameters, it verifies whether the measurement
system meets the quality limit parameters or not. The
measured deviations must not exceed the limit values given by
the manufacturer, which are specific for various scanner types,
measuring volumes and parameters. The ATOS device test is
governed by manufacturer’s (GOM) specifications and is in
accordance with the VDI/VDE 2634 – part 3 [VDI/VDE 2634
2008] standard, related to optical 3D systems. The standard
describes the practical part of the test, defines the calibration
etalon, characteristic values, measurement conditions and the
evaluation method. The manufacturer determines further
specifics that must be maintained during the test [GOM mbH
2012, 2014]:
The sensor and its parts are factory-adjusted. Check
whether the settings comply with the specification
before performing reverification measurement. In
case the settings do not comply with the
specifications, set up the sensor according to the
respective User Manual Hardware.
Calibrate the sensor. Maintain the warm-up time and
the calibration limit values.
Carry out the measurements with the quality setting
set to High and the resolution set to full.
Select the exposure time so that the measuring
images are well exposed. Avoid overexposures.
Polygonise single scans to a mesh using the Standard
setting.
For calculating the spheres, the software uses only
measurement data above a defined plane. This plane
is aligned parallel to the artefact base plate. Also, its
plane intersects the sphere at 10° south latitude. The
software determines the spheres using the least
squares method. During the process, the software
rejects 0.3% of the measured values as outliers. This
value corresponds to a 3 sigma setting.
The software determines the Length measurement
error parameter using Method C (see VDI/VDE 2634
Part 3 for more information).
The ambient temperature and the artefact
temperature have to be identical.
The measuring environment must be free of
mechanical vibrations.
The ambient light must not vary extensively during
the measurement. Avoid extremely bright external
light sources.
MM SCIENCE JOURNAL I 2016 I DECEMBER
1567
3.1 Acceptance Test parameters
The rules of evaluation are based directly on the mentioned
standard. However, when performing the Acceptance Test, the
conditions are not as strict and the manufacturers have the
right to choose their own methods. The parameters measured
during the Acceptance Test are:
Probing error form (PF)
Probing error size (PS)
Sphere spacing error (SD)
Length measurement error (E)
When evaluating the parameters, the measured errors are
compared to MPExx (maximum permissible error) parameter.
This limit is set exclusively by the manufacturer of the
measurement device.
Probing error form (PF) (Fig.3 - left) shows shape deviations
(sphericity). The highest and the lowest deviation from an ideal
sphere is being identified (from all scanned points).
PF (sigma) = (1)
PF (range) = |max – min| (2)
Figure 3. Schematic representation of the “Probing error” calculation
[GOM mbH 2014]
Probing error size (PS) (Fig.3 - right) shows deviation of fitted
sphere size. The sphere size is measured by means of Fitting
Sphere method. The diameter error is described as a difference
between Da measured diameter and Dn reference diameter
value.
PS (size) = Da - Dn (3)
Sphere spacing error (SD) (Fig. 4) shows spacing deviation of
the two sphere’s centres. It is used to determine whether the
scanner measures in the correct scale on a defined length.
Figure 4. Schematic representation of the ”Sphere spacing error“
calculation [GOM mbH 2014]
Length measurement error (E) (Fig. 5) shows deviation of
length measurement. It determines whether the scanner
measures in the right scan, including the effect of the scan
noise.
Figure 5. Schematic representation of the "Length measurement error"
calculation [GOM mbH 2014]
4 CALIBRATION ETALON
In order to perform the test and evaluate the required
parameters, it is necessary to use a proper calibration standard.
The etalon must be designed so that it offers evaluation of
more measuring volumes. Spherical objects are used in most
calibrations (various metrology branches). The same applies
when calibrating optical devices.
Due to usage of various lenses (measuring volumes) and
regarding to evaluation parameters, the etalon consists of pairs
of spheres with various diameters and spacings. In our case, 3
pairs of spheres were used. Their diameters and spacing were
in accordance with the VDI/VDE 2634 standard and are listed in
Tab. 2, and in Fig. 6.
Measuring volume
Diame
ter
of spheres
Sphere
spacing
55x44x30 (MV55)
8 [mm]
26 [mm]
250x200x200 (MV 250)
20 [mm]
115 [mm]
700x560x560 (MV 700)
40 [mm]
320 [mm]
Table 2. Selected dimensions of measured elements on the etalon
Figure 6. Etalon design for the Acceptance Test [Frkal 2015]
Stainless steel was selected as a base plate material, specifically
AISI 304 chrome-nickel austenitic steel. The spheres were
bought from Redhill Balls, a company with an office in Prague.
Due to the requirement of having objects with small roughness,
the balls are polished and made of AISI 304 material and offer
G100 accuracy degree. Tab. 3 shows accuracy of balls provided
by the manufacturer.
MM SCIENCE JOURNAL I 2016 I DECEMBER
1568
Grade Ball Diameter
Variation
Deviation
from
Spherical
Form
Surface
Roughness
G100
2.5 [µm]
2.5 [µm]
0.100 [µm]
Table 3. Degree of accuracy G100 according to ISO 3290 [Redhill 2016]
The balls were glued by means of two-component epoxy glue
(Bison Epoxy Metal) into seatings in the base plate machined
earlier. The created calibration etalon fitted with reference
points is shown in Fig. 7.
Figure 7. Physical model of the calibration etalon (with and without the
matte coating)
4.1 Determining nominal dimensions
The real, for our purpose referential (nominal) element
dimensions were repeatedly measured on DEA GLOBAL Status
7.10.5, a 3-axis coordinate measuring system, manufactured by
Hexagon Metrology. The CMM measurement accuracy was
higher by more than an order of magnitude in comparison to
the assumed measurement accuracy of 3D scanners. The
calibration sheet offers measurement accuracy of 2.5 µm. A
thermal correction of dimensions to 20 °C was performed,
while the measurement results were processed in a statistic
manner (average values from ten measurements were
calculated as well as type A, B and C standard measurement
uncertainty). The results of variables along with U expanded
uncertainty is shown in Tab. 4.
Measured variable Result of measurement
Sphere L Ø8 mm 8.000 ± 0.005 [mm]
Sphere R Ø8 mm 7.999 ± 0.005 [mm]
Sphere L Ø20 mm 20.000 ± 0.005 [mm]
Sphere R Ø20 mm 20.000 ± 0.005 [mm]
Sphere L Ø40 mm 40.002 ± 0.005 [mm]
Sphere R Ø40 mm 40.001 ± 0.007 [mm]
Sphere spacing 26 mm 26.017 ± 0.005 [mm]
Sphere spacing 115 mm 115.005 ± 0.006 [mm]
Sphere spacing 320 mm 319.933 ± 0.007 [mm]
Outer distance 34 mm 34.016 ± 0.005 [mm]
Outer distance 135 mm 135.005 ± 0.006 [mm]
Outer distance 360 mm 359.935 ± 0.007 [mm]
Table 4. Measurement results of etalon on the CMM
4.2 Scanning the etalon by means of ATOS II 400
When performing an acceptance test, the procedure is
precisely recommended by the aforementioned standard for
scanning a calibration etalon. Scanning is performed in three
series for each measuring volume. In each series, the position
of the scanner towards the etalon is different, while in each
position, a total of 10 images is created. During each
measurement, the etalon must be stable and, of course, fitted
with reference points. Since the spheres are very glossy, an
anti-reflection coating in a form of titanium dioxide is applied.
To obtain uniform thickness of anti-reflection coating is most
commonly used aerograph (Airbrush) for application. In order
to ensure high accuracy, all measurements were performed in
constant conditions, specifically in temperature of 20 ± 1°C and
relative humidity of 50 ± 10 %.
The measurement procedure is as follows [GOM mbH 2014]
(1st measurement series):
Place the etalon horizontally onto a rotary plate and
ensure its stability.
Tilt the scanner by 45° towards a vertical axis.
Set the scanner distance so that the centre of
measuring volume is located in the centre of axis
linking the measured spheres.
Create 8 images in each position. The etalon is
rotated by 45° around its vertical central axis for each
image.
Set the scanner so that the centre of the measuring
volume points to intersection of axes linking the
spheres and the outer surface of the left sphere. In
comparison to the initial position, rotate the etalon
by 90° and create a ninth image.
Rotate the etalon by 180°, and, similarly to the
previous case, set the measuring volume centre to
point at intersection of axes linking the spheres and
the outer surface of the right sphere. Create a tenth
image.
During the second, or third, measurement series, the
procedure is analogical, but the scanner is tilted in its horizontal
axis by 45° clockwise, or counter-clockwise. The imaging
positions for 3rd measurement series are shown in Fig. 8.
Figure 8. Position of individual images when scanning the etalon
(MV 250, 3rd measurement series)
5 DATA EVALUATION
The next step was to process the scanned date in GOM Inspect
Professional software v8, which provided information about the
required values. In order to determine sphere diameters, a
“Fitting Sphere” was used, allowing calculation of a geometrical
element using a large number of scanned points (so called
point cloud). In compliance with the standard, a “Gauss Best-
Fit” was used as a calculation method, while 3 (i. e. 99.73 %)
points from the selection were used to calculate the element
(see Fig. 9).
MM SCIENCE JOURNAL I 2016 I DECEMBER
1569
Figure 9. Construct Fitting Sphere feature
Calculation of sphere spacing was performed by creating a “2-
point distance” inspection element, measuring distance
between the centres of left and right spheres of the given pair.
5.1 Parameters affecting the measurement accuracy
There is a wide range of parameters that are affecting the
accuracy of scanned data. Measurement conditions (such as
ambient temperature, humidity, lighting, dust, etc.) can be
influenced up to a certain point and adapted to the
requirements. However, there are other variables that may
vary during individual measurements, which are significantly
affecting the measurement results, as confirmed by
experiments. Among these factors are for example fitting the
etalon with an appropriate anti-reflection coating (uniform
layer), time since last calibration of the device, or a data
evaluation method (software).
Effect of device calibration
Generally, it is recommended to perform the calibration
(meaning user calibration using the calibration board) regularly
in given time intervals, every time the device is transported,
after a significant change of ambient temperature or change of
scanner optics. Based on an internal inspection procedures, the
system is able to autonomously point out to the user that the
scanner is probably not calibrated, and that it is necessary to
perform the calibration. When the scanner is used in a
laboratory with stable conditions and the system does not warn
about the necessity to calibrate the device, the circumstances
tempts the user to not perform the calibration very often.
Figure 10. Effect of calibration on sphere diameters (MV 250,
measurement series No. 1)
During the first experiment, where the aforementioned
methods were used, we performed an etalon digitisation and
evaluation of parameters such as sphere diameters and
spacing. The measurement was performed using procedures for
first series (part 4.2) and MV 250 measuring volume. First, the
digitisation was performed on a device, whose calibration
period expired approximately a week before. However, the
device was in stable laboratory conditions for that time and
was occasionally used for measurement. There was a total of 5
measurements. Then, a new user calibration of the device was
performed, and consequently, the digitisation process was
repeated five times. Results for various sphere diameters are
shown in Fig. 10, spacing is shown in Fig. 11.
Figure 11. Effect of calibration on spacing of spheres (MV 250,
measurement series No. 1)
The measured values clearly show that the results of all
measurements did not fluctuate in any of the cases – the
values were only slightly deviating from the average value.
However, upon comparison of measurements before and after
the calibration, the value is clearly different. After the
calibration, the value was much closer to the nominal
dimension. The deviation of sphere diameter (difference
between sphere diameter obtained by digitisation and the
nominal dimension) was approximately twice as large in case of
“non-calibrated” system. In case of spacing, the error of “non-
calibrated” system was almost four times larger in comparison
to the newly calibrated device. Therefore, it is clear that in
order to perform as accurate measurement as possible, it is
necessary to perform calibration of the device often and
regardless of the seemingly non-problematic operation of the
device.
Effect of the anti-reflection coating
In the next part of the experiment, the impact of application
and thickness of the matte coating was tested. The anti-
reflection coating is applied manually. When regarding the
shape complexity of the scanned parts, it is clear that the
resulting layer will not be uniform on the whole surface. In case
of our testing, two coatings were applied. During the first
application, a smaller amount of powder was used, while the
second coating was applied to ensure more uniform
distribution and better covering.
The following graph (Fig. 12) compares diameters of left and
right sphere after the first and the second coating with the
nominal values. After the second coating, the average value
was higher by 0.003 mm and was therefore logically further
from the nominal one due to the layer of applied powder.
MM SCIENCE JOURNAL I 2016 I DECEMBER
1570
Figure 12. Effect of coating on sphere diameter (MV 250, measurement
series No. 1)
The figure below (Fig. 13) proves that the quality, performance
and size of coating affects the spacing only slightly (which is, in
calculation principle, logical). Values measured on the sample
were almost identical (coordinates of sphere centres remain
unchanged).
Figure 13. Effect of coating on spacing of spheres (MV 250,
measurement series No. 1)
However, the quality of coating application (especially its lack)
has a certain effect on the uniformity of scanned data. That is
confirmed by a colour representation of deviations of the
scanned sphere surface from the ideal spherical element – see
Fig. 14. The figure clearly shows that when having a sufficient
and well performed coating, the surface of the element is
scanned more uniformly (Fig. 14 on the right), while when the
matte coating is insufficient, the local reflections may cause
certain irregularities of the scanned surface and lead to local
errors (Fig. 14 on the left). This has a negative effect on the
calculation of individual points’ coordinates and decreases the
measurement objectivity. In contrary to reality, this effect
increases the magnitude of Probing error form parameter and
generally increases the error of shape of the scanned elements.
Therefore, the experiment shows that the anti-reflection
coating must be performed not only uniformly, but with
sufficient amount as well (neither too little nor too much). That
however requires a very experienced operator.
Figure 14. Deviation colour map with regard to quality of the
performed coating (left – first coating {insufficient in the equator line
area}, right – second coating)
5.2 Results of the acceptance test
The goal of our research was to try to perform a test of
measurement accuracy of optical 3D scanner in research
laboratory conditions using own etalon in a manner so that the
performance is based on corresponding recommendations and
standards and is comparable with an acceptance test
performed in metrology laboratories. For this purpose a
digitisation of the manufactured etalon was performed
pursuant to a procedure (see part 4.2) for all 3 measurement
series. Consequently, the characteristic values were evaluated.
The objective was to determine all 4 determining parameters
(see part 3.1) for all three measuring volumes. Using the optical
digitisation, we obtained a combined total of 54 values. These
values were then compared to the reference dimensions
obtained by means of CMM, and the deviation was determined.
Example of evaluation for “Probing error size” parameter (left
sphere) is listed in Table 5, the “Sphere spacing error” is listed
in Table 6. Both evaluated for MV 250.
Measurement
series
Number of
images
Selected
points
Actual
diameter (D
a
)
[mm]
Nominal
diameter (D
n
)
[mm]
Probing error
size (PS) [mm]
1
10
2774
19.992
20.001
-
0.009
2
10
2763
19.983
20.001
-
0.018
3
10
2615
19.991
20.001
-
0.010
Table 5. Evaluation of the “Probing error size” parameter (MV250, left
sphere)
Measurement
series
Number of
images
Actual
spacing [mm]
Nominal
spacing [mm]
Sphere
spacing error
(SD) [mm]
1
10
115.015
115.009
0.006
2
10
115.008
115.009
-
0.001
3
10
115.021
115.009
0.012
Table 6. Evaluation of the “Sphere spacing error” parameter (MV250)
In order to make final evaluation of the acceptance test, the
standard states that the maximum (absolute) value of each
parameter obtained during all three measurement series is the
decisive one. With regard to that, the table below (Tab. 7) lists
the resulting overview of all evaluated parameters of the
acceptance test for three measuring volumes that were used.
The table also lists magnitudes of errors identified during the
last test in an official metrology laboratory.
Paramete
r
Results of
our study
Metrology
laboratory
MV 55
Probing error form (sigma)[mm]
0.002
0.001
Probing error size [mm]
0.023
0.003
Sphere spacing error [mm]
-
0.005
-
0.002
Length measurement error [mm]
0.018
0.003
MV 250
Probing error form (sigma)[mm]
0.003
0.004
Probing error size [mm]
-
0.018
-
0.020
Sphere spacing error [mm]
0.012
-
0.017
Length measurement error [mm]
-
0.018
-
0.035
MV 700
Probing error form (sigma)[mm]
0.009
0.024
Probing error size [mm]
-
0.087
-
0.103
Sphere spacing error [m
m]
0.012
-
0.044
Length measurement error [mm]
-
0.080
-
0.187
Table 7. Comparison of obtained maximal values with the values
provided by the manufacturer
MM SCIENCE JOURNAL I 2016 I DECEMBER
1571
The comparison clearly shows that in case of MV250, the
obtained error values were comparable with the test
performed in GOM laboratories, in case of MV700, the results
were even better, but conversely, the results were slightly
worse with MV55. However, it shall be noted that in all cases,
the errors are in orders not higher than hundredths of mm. This
proves that the system is measuring properly and that all the
elements of the system (board for user calibration, cameras,
and projector), measurement conditions and evaluations
methods are fine. Also, it can be said that the methodology
and the procedures listed in the paper are valid and generally
applicable for similar systems.
In terms of official acceptance test, the values of determined
deviations are compared to the limit values provided by the
manufacturer. If all the parameters are below the determined
limit, an Acceptance Test certificate is issued stating that the
device has passed. In case of our test, we maintained to keep
below the known limits in all cases. An example of known limits
for MV 250 is listed in Tab. 8.
Parameter
Results of
our study
Limit
Probing error form (sigma)[mm]
0.003
0.007
Sphere spacing error [mm]
0.012
0.020
Table 8. Comparison of measured maximum values with the
manufacturer’s limits
All the measuring volumes we used met the expectations. The
measured data show capability of the scanner to scan smaller
objects (their volume is a fraction of the cameras’ measuring
volume) and larger distances through the measuring volume
area. Scanning in three measurement series proved that the
ATOS scanner is able to scan objects with identical accuracy in
various mutual positions.
6 CONCLUSIONS
This paper presented methodology for evaluating eligibility and
accuracy of measuring by means of optical contact-less
systems. It is a rather complicated process, during which many
procedures have to be followed. The progress and procedure of
the measurement used for verifications of the scanner accuracy
is defined in so called Acceptance Test. So far, this test is the
only possible way to numerically express accuracy of the ATOS
optical measurement system. The test is based on VDI/VDE
2634 - part 3 standard, which is currently the only general
recommendation on how to evaluate accuracy of optical
systems. By respecting the parameters defined by the standard,
it is possible to determine with what accuracy the scanner
operates.
In terms of our research and verification of this methodology,
an own calibration etalon was designed and manufactured,
providing with the possibility to determine accuracy of ATOS II
400 optical 3D scanner. Additionally, the etalon was used to
perform many experiments. For example, it was used to
examine the effect of calibration and matte coating on the
accuracy of digitisation. It turned out that even a seemingly
well calibrated system may not be measuring entirely
accurately, if the last calibration was performed a while ago. In
order to achieve the highest possible measurement accuracy,
the calibration should be performed as often as possible.
Additionally, an increased attention should be paid to the
uniformity and sufficiency of anti-reflection coating, since
insufficiently matt surface may result in increase of noise and
shape error of the inspected elements.
The priority of our research was to perform so called
acceptance test of ATOS contact-less 3D scanner. All the
parameters given by the aforementioned standard were
successfully evaluated in all measuring volumes, leading to
determination of measurement accuracy of the device in
laboratory conditions. All observed parameters were below the
limits given by the manufacturer. It can therefore be stated
that in terms of this test, the mentioned system passed and
measured within the declared accuracy. Additionally, the
measurement proved that even in local conditions, it is possible
to achieve results similar to those provided by an approved
laboratory.
An own measurement cannot of course substitute a test
performed in a certified metrology laboratory, it however
provides with a possibility to perform the scanner eligibility test
more often. That might save considerable amount of financial
resources, since the official price of an acceptance test is very
high. Nevertheless, the most important fact and output of this
research is the possibility to use an approved etalon in terms of
many other tests and experiments with the goal to verify the
measurement capabilities of a scanner in various conditions.
Therefore, we will be able to evaluate the effect of external
conditions and internal digitisation parameters on the accuracy
of measuring by means of contact-less 3D optical scanners. In a
further study is planned also to evaluate different optical
systems (scanners from other manufacturers) using the same
etalon and methods, in the same conditions and to compare
accuracy of measuring of different systems with each other as
well as to compare with the values declared by the
manufacturer of the systems.
ACKNOWLEDGEMENT
This publication was written at the Technical University of
Liberec as part of the „Project 21130 - Research and
development in the field of 3D technology, manufacturing
systems and automation" with the support of the Specific
University Research Grant, as provided by the Ministry of
Education, Youth and Sports of the Czech Republic in the year
2016.
REFERENCES
[Acko 2012] Acko, B., McCarthy, M., Haertig, F., Buchmeister, B.
Standards for testing freeform measurement capability of
optical and tactile coordinate measuring machines.
Measurement Science and Technology. Vol. 23, 2012. ISSN
09570233
[Barbero 2011] Barbero, B., R., Ureta E., S. Comparative study
of different digitization techniques and their accuracy.
Computer-Aided Design [online]. 2011, vol. 43, no. 2, pp. 188-
206 [cit. 2015-06-30]. ISSN 00104485. Available from:
http://linkinghub.elsevier.com/retrieve/pii/S0010448510002150
[Burchardt 2015] Bräuer-Burchardt, Ch., Kühmstedt, P., Notni,
G. Combination of Air- and Water-Calibration for a Fringe
Projection Based Underwater 3D-Scanner. In: G. Azzopardi, N.
Petkov, ed. Computer Analysis of Images and Patterns. 16th
International Conference, CAIP 2015, Valletta, Malta,
September 2015, Part II: Springer, pp 49-60. ISSN 0302-9743
[Campanelli 2016] Campanelli, V., Howell, S. M., Hull, M. L.
Accuracy evaluation of a lower-cost and four higher-cost laser
scanners. Journal of Biomechanics. Vol. 49, 2016. pp. 127-131.
ISSN 00219290
[Dokoupil 2013] Dokoupil, F. Determination of the
measurement error 3D optical scanner. Brno, 2013. Thesis.
Brno university of technology. Faculty of mechanical
engineering. (in Czech).
MM SCIENCE JOURNAL I 2016 I DECEMBER
1572
[Dury 2015] Dury, M. et al. 3D Optical Scanner Dimensional
Verification Facility at the NPL’s “National FreeForm Centre”. In.
L. Blunt, H. N. Hansen, ed. Laser Metrology and Machine
Performance XI - 11th International Conference and Exhibition
on Laser Metrology, Machine Tool, CMM and Robotic
Performance, LAMDAMAP 2015: Euspen, pp 187-197. ISBN:
978-095667905-5
[Flack 2011] Flack, D., CMM verification (National Physical
Laboratory). Measurement Good Practice Guide. 2011, No. 42,
ISSN 1368-6550.
[Frkal 2016] Frkal, M. Design of calibration standard for
determining measurement accuracy of optical 3D scanners.
Liberec, 2016. Thesis. Technical University of Liberec.
Department of Mechanical Engineering. (in Czech).
GOM mbH 2012] GOM mbH. GOM Acceptance Test
(Certificate): Acceptance/Reverification According to VDI/VDE
2634, Part 3. Braunschweig, Germany, 2012.
[GOM mbH 2014] GOM mbH. GOM Acceptance Test – Process
Description, Acceptance Test according to the Guideline
VDI/VDE 2634 Part 3. Braunschweig, Germany, 2014.
[Gorthi 2010] Gorthi, S. S., Rastogi, P. Fringe Projection
Techniques: Whither we are? Optics and Laser in Engiering
[online]. 2010, pp. 133-140 [cit. 2016-11-10]. Available from:
http://infoscience.epfl.ch/record/140745/files/OLEN.pdf
[Harding 2013] Harding, K. Handbook of Optical Dimensional
Metrology. Boca Raton: CRC Press, 2013. ISBN: 978-1-4398-
5481-5.
[Keller 2003] Keller, P. Contactless measurement of flat
dimensions using digital image processing methods. In. XIV.
gemeinsames Wissenschaftliches Kolloquium TU Dresden - TU
Liberec, TU Dresden, Germany, pp. 7- 12, ISBN 3-86005-371-X.
(in Czech).
[McCarthy 2011] McCarthy, M., et al. NPL freeform artefact for
verification of non-contact measuring systems. In: SPIE - the
International Society for Optical Engineering, San Francisco, CA;
United States, January 2011. ISSN 0277786X
[Mendricky 2015] Mendricky, R. Analysis of measurement
accuracy of contactless 3D optical scanners. MM Science
Journal, vol. 2015, no. OCTOBER, DOI:
10.17973/MMSJ.2015_10_201541, pp. 711-716, ISSN 1803-
1269.
[Palousek 2015] Palousek, D. et al. Effect of matte coating on
3D optical measurement accuracy. Optical Materials. Vol. 40,
2015. pp. 1-9. ISSN 0925-3467.
[Redhill 2016] Redhill Precision. Grade ISO 3290. [online]. 2016
[cit. 15/09/2016]. Available from:
http://www.redhillballs.com/north-america/en/company/iso-
3290
[VDI/VDE 2634 2008] Verein Deutscher Ingenieure - Verband
der Elektrotechnik Elektronik Informationstechnik. VDI/VDE
2634, Part 3. Optische 3-D-Messsysteme Bildgebende Systeme
mit flächenhafter Antastung in mehreren Einzelansichten.
Düsseldorf, Germany, 2008.
[Zhang 2013] Zhang, S. Handbook of 3D Machine Vision: Optical
Metrology and Imaging. Boca Raton: CRC Press, 2013. ISBN:
978-1-4398-7219-2.
CONTACTS:
Ing. Radomir Mendricky, Ph.D.
Technical University of Liberec
Faculty of Mechanical Engineering
Department of Manufacturing Systems and Automatization
Studentska 2, 461 17 Liberec 1, Czech Republic
e-mail: radomir.mendricky@tul.cz
Tel.: +420 485 353 356
www.ksa.tul.cz
