Reliability and validity of handheld structured light scanners and a static stereophotogrammetry system in facial three-dimensional surface imaging

Abstract

Several new systems for three-dimensional (3D) surface imaging of the face have become available to assess changes following orthognathic or facial surgery. Before they can be implemented in practice, their reliability and validity must be established. Our aim, therefore, was to study the intra- and inter-system reliability and validity of 3dMD (stereophotogrammetry), Artec Eva and Artec Space Spider (both structured light scanners). Intra- and inter-system reliability, expressed in root mean square distance, was determined by scanning a mannequin’s head and the faces of healthy volunteers multiple times. Validity was determined by comparing the linear measurements of the scans with the known distances of a 3D printed model. Post-processing errors were also calculated. Intra-system reliability after scanning the mannequin’s head was best with the Artec Space Spider (0.04 mm Spider; 0.07 mm 3dMD; 0.08 mm Eva). The least difference in inter-system reliability after scanning the mannequin’s head was between the Artec Space Spider and Artec Eva. The best intra-system reliability after scanning human subjects was with the Artec Space Spider (0.15 mm Spider; 0.20 mm Eva; 0.23 mm 3dMD). The least difference in inter-system reliability after scanning human subjects was between the Artec Eva and Artec Space Spider. The most accurate linear measurement validity occurred with the Artec Space Spider. The post-processing error was 0.01 mm for all the systems. The Artec Space Spider is the most reliable and valid scanning system.

Full text

1
Vol.:(0123456789)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports
Reliability and validity of handheld
structured light scanners
and a static stereophotogrammetry
system in facial three‑dimensional
surface imaging
J. A. M. Schipper
1*, B. J. Merema
1, M. H. J. Hollander
1, F. K. L. Spijkervet
1, P. U. Dijkstra
2,3,
J. Jansma
1, R. H. Schepers
1 & J. Kraeima
1
Several new systems for three‑dimensional (3D) surface imaging of the face have become available to
assess changes following orthognathic or facial surgery. Before they can be implemented in practice,
their reliability and validity must be established. Our aim, therefore, was to study the intra‑ and
inter‑system reliability and validity of 3dMD (stereophotogrammetry), Artec Eva and Artec Space
Spider (both structured light scanners). Intra‑ and inter‑system reliability, expressed in root mean
square distance, was determined by scanning a mannequin’s head and the faces of healthy volunteers
multiple times. Validity was determined by comparing the linear measurements of the scans with the
known distances of a 3D printed model. Post‑processing errors were also calculated. Intra‑system
reliability after scanning the mannequin’s head was best with the Artec Space Spider (0.04 mm
Spider; 0.07 mm 3dMD; 0.08 mm Eva). The least dierence in inter‑system reliability after scanning
the mannequin’s head was between the Artec Space Spider and Artec Eva. The best intra‑system
reliability after scanning human subjects was with the Artec Space Spider (0.15 mm Spider; 0.20 mm
Eva; 0.23 mm 3dMD). The least dierence in inter‑system reliability after scanning human subjects
was between the Artec Eva and Artec Space Spider. The most accurate linear measurement validity
occurred with the Artec Space Spider. The post‑processing error was 0.01 mm for all the systems. The
Artec Space Spider is the most reliable and valid scanning system.
ree-dimensional (3D) surface imaging of the face is used for planning and evaluating orthognathic and facial
surgery, for orthodontic diagnostics, and for research purposes1. e 3dMD (3dMD Inc., Atlanta, GA, USA) is
the most widely used scanning system, using stereophotogrammetry, and its reliability and validity is reported in
the literature2,3. However, the disadvantages are the system’s immobility and that certain areas of the face cannot
be imaged completely due to the camera’s xed orientation and focus point. An advantage is that the 3dMD’s
modular system oers several set-up possibilities, enabling it to scan the whole body or parts of it. However, the
modular system may lead to dierent measurement results due to having to apply a dierent number of pods
and dierent lenses.
Several handheld scanners using structured light are gaining popularity and the results have been compared
to those of the 3dMD system4–6. However, these structured light scanners use a sequence of pictures that form the
3D model instead of an individual 3D picture, whereby accuracy may be lost during post-processing. Structured
light scanners acquire the surface of the face through continuous light emission, which undergoes distortions
and deformations due to the irregularity of the surface7. Stereophotogrammetry, on the other hand, captures
two or more images simultaneously from dierent angles. Before these new hand-held structured light scanners
can be applied clinically, their reliability and validity need to be determined.
OPEN
1Department of Oral and Maxillofacial Surgery, University Medical Center Groningen, University of Groningen,
Groningen, The Netherlands. 2Department of Rehabilitation Medicine, University Medical Center Groningen,
University of Groningen, Groningen, The Netherlands. 3Sirindhorn School of Prosthetics and Orthotics,
Faculty of Medicine Siriraj Hospital, Mahidol University, 14 Arun Amarin Rd, Bangkok 10700, Thailand. *email:
j.a.m.schipper@umcg.nl
Content courtesy of Springer Nature, terms of use apply. Rights reserved

2
Vol:.(1234567890)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports/
e Artec Eva (Artec Group Inc., Luxembourg, Luxembourg) has been used in multiple studies, showing
poorer reliability then the 3dMD system4. According to the manufacturer’s specications, the validity of the Artec
Space Spider (Artec Group Inc., Luxembourg, Luxembourg) is better than the Eva system (www. artec 3d. com).
is manufacturer determined the systems’ validity by scanning “scale ball bars” with known distances between
the balls. However, no study has compared the reliability and validity of the Artec Space Spider with the 3dMD
system regarding scanning the facial region. e Artec Eva has a larger working distance than the Artec Space
Spider, making the Eva system more useful for scanning larger areas such as the whole body. e manufacturer
recommends the Artec Eva for scanning the face. e Artec Space Spider has a narrow eld of view which leads to
many small images being stitched together to create a 3D facial model. is procedure may introduce processing
errors when scanning larger areas. Since more literature is available about the 3dMD, we selected two handheld
structured light scanners with a dierent working distance and dierent reported validity to compare with the
3dMD. e aim of this study is therefore to determine the reliability and validity of 3D surface imaging systems,
namely Artec Eva, Artec Space Spider and 3dMD, for the facial region by calculating both their intra-system and
inter-system reliability, and the validity of the linear measurements.
Material and methods
e study was approved by the medical ethics committee of the University of Groningen and University Medical
Centre Groningen, the Netherlands (study number METC2021/476). e study protocol was in accordance with
institutional guidelines and the Declaration of Helsinki. Written informed consent was obtained from all the
participants prior to the study. e COSMIN taxonomy of measurement properties was used for the reliability
and validity terminology and denitions (www. cosmin. nl). e subjects were healthy employees and students
recruited from the department of oral and maxillofacial surgery at the University Medical Center Groningen
(UMCG), hence we performed convenience sampling. An email was sent to the employees and students to ask if
they would participate in this study. Volunteers were excluded from the study if there was severe facial deformity,
excessive facial hair or if they had ever experienced epilepsy (because of the bright ashing light of the hand-held
scanners). e volunteers were scanned with the static 3dMD, the hand-held Artec Eva and the Artec Space
Spider systems. To ensure a natural head position, the volunteers were asked to look into the mirror, gently bite
in maximum intercuspation, swallow before the scanning, relax their lips and to keep their eyes open8.
Technical comparison of the scanning/scanner systems
e working distance of the Artec Eva is the smallest (0.2–0.3m). e acquisition time and the processing time
of the 3dMD are the smallest, 1.5ms and immediate, respectively. e manufacturer reported that the Artec
Space Spider (0.1mm) has the highest accuracy, i.e. validity. e models produced by the Artec Space Spider
capture the most details (Table1).
Data processing and analysis
e data obtained aer using the Artec Eva and Artec Space Spider were processed using the Artec Studio 14
Professional soware (Artec Group, Luxembourg, Luxembourg) to acquire 3D models (Fig.1). e 3dMD images
were processed with the Vultus soware (3dMD, Inc, Atlanta, GA, USA). As both Artec Eva and Artec Space
Table 1. Overview of scanner systems.
3dMDface system Artec Eva Artec Space Spider
3D model example
Modality Stereophotogrammetry Structured light Structured light
Handheld working distance Not applicable Yes (0.2–0.3m) Yes (0.4–1.0m)
Acquisition time 1.5ms ~ 20s ~ 60s
Processing time Immediate reconstruction Up to 10min Up to 30min
Accuracy (reported by manufacturer) 0.2mm 0.1mm 0.05mm
Resolution 0.5mm 0.5mm 0.1mm
Cost > 70,000 USD > 15,000 USD > 25,000 USD
Content courtesy of Springer Nature, terms of use apply. Rights reserved

3
Vol.:(0123456789)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports/
Spider need additional steps to acquire a 3D model from the several frames, a standardized protocol from the
manufacturer’s website9 was used for the processing in Artec Studio. e 3dMD Vultus soware was utilized
to match all the scan models by performing surface-based registration according to the iterative closest point
method (this was performed for all scanning systems). Aer the images were manually aligned in the 3dMD
Vultus soware, a specic T-shaped area of the forehead (Fig.1) and dorsum of the nose was marked on each scan
as the registration surface. e quality of the registration was determined in the Vultus soware and expressed as
registration root mean square (RMS) error. e quality of the matching process was also determined based on a
visual inspection. e scans were then cut with the 3-matic soware (Materialise, Leuven, Belgium) by selecting
planes formed by the landmarks tragus-forehead and tragus-menthon (Fig.1). en the scans were imported
into CloudCompare (CloudCompare, version 2.11 alpha) and the cloud-to-mesh distance was computed with
automatic octree level to produce signed distances (Fig.1). In this mode, CloudCompare will simply search the
nearest triangle in the reference mesh of each point of the compared cloud, i.e. the Euclidean distance. e RMS
was calculated from these signed distances by RStudio (version 2021.09.2+382).
Mannequin head intra‑system reliability
To check for the intra-system reliability of the scanners without interference of facial variability, a mannequin’s
head was scanned three times with each system (Fig.2). e RMS distances between the 3 scan pairs (1st scan
vs. 2nd, 1st vs. 3rd, 2nd vs. 3rd) of each system were calculated.
Mannequin head inter‑system reliability
Distance maps were calculated for the mannequin head scans aer being registered in the 3dMD Vultus soware
using histograms with identical colour-corresponding dierences. e data acquired for the intra-system reli-
ability were also used to analyse inter-system reliability. e RMS distances between the 1st scan of each system,
between the 2nd scan of each system and the 3rd scan of each system, were calculated (1st scan 3dMD vs. 1st
scan Artec Eva, 1st scan 3dMD vs. 1st scan Artec Spider, and 1st scan Artec Spider vs. 1st scan Artec Eva etc.).
Intra‑system reliability in 3 volunteers
To assess intra-system reliability on humans, 3 volunteers were scanned 6 times with each system (Fig.2). e
RMS distance was calculated for all 15 scan pairs of each system used on each volunteer (1st scan vs. 2nd, 1st vs.
3rd, 1st vs. 4th, 1st vs. 5th, 1st vs. 6th, 2nd vs. 3rd, 2nd vs. 4th, 2nd vs. 5th, 2nd vs. 6th, 3rd vs. 4th, 3rd vs. 5th,
3rd vs. 6th, 4th vs. 5th, 4th vs. 6th, 5th vs. 6th).
Inter‑system reliability in 16 volunteers
To assess inter-system reliability on humans, 16 volunteers were scanned with each imaging system once (Fig.2).
Each participant’s RMS distances were calculated and compared, Artec Eva versus 3dMD, Artec Space Spider
versus 3dMD and Artec Eva versus Artec Space Spider. e combined participant RMS distance means were
calculated for each of these pairs.
Validity using a reverse engineered mannequin’s head as the gold standard
Since the precise surface dimensions of the mannequin’s face were unknown, we reverse engineered the man-
nequin’s face. e face was scanned using the Artec Space Spider and cylinders with known linear dimensions
in multiple directions were modelled from the scan using the 3-matic soware (Materialise, Leuven, Belgium).
A 3D printer (PA12 material, EOS SLS printer, Oceanz, Ede, the Netherlands) was then used to print this refer-
ence model with known dimensions (Fig.3). To compensate for potential errors in the 3D printing process, 2
Acquision and
processing
(Artec/3dMD)
Surface-based
registraon using
the ICP method
(Vultus)
Cut (3-mac)
Differences are
calculated
(CloudCompare)
.cvs files analysed
to produce the
RMS value
(Rstudio)
RMS =
0.37 ±0.55
Figure1. Processing steps (IC p = iterative closest point method; RMS = root mean square).
Content courtesy of Springer Nature, terms of use apply. Rights reserved

4
Vol:.(1234567890)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports/
observers took 3 repeated linear measurements with precise calipers (micrometres; with an accuracy of 0.01mm)
of the length of the cylinders of the 3D printed mannequin model to act as a gold standard. e 3D printed face
was then scanned three times consecutively with the 3dMD, Artec Eva and Artec Space Spider systems. Linear
measurements of these 3D scans were made from these cylinders with the 3-matic soware (Materialise, Leuven,
Belgium). e dierence between the gold standard linear measurements and the 3D scanned measurements
were used as validity measures. RMS distances were calculated for the 3dMD, Artec Eva and Artec Space Spider
systems in relation to the reference model (the 3D printed mesh le).
Post‑processing error
Post-processing error was recorded by repeating the analysis process 5 times for one scan pair.
Statistical analyses
All statistical analyses were performed using RStudio (version 2021.09.2+382). For the mannequin intra- and
inter-system reliability descriptive statistics were provided, however no statistical testing was performed due to
the limited sample size of 1 mannequin. Continuous variables were denoted as mean and standard deviation
(SD). Normal distribution or skewness was assessed by visual inspection of the histograms and statistically tested
with the Shapiro–Wilk test. Equality of variances was tested with the F-test. One-way repeated measures analysis
of variance (ANOVA) was performed with a post hoc Tukey’s honestly signicant dierence test to check for the
Figure2. Study diagram (RMS = root mean square).
Content courtesy of Springer Nature, terms of use apply. Rights reserved

5
Vol.:(0123456789)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports/
inter-system reliability aer scanning human subjects. e Friedman test was performed as a nonparametric test
when the conditions for parametric testing were not met.
Informed consent
e human subject in the gures is the rst author of this manuscript. e rst author consents to publishing
the gures.
Results
Mannequin head intra‑system reliability
e Artec Space Spider had the best intra-system reliability (RMS 0.04mm) whereas the reliability of scan-
ning with the 3dMD and Artec Eva was similar (Table2). e registration RMS error ranged between 0.02 and
0.04mm.
Mannequin head inter‑system reliability
Visual inspection of the distance maps between the systems illustrated that 3dMD was less reliable than Artec
Eva and Artec Space Spider with the main dierences being in the following regions: under the nose, under
the lip, and at the side of the face (Table3). e distance maps showed more positive dierences between the
Artec systems and the 3dMD, especially at the sides of the face. Clinically this means that the Artec systems
over-estimate or the 3dMD under-represents the true shape of the side of the face. In the Artec Eva vs. Artec
Space Spider distance map, ne lines could be seen that were cuts made into the mannequin for a previous study.
ese cuts were captured better by the Artec Space Spider than the Artec Eva. e registration RMS error was
0.06mm for Artec Eva versus Artec Space Spider, 0.08mm for Artec Eva versus 3dMD, and 0.06mm for Artec
Space Spider versus 3dMD.
e best inter-system scanning reliability was between the Artec Space Spider and Artec Eva.
Figure3. Reverse engineered 3D printed mannequin with known geometry and cylinders.
Table 2. Intra-system reliability results aer scanning a mannequin’s head 3 times with each system. e RMS
represents the mean dierence between the 1st scan versus 2nd, 1st versus 3rd, 2nd versus 3rd. RMS root mean
square
3dMD (RMS in mm ± SD) Artec Eva (RMS in mm ± SD) Artec Space Spider (RMS in mm ± SD)
Mean dierence ± SD 0.07 ± 0.01 0.08 ± 0.02 0.04 ± 0.01
Registration error 0.02 0.04 0.02
Content courtesy of Springer Nature, terms of use apply. Rights reserved

6
Vol:.(1234567890)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports/
Human subjects
Sixteen volunteers were scanned, but 14 scans were included in the statistical analyses since 2 volunteers had facial
hair that produced errors on the surface models. Regarding the intra-system reliability measurements, 3 volun-
teers were randomly selected for extra scans. All 14 scans were used for the inter-system reliability assessment.
Intra‑system reliability in 3 volunteers
e Shapiro–Wilk test showed no signicant deviation from the normal distribution. e F-test showed equal
variances between the systems. Intra-system reproducibility did not dier signicantly between the systems
(repeated measures one-way ANOVA, p = 0.498). e mean RMS dierences were smallest for Artec Space
Spider (0.15 ± 0.02mm) (Table4). e registration RMS error was 0.07mm for 3dMD, 0.09mm for Artec Eva
and 0.07mm for Artec Space Spider.
Inter‑system reliability in 14 volunteers
e Shapiro–Wilk test showed a deviation from the normal distribution for the 3dMD versus the Eva (p = 0.01),
the 3dMD versus the Spider (p = 0.02) but not for the Spider versus the Eva system (p = 0.20), and so the Fried-
man test was used. e comparative inter-system scanning reliability diered signicantly between the three
systems, namely between the Artec Eva-3dMD, the Artec Space Spider-3dMD and the Artec Eva-Artec Space
Spider (Friedman test; χ2 = 22.29; p < 0.01). Post-hoc Conover testing showed that the dierence between the
Spider vs the Eva system was less compared to the 3dMD versus the Eva system (p < 0.01; 0.26 ± 0.06 mm vs.
0.39 ± 0.11mm, respectively), and the dierence between the Spider versus the Eva system was less compared
to the 3dMD versus the Spider system (p < 0.01; 0.26 ± 0.06mm vs. 0.45 ± 0.11mm, respectively) but there
Table 3. Inter-system reliability results aer scanning a mannequin’s head 3 times with each system. e RMS
represents the dierence between the 3dMD versus Artec Eva, 3dMD versus Artec Space Spider, and Artec
Space Spider versu Artec Eva system. Fine lines are visible on the head (arrow). ese lines were cut into the
mannequin for a previous study. RMS root mean square
3dMD versus Artec Eva (RMS in mm ± SD) 3dMD versus Artec Space Spider (RMS in
mm ± SD) Artec Space Spider versus Artec Eva (RMS in
mm ± SD)
Distance map
1st scan 0.22 ± 0.20 0.17 ± 0.15 0.08 ± 0.15
2nd scan 0.19 ± 0.18 0.19 ± 0.19 0.15 ± 0.22
3rd scan 0.21 ± 0.17 0.16 ± 0.14 0.12 ± 0.16
Mean dierence ± SD 0.21 ± 0.01 0.17 ± 0.03 0.12 ± 0.01
Registration error 0.07 0.06 0.06
Table 4. Intra-system reliability results aer scanning 3 human subjects 6 times with each system. RMS
represents the mean dierence between the scans for every person. e six scans of every person were
compared in every possible combination which resulted in 15 scan pairs (1st scan vs. 2nd, 1st vs. 3rd, 1st vs.
4th, 1st vs. 5th, 1st vs. 6th, 2nd vs. 3rd, 2nd vs. 4th, 2nd vs. 5th, 2nd vs. 6th, 3rd vs. 4th, 3rd vs. 5th, 3rd vs. 6th,
4th vs. 5th, 4th vs. 6th, 5th vs. 6th). e intra-system reproducibility did not dier signicantly between the
systems (repeated measures one-way ANOVA, p = 0.498). RMS root mean square
3dMD (RMS in mm) Artec Eva (RMS in mm) Artec Space Spider (RMS in mm)
Person 1 (15 scan pairs) 0.17 ± 0.03 0.26 ± 0.09 0.13 ± 0.05
Person 2 (15 scan pairs) 0.33 ± 0.12 0.16 ± 0.08 0.14 ± 0.03
Person 3 (15 scan pairs) 0.18 ± 0.03 0.18 ± 0.08 0.18 ± 0.04
Mean dierence ± SD 0.23 ± 0.08 0.20 ± 0.05 0.15 ± 0.02
Registration error 0.10 0.06 0.06
Content courtesy of Springer Nature, terms of use apply. Rights reserved

7
Vol.:(0123456789)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports/
was no dierence between the 3dMD versus the Eva system and the 3dMD versus the Spider system (p = 0.80;
0.39 ± 0.11mm vs. 0.45 ± 0.11mm, respectively). On comparing the 14 volunteers’ scanning results, the best
inter-system reliability was between the Artec Eva and the Artec Space Spider systems (Table5). e registra-
tion RMS error was 0.12mm for the Artec Eva versus the Artec Space Spider system, 0.14mm for the Artec Eva
versus the 3dMD system, and 0.13mm for the Artec Space Spider versus the 3dMD system.
Validity using reverse engineered mannequin head as gold standard
Digitally performed linear measurements on the 3D scans compared to the gold standard showed that the Artec
Space Spider had the best validity (Table6). e Artec Eva had consistently higher measurements, while the
3dMD demonstrated consistently lower measurements. When comparing the 3D printed model to the three
systems’ scans, the Artec Space Spider was also the most accurate in scanning the geometry, showing the lowest
RMS dierence between these meshes. e distance maps showed that the three systems’ scans mostly diered
in the lower part of the face.
Post‑processing error
Aer repeating all the described processing steps ve times for one scan pair, a standard deviation of 0.01mm
was found for all 3 scanning systems.
Table 5. Inter-system reliability results aer scanning 14 human subjects once with each system. RMS: Root
mean square. RMS represents the mean dierence between the scans from the dierent systems. e Friedman
test and post-hoc Conover testing showed signicant dierences between Spider versus Eva and 3dMD versus
Eva (p < 0.01), Spider versus Eva and 3dMD versus Spider (p < 0.01), but no dierence between 3dMD versus
Eva and 3dMD versus Spider (p = 0.80).
3dMD versus Artec Eva (RMS in mm ± SD) 3dMD versus Artec Space Spider (RMS in
mm ± SD) Artec Space Spider versus Artec Eva (RMS in
mm ± SD)
10.31 ± 0.31 0.33 ± 0.52 0.26 ± 0.24
20.69 ± 0.57 0.47 ± 0.53 0.21 ± 0.19
30.53 ± 0.64 0.41 ± 0.75 0.31 ± 0.36
40.27 ± 0.35 0.48 ± 0.95 0.24 ± 0.23
50.34 ± 0.25 0.38 ± 0.26 0.10 ± 0.14
60.35 ± 0.39 0.37 ± 0.55 0.21 ± 0.27
70.29 ± 0.75 0.34 ± 0.83 0.29 ± 0.25
80.32 ± 0.34 0.50 ± 0.88 0.24 ± 0.22
90.45 ± 0.54 0.75 ± 0.17 0.31 ± 0.27
10 0.42 ± 0.39 0.44 ± 0.54 0.26 ± 0.23
11 0.32 ± 0.33 0.40 ± 0.74 0.26 ± 0.24
12 0.45 ± 0.41 0.44 ± 0.74 0.30 ± 0.31
13 0.35 ± 0.35 0.34 ± 0.62 0.34 ± 0.32
14 0.35 ± 0.35 0.59 ± 1.00 0.34 ± 0.28
Mean dierence ± SD 0.39 ± 0.11 0.45 ± 0.11 0.26 ± 0.06
Registration error 0.14 0.13 0.12
Table 6. Reverse engineered mannequin head with cylinders for gold standard use (RMS = root mean square).
3dMD versus mannequin model Artec Eva versus mannequin model Artec Space Spider versus mannequin model
Distance maps
Linear measurements error (deviation in mm ± SD) 0.30 ± 0.07 0.30 ± 0.10 0.03 ± 0.02
Mean dierence (RMS in mm ± SD) 0.38 ± 0.06 0.36 ± 0.07 0.31 ± 0.03
Registration error 0.21 0.39 0.10
Content courtesy of Springer Nature, terms of use apply. Rights reserved

8
Vol:.(1234567890)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports/
Discussion
ree-dimensional surface imaging has proven to be a valid and reliable imaging modality for evaluating several
surgical treatment results of the face. New handheld scanning systems have been released which advertise even
higher accuracy than the previously widely used static systems. Reliability studies of these new scanners are
needed before they can be applied safely in clinical practice and research.
Intra-system and inter-system reliability was established in this study by scanning both healthy volunteers
and a mannequin’s head. We found that the Artec Space Spider gave better intra-system reliability results for
both the volunteers and the phantom head. e Artec Space Spider captured more details, since small wrinkles
and small protuberances of the face were visualized very well. e dierences in reliability between the Artec
Space Spider and Artec Eva were small whereas, compared to these handheld scanners, the 3dMD system intro-
duced more inaccuracy as illustrated by the distance maps, especially under the nose, under the chin and at the
side of the face. ese dierences could possibly be explained by the xed orientation of the 3dMD system. e
Artec handheld systems can scan in several angles around the face to capture iterative meshes that are stitched
together to produce a complete 3D model. erefore, the geometry underneath the nose or chin can be imaged
more completely using handheld systems. Of note, though, is that due to the narrow eld of view of the Artec
Space Spider, many iterative meshes must be captured and stitched together to create a 3D model. is process
can hypothetically introduce a processing error when scanning large areas. However, since the overall reliability
and validity of the Artec Space Spider was high in our study aer scanning the face, we consider the relevance
of this error to be low.
Moreover, since the errors were all below 0.5mm, the clinical relevance of these dierences are very limited.
Other properties such as ease-of-use or the time needed to generate a 3D model could dictate the choice for a
specic system. However, when measuring volume by comparing larger surface scans of the face, these small
dierences add up to more inaccurate volume measurements. erefore, we recommend using a system with the
smallest errors, such as the Artec Space Spider, for pre- and post-surgery volume measurements.
In previous studies, 3dMD was used as a reference when comparing between scanners6. Yet, since the reli-
ability of the other scanners tested in this study was better than 3dMD, we believe that 3dMD should not act as a
gold standard anymore. We 3D printed a mannequin phantom head with cylinders. Since 3D printing introduces
errors, we took linear measurements of the cylinders, in multiple directions on the mannequin’s head, to opera-
tionalize this as a gold standard. ese measurements show that the Artec Space Spider is more valid in capturing
the geometry of the face. Based on the linear measurements error, it seems that the Artec Eva overestimates the
volume of the face and the 3dMD underestimates the volume of the face.
Most studies have analysed the reliability and validity of the 3dMD stereophotogrammetry system2,3,10. A
recent study which scanned dental casts showed that the Artec Space Spider had high reliability, but low validity
(0.4mm)11. However, the authors attributed the error to the automatic reconstruction function of the soware,
a function we did not use. e Artec Space Spider gave one of the best validity measures compared to the Pri-
mescan, Trios, Pritiface and iPhone systems when used to scan nasal, orbital and auricular models that were
manufactured using stereolithography12. e Artec Space Spider’s results were found to be most valid compared
to Artec Eva, Vectra H1, Bellus and SNAP aer scanning plaster statues with balls attached; the linear measure-
ments were used for comparison purposes13. e reliability of scanning the peri-orbital region with the Artec
Space Spider was excellent, at 0.1–0.2 cmm14.
In this study, we showed that all 3 scanners produce clinically acceptable results, with errors of less than
0.3mm for intra-system reliability. Variation in facial expression is known to introduce an error when com-
paring multiple facial 3D surface scans, which explains the dierences between the human volunteers and the
phantom head15. e substantial dierence between the phantom head and the faces’ RMS distances shows that
the variability in facial expression introduces more error than the scanning systems themselves. is nding
highlights that standardization of facial expression during scanning could be even more important than scanning
system accuracy. Regarding clinical use, certain properties can dierentiate their usefulness for specic applica-
tions. e Artec Space Spider is especially useful for acquiring scans of objects with complex geometry, since it
showed the best validity and captured the most detail. Artec Eva and Artec Space Spider are mobile scanners,
a property which makes them especially useful when the scanner needs to be used at multiple sites, such as the
operating room or multiple clinical centres. In addition to this, the storage space needed for these mobile scan-
ners is substantially less than the static 3dMD scanning system, which needs a designated room for its setup.
However, training and experience is required for the mobile scanners to produce acceptable scan quality since
slow and homogenous movements are required during the recording. e 3dMD obtains surface models with a
single ash which reduces the risk of movement artefacts during scanning, and provides fast acquisition times.
One of the strengths of our study is that we included both intra- and inter-system reliability of scanning a
mannequin’s head and human subjects. is facilitates proper assessment of the reliability of both a rigid object
with the geometry of a face and the face of a human subject. We also included validity analysis of a 3D printed
face and linear measurements of cylinders on this 3D print. However, the main limitation of our study is that
the true geometry of the human face is not known which inuences validity measures. It is therefore impos-
sible to dene a gold standard which can facilitate the assessment of the validity of capturing the true complex
geometry of the face. We therefore also assessed a 3D printed face and compared this with the original printed
mesh. However, as 3D printing also introduces a printing error, we also made linear measurements with calipers
and compared these with the linear measurements of the 3D scans, consequently showing again that the Artec
Space Spider’s results are the most valid. Other limitations are the study’s small sample size and that we did not
perform a formal sample size calculation, but used convenience sampling based on sample sizes from the lit-
erature. Despite these limitations, the multiple analyses give consistent results: the Artec Space Spider scanning
Content courtesy of Springer Nature, terms of use apply. Rights reserved

9
Vol.:(0123456789)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports/
system is the most reliable and valid. Future research on the validity of new 3D surface imaging systems should
therefore focus on establishing a proper gold standard for facial scanning.
We have proven that the three compared scanning systems are reliable and valid but conclude that the Artec
Space Spider is the most reliable and valid. Other properties such as ease-of-use or mobility of the 3D surface
imaging systems can dictate which system to choose in practice. Nonetheless, 3D surface imaging is a promising
and radiation-free imaging modality that can be safely implemented to assess facial surgical outcomes during
the follow-up.
Data availability
Data is available at the research databank of the department of Oral and Maxillofacial Surgery in the University
Medical Centre of Groningen, the Netherlands. Data can be requested through the rst author, J.A.M. Schipper
(j.a.m.schipper@umcg.nl). It can also be requested through our department administration (+31 503613840;
l.kempers@umcg.nl).
Received: 28 July 2023; Accepted: 18 March 2024
References
1. Kau, C. H., Richmond, S., Incrapera, A., English, J. & Xia, J. J. ree-dimensional surface acquisition systems for the study of facial
morphology and their application to maxillofacial surgery. Int. J. Med. Robot. 3, 97–110 (2007).
2. Lübbers, H. T., Medinger, L., Kruse, A., Grätz, K. W. & Matthews, F. Precision and accuracy of the 3dmd photogrammetric system
in craniomaxillofacial application. J. Craniofac. Surg. 21, 763–767 (2010).
3. Aldridge, K., Boyadjiev, S. A., Capone, G. T., DeLeon, V. B. & Richtsmeier, J. T. Precision and error of three-dimensional phenotypic
measures acquired from 3dMD photogrammetric images. Am. J. Med. Genet. A 138A, 247–253 (2005).
4. Verhulst, A. et al. ree-dimensional imaging of the face: A comparison between three dierent imaging modalities. Aesthet. Surg.
J. 38, 579–585 (2018).
5. Modabber, A. et al. Evaluation of the accuracy of a mobile and a stationary system for three-dimensional facial scanning. J. Crani o
Maxillofac. Surg. 44, 1719–1724 (2016).
6. Knoops, P. G. M. et al. Comparison of three-dimensional scanner systems for craniomaxillofacial imaging. J. Plast. Reconstr.
Aesthetic Surg. 70, 441–449 (2017).
7. Pellitteri, F., Scisciola, F., Cremonini, F., Baciliero, M. & Lombardo, L. Accuracy of 3D facial scans: a comparison of three dierent
scanning system in an invivo study. Prog. Orthod. 24, 44 (2023).
8. Jakobsone, G., Vuollo, V. & Pirttiniemi, P. Reproducibility of Natural head position assessed with stereophotogrammetry. Orthod.
Craniofacial Res. 23, 66–71 (2020).
9. Artec3D Website. https:// www. artec 3d. com/ porta ble- 3d- scann ers.
10. Metzger, T. E., Kula, K. S., Eckert, G. J. & Ghoneima, A. A. Orthodontic so-tissue parameters: A comparison of cone-beam
computed tomography and the 3dMD imaging system. Am. J. Orthod. Dentofac. Orthop. 144, 672–681 (2013).
11. Piedra-Cascón, W., Methani, M. M., Quesada-Olmo, N., Jiménez-Martínez, M. J. & Revilla-León, M. Scanning accuracy of non-
dental structured light extraoral scanners compared with that of a dental-specic scanner. J. Prosthet. Dent. 126, 110–114 (2021).
12. Unkovskiy, A. et al. Accuracy of capturing nasal, orbital, and auricular defects with extra- and intraoral optical scanners and
smartphone: An invitro study. J. Dent. 117, 103916 (2022).
13. Tsuchida, Y., Shiozawa, M., Handa, K., Takahashi, H. & Nikawa, H. Comparison of the accuracy of dierent handheld-type scan-
ners in three-dimensional facial image recognition. J. Prosthodont. Res. https:// doi. org/ 10. 2186/ jpr. JPR_D_ 22_ 00001 (2022).
14. Hollander, M. H. J. J. et al. Reproducibility of 3D scanning in the periorbital region. Sci. Rep. 11, 3671 (2021).
15. Maal, T. J. J. et al. Variation of the face in rest using 3D stereophotogrammetry. Int. J. Oral Maxillofac. Surg. 40, 1252–1257 (2011).
Author contributions
J.A.M.S.: Writing—Original dra, Conceptualization, Investigation, Formal analysis, Data curationB.J.M.:
Writing—Review and Editing, Software, Data curationM.H.J.H.: Writing—Review and Editing,
ConceptualizationF.K.L.S.: Writing—Review and Editing, SupervisionP.U.D.: Writing—Review and Editing,
Methodology, SupervisionJ.J.: Writing—Review and Editing, Conceptualization, SupervisionR.H.S.: Writing—
Review and Editing, Conceptualization, SupervisionJ.K.: Writing—Review and Editing, Conceptualization,
Methodology, Soware, Supervision
Competing interests
e authors declare no competing interests.
Additional information
Correspondence and requests for materials should be addressed to J.A.M.S.
Reprints and permissions information is available at www.nature.com/reprints.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional aliations.
Content courtesy of Springer Nature, terms of use apply. Rights reserved

10
Vol:.(1234567890)
Scientic Reports | (2024) 14:8172 | https://doi.org/10.1038/s41598-024-57370-x
www.nature.com/scientificreports/
Open Access is article is licensed under a Creative Commons Attribution 4.0 International
License, which permits use, sharing, adaptation, distribution and reproduction in any medium or
format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the
Creative Commons licence, and indicate if changes were made. e images or other third party material in this
article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the
material. If material is not included in the article’s Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from
the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.
© e Author(s) 2024
Content courtesy of Springer Nature, terms of use apply. Rights reserved

1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com