Content uploaded by Jan Oltmann
Author content
All content in this area was uploaded by Jan Oltmann on Feb 28, 2019
Content may be subject to copyright.
AST 2019, February 19–20, Hamburg, Germany
TECHNOLOGY INNOVATIONS FOR A FASTER
AIRCRAFT CABIN CONVERSION
Constantin Deneke1, Jan Oltmann2, Thorsten Schüppstuhl1, Dieter Krause2
1Insitute for Aircraft Production Technology, Hamburg University of Technology
Denickestraße 17, 21073 Hamburg, Germany
2Insitute for Product Development and Mechanical Engineering,
Hamburg University of Technology
Denickestraße 17, 21073 Hamburg, Germany
constantin.deneke@tuhh.de
Abstract
During the life cycle of commercial aircraft, passenger cabins are converted multiple times.
The planning database for these conversions often lack consistent, up-to-date geometry data
which eventually leads to assembly problems and an extension of the aircraft ground time. In
this paper, concepts for the use of innovative technologies like Augmented Reality (AR) and
mobile 3D scanning are presented in order to quickly recognize potential assembly problems
and to increase the completeness and accuracy of the planning database for cabin conversion.
1 INTRODUCTION
Commercial passenger air traffic has been increasing for decades and the biggest
aircraft manufacturers, Airbus and Boeing, forecast a further doubling in the next 15
and 20 years respectively ([1], [2]). The passenger cabin plays a major role for
passenger comfort and safety. Passenger cabins are converted multiple times during the
airplane’s life span which can exceed 25 years in order to maintain high levels of
comfort and safety. In [3] a demand of more than 33.000 cabin conversions for
commercial airplanes is forecasted between 2010 and 2029. To meet these high
demands of cabin conversions and to resist the growing competition in this field of
industry, an efficient cabin conversion process is required at the out-carrying
companies like OEMs or MROs.
The cabin conversion process can be separated into two main phases as shown in
Figure 1, the planning and design (PD) and the modification and installation (MI). In
the PD-phase prior to the actual conversion on site the process consists of different
steps. First, the preconditions for the design phase are set. This includes the inspection
of an aircraft and gathering of required geometrical and other information. This is
usually done by investigating all relevant documents which include documents from
the airplane manufacturer and previous maintenance and modification documents.
Constantin Deneke, Jan Oltmann, Thorsten Schüppstuhl, Dieter Krause
2
After that the designing of the cabin layout as well as the verification of the assembly
with respect to design clashes can be done. In case of design clashes feasible solutions
are proposed [3].
Figure 1 – current process of cabin conversion
Based on the analyses at a MRO company that performs the whole cabin conversion
process hindrances of an efficient conversion process are derived. There are 3 main
aspects that lead to an insufficient planning database.
The airplane has undergone numerous maintenance and modification programs,
the documentation for which is only paper-based.
The airplane might not be designed with modern 3D CAD systems, and in case
3D CAD data is existent, it is proprietary to the aircraft manufacturers and not
accessible to the MRO company.
The airplane might have had several owners, operators and MROs, with
inconsistent data management.
These circumstances lead to the information about the current cabin state, which
engineers gather from various documents for design planning, being partially
incomplete, not fully up-to-date, or even contradictory. To compensate for this lack of
consistent, up-to-date information, design engineers use the few opportunities they
have to gather more information about the current state of the airplane’s cabin during
ground times in so-called surveys or physical fit checks (see Figure 1). This might be
during maintenance programs or overnight stops of the airplane. However, these
opportunities are very time limited and it is rarely possible to get all required
information from the current cabin. As a consequence, the design planning of a cabin
conversion can only be based on the available information which is still not complete
and fully up-to-date. This leads to the occurrence of assembly problems during the
conversion time.
Figure 2: Example of occurring clashes during a cabin component installation
For example design clashes between cabin components and cabin structure elements,
like brackets, regularly obstruct the installation of cabin components. Assembly
AST 2019, February 19–20, Hamburg, Germany
3
problems like the one shown in Figure 2 lead to the need of an additional modification
process and thus an extension of the aircraft ground time and a significant increase of
costs. In the next section we describe two technologies which can assist with finding
planning errors at an earlier stage and which are capable of acquiring up-to-date cabin
geometry data to reduce and avoid the occurrence of assembly problems.
2 TECHNOLOGIES FOR DATA ACQUISITION AND AS-BUILD
TO AS-DESIGNED ANALYSES
One technology which is capable to recognize as-build and as-designed deviations -
and thus eligible for the detection of planning errors - is Augmented Reality.
Augmented Reality system supplements the real world with virtual objects that appear
to coexist in the same space as the real world [4]. Proposed AR applications in the
aircraft industry have been around for decades. [5] describes an AR demonstration
system by Boeing for assembly tasks such as wire installation. With the growing
available computational power, AR is being used more and more in industry on a daily
basis. A current example for the application in the aircraft industry is the Smart
Augmented Reality Tool developed by the Airbus company Testia [6]. AR has also
been proposed to support planning tasks for other industrial applications ([7], [8]). The
key technology of any mobile AR application is the pose tracking of the AR device, so
virtual and real components can be correctly registered with each other to give the user
a realistic and immersive visualization. Mobile Augmented Reality devices usually use
visual tracking methods which can be distinguished between marker-based and
markerless methods. Marker-based methods extract a predefined, distinguishable
marker in the surrounding to determine the device’s position relative to the marker.
Markerless methods, however, extract natural features in the camera image to track the
device’s position. This means this method has a much higher computational
complexity, with its accuracy and robustness being highly dependent on the
surroundings [9].
A common technology for the acquisition of comprehensive geometry data is 3D
scanning. Among the key technologies in 3D scanning devices is the method of how
3D depth data is acquired. Most laser scanners use geometric triangulation [10]. Other
methods include time-of-flight measurement, structured light projection and
photogrammetry. Mobile 3D scanning devices use pose tracking methods for the
registration of single scans to each other to be able to fully digitalize an object’s
geometry. Similar to AR pose tracking, the pose tracking of mobile 3D scanner can be
either marker-based or markerless. In an additional post-processing step, algorithms
like the ICP (Iterative Closest Point), whose purpose it is to register two coarsely
aligned point clouds [11], might be used for increasing the accuracy of the registration.
The accuracy and precision of the 3D data of an object depends on the used 3D
sensor technology as well as on the accuracy of the registration. Effects on the
accuracy, the precision, but also whether a depth measurement of a 3D scan can be
successfully taken in the first place, are also dependent on the object and its
surroundings. For example dispersion of the emitted light and the reflectivity or
absorption properties of the object are influencing variables for the quality of the 3D
scan [12]. To improve the aircraft conversion, concepts and processes for the
Constantin Deneke, Jan Oltmann, Thorsten Schüppstuhl, Dieter Krause
4
integration of AR and 3D scanning technologies are presented in the next section.
3 CONCEPT FOR A FASTER CABIN CONVERSION
3.1 AR digitalisation system
In order to integrate smart tools such as Augmented Reality and 3D scanning devices
into the cabin conversion process, there are specific requirements that the devices and
technologies have to meet. Especially the opportunities to gather additional information
of the actual cabin structure prior to ground time face strong limitations. These
opportunities are not bound to a certain airport or hangar location but can occur at
airports or maintenance sites worldwide. Furthermore, these opportunities face strong
time restrictions, because the aircraft might only be accessible for a few hours.
Therefore, the smart tool must have high mobility and – since there might not be
enough time to transport it from one global location to another – high availability. From
this requirement it can be further derived that the smart tool must be low cost. The tool
must have the capability to gather as much relevant information as possible within a
short period of time. This also means that an efficient process for the acquisition of the
information is required. Furthermore, the application of the tool cannot be limited to
specialists who are trained to use the device, as they might not be available when the
aircraft is accessible. Thus, the application of the tool must be user-friendly, intuitive
and needs efficient user guidance.
To meet these requirements, we propose the use of an Augmented Reality system.
Augmented Reality in general has the capability to fulfill the mentioned requirements,
e.g. regarding mobility, costs, and speed. There are two limitations though, which
restrict the use of Augmented Reality. The first one is that the acquisition of
geometrical information from the surroundings is very restricted with common AR
devices like tablets. These devices are not able to acquire complete 3D data with an
acceptable accuracy. To overcome this limitation, we propose to enhance Augmented
Reality technology with 3D scanning technology. With this, the geometry of the aircraft
cabin structure can be acquired. This geometry data can be enriched with information
by AR user interaction on-site and then used later for the planning of the cabin
conversion.
3.2 Local high resolution refinement
The second limitation of the use of Augmented Reality is its accuracy which mainly
depends on the pose tracking of the device. As a result, with current AR technology it
is not possible to gather geometrical information with millimeter accuracy which is
desired for product design. Thus, our concept proposes to use Augmented Reality to
determine planning errors and acquire additional information within the AR accuracy.
For cases where AR does not show unambiguous results, for example a cabin element
may or may not clash with a structure element, we propose to perform local 3D scans
with a high-resolution sensor at the beginning of the modification and installation
phase. Since there are only a few locations where the modification and installation
phase can take place it would be possible to have a high-resolution 3D scanner available
at those locations for employees who are trained to use it. Performing local scans in
AST 2019, February 19–20, Hamburg, Germany
5
limited, by the AR system pre-defined locations, is an option to acquire highly accurate
geometrical information in the beginning of the modification and installation phase.
The more precise scanned point clouds can be used either to check possible
clashes of the cabin layout, planned with respect to the ideal, or to update more details
to the planning model. However, in current CAD software there are only a few
functions that allow to work with data point clouds and carry out design clash analyses.
Therefore, software support is being developed to help engineers to find design clashes
of point clouds and CAD models. Moreover additional functions to quickly adjust
current CAD parts or to update the planning model shall be implemented.
4 IMPLEMENTATION OF CONCEPT
In this section we evaluate the accuracy of devices which could be integrated in our
concept show the current implementation of the two technology approaches.
4.1 Accuracy evaluation in aircraft cabin mock-up
Specifications about accuracies and precision of addressed technologies – as they are
stated in data sheets – have been determined with standardized methods under
standardized environmental conditions. But the usability and accuracy of addressed
technologies are – as described in chapter 2 – highly dependent on the application
environment. Therefore, tests were performed in an aircraft cabin mock-up to
determine their capabilities for the addressed application. We considered the accuracy
of 3D scans recorded by these devices as an indication of whether the devices were
usable for the addressed application. 3D-scans can only be accurately taken if the depth
sensor can handle the surface of objects and if the pose tracking works sufficiently in
the environment, in our case an aircraft cabin mock-up.
Cabin mock-up - Figure 3 shows the cabin mock-up and the positions of the reference
points as white dots. The mock-up consists of an aircraft panel with the original frame
and stringer layout. The aircraft floor was rebuilt with true aircraft dimensions and seat
track positions. The dimensions of the whole mock-up are about 4.3 x 2.1 x 3.3 m.
Figure 3 – used positions of reference points in the cabin mock-up
Reference Data - As reference for the evaluation, the position of 28 reference points
are measured with a laser tracker (model: Leica LTD 800) with a submillimeter
accuracy. From each scan the reference points are extracted and distances between all
points are calculated. These distances are compared to the true distances of the
Constantin Deneke, Jan Oltmann, Thorsten Schüppstuhl, Dieter Krause
6
reference points.
3D scanning devices - Figure 4 shows the devices used for accuracy evaluation. The
selection of scanners represents the state of the art of 3D Scanners to a good extend for
large scanning volumes from low to high cost scanning sensors. A high precision
terrestrial scanner, the Faro Focus 3D X130, is used to compare mobile and stationary
measurements. Two different commercially available high precision mobile 3D
scanners as well as a low cost AR system, the Lenovo Phab 2, were chosen to compare
different measuring methods but also different device classes. The Handyscan 3D 700
uses marker-based pose tracking, while the others work markerless. The Artec EVA on
the other side is not fully mobile itself, because it requires a laptop connection during
operation. The Lenovo Phab 2 is a commercially available mobile on Android basis
which uses a built-in time of flight camera for 3D scanning.
Figure 4 - 3D scanning devices used for analysis
mobile
terrestrial
Creaform
Handyscan
3D 700
Artec Eva
Lenovo
Phab 2
(res. 10 mm)
Faro Focus
3D X130
No. of reference points
18
18
28
28
Average deviation [mm]
2.94
4.36
12.67
4.19
Standard deviation [mm]
2.01
2.62
10.57
3.07
Min. deviation [mm]
0.06
0.05
0.08
0.01
Max. deviation [mm]
9.54
10.57
45.36
10.81
Scan boundary dimensions
[mm]
L: 4310
H: 2160
D: 530
L: 4310
H: 2160
D: 530
L: 4310
H: 2160
D: 3340
L: 4310
H: 2160
D: 3340
Table 1 – Measurement results of accuracy evaluation from performed tests in cabin mock-up
Results – The results are depicted in Table 1. The Creaform Handyscan sensor achieves
in the performed tests the best accuracy having an average deviation of 2.94 mm. One
reason for this is that it uses a marker-based tracking system in order to align single
scans accurately to each other. The results of the Artec Eva scanner and the terrestrial
Faro scanner are in a similar order of magnitude around 4 mm in average. However, it
has to be stated that the Artec Eva Scanner originally was not designed for such large
volume scans, but nevertheless gives good results. With Lenovo Phab2 two scans were
recorded, one with a 10 mm resolution which covers the entire cabin mock-up and one
with the 5 mm resolution which covers only a section of it – due to limited
computational power. Both scans show – as expected – a lower accuracy than the high-
AST 2019, February 19–20, Hamburg, Germany
7
end scanning devices. While the smaller scan with the higher resolution - with an
average accuracy of 5.9 mm - delivers comparable results to the high-end devices, the
full mock-up scan has a lower accuracy of 12.6 mm in average. The reason for this is
a combination of a less accurate position tracking system, whose accuracy might
decrease over moved distance, and a less accurate 3D sensor compared to the high end
systems.
It must be concluded that no device delivers an absolute high precision result.
But all scanning devices deliver results that can support cabin conversion planning
dependent on the scenario in which the devices are used. Even the Lenovo Phab 2
results are fairly good and can be sufficient for the intended AR-concept. Furthermore,
it has to be pointed out that presented results are only valid for the described scenario
and test conditions. They do not allow to derivate statements about general device
performances.
4.2 AR digitalisation system
Crucial for any mobile Augmented Reality system is its localization in the environment,
which can be divided into position initialization and tracking. The localization in the
addressed application needs to be set up in a short amount of time due to the restrictions
in the PD-phase. Marker-based approaches require the installation of markers which
can be time-consuming, especially in large environments like aircraft cabins. For the
position initialization the use of a reference model is proposed which is put in place by
user interaction. With this, the device’s coordinate system can be transformed to the
aircraft coordinate system, and is consistent with aircraft and CAD models used for
planning. The position tracking can be performed by markerless visual tracking
methods as indicated to be working robustly in chapter 3. Among the functionalities of
the proposed AR system is the viewing and positioning of virtual models like the cabin
components. With these, the user can recognize potential design clashes or divergences
between expected and real conditions on-site. For proper documentation for later
planning adaption several additional functions are proposed. These include 3D
scanning to acquire the local 3D geometry of the cabin structure and its enrichment
with virtual information by user-interaction. This enrichment includes the ability
to overlay scan data with virtual objects, for example where scan data is of low
quality due to object’s surface properties,
to input localized user annotations, for example for the assignment of part
numbers to objects and
to take localized pictures to give the design engineers a visual reference of a
certain object or region.
An AR system with these functionalities is currently under development. For the first
implementation a Lenovo Phab 2 phablet device is used. Figure 5 shows the application
demonstrating an example task. Shown is the cabin mock-up with augmented virtual
models of brackets (in blue color) which in this case demonstrate all expected positions
of brackets. The user can see if brackets are in the environment which were not
expected or if expected positions of certain brackets differ from the reality. In these
cases the user can provide information about missing or wrongly positioned brackets.
Furthermore, 3D scans can be recorded to generate comprehensive information about
Constantin Deneke, Jan Oltmann, Thorsten Schüppstuhl, Dieter Krause
8
the real geometry of the area.
Figure 5 – AR demonstration example
4.3 Local high resolution refinement
In case local high resolution scanning is required the commercially available terrestrial
or mobile scanning devices can be used. This data can be taken to analyze possible
clash scenarios on the cabin design with respect to the cabin layout.
However, the use of data point clouds in CAD environment is still a problem,
since there are no possibilities to compare data point clouds and CAD data. There is
the opportunity to use mesh data, but there will be a similar problem because the
scanned mesh data is not enclosed and cannot be properly analysed in a CAD system.
For this reason, a software tool in C# for the commercial software SolidWorks via its
API is developed to enhance the use of measurement data in a CAD environment. In
the future the software tool, amongst others, shall allow to
detect design clashes between CAD planning data and measured point clouds
or mesh data, and characterize them for redesign in the CAD system
facilitate a design space analysis based on measured data,
allow fitting and mapping of CAD geometry to scanned data to update planning
data (e.g. new placement of brackets)
output statistical uncertainties for localization with respect to measured data
The design clash analysis is implemented as interference detection of a point within a
CAD part using the ray-casting algorithm. For that infinite rays are taken starting from
every measurement point to a random direction. The number of intersections with the
surfaces of a CAD part determines whether the point is or is not within the part. An odd
number says the point is within the part and vice versa.
In the current state of implementation, the first two aspects have already been
addressed. The focus was then set on the data processing of large point clouds on a
common desktop working station with respect to usability and performance for
usability concerns. This includes, above all, a pre-import data structuring via octree
method [13], a manual selection of the design space of interest, and the consideration
of bounding boxes for every CAD during the analysis part. These aspects fasten the
analysis of the point cloud to less than 10 % of the time without. Using a point cloud
with more than 5 million points and a defined interference with a CAD part as shown
AST 2019, February 19–20, Hamburg, Germany
9
in Figure 6 leads to an analysis duration of 30-40 min.
Figure 6 –Implementation of clash analysis tool in SolidWorks (top) and interference
detection process from aircraft mock-up of aircraft panel and OHSC (bottom)
One difficulty working with point clouds in SolidWorks is a performance reduction of
the SolidWorks environment since API requests are slow. For that reason, it is
necessary to use two different point clouds, a reduced one for the visualization and the
full point cloud for the clash analysis. An example is shown in Figure 6 (bottom). The
difference between the detailed design clash results imported to SolidWorks and the
reduced visualization of the full panel is highlighted. Finally, the clash detection can
either be used for design changes of current CAD models or to update the planning
database accordingly. Here the automatic mapping of existing CAD parts to
measurement data shall facilitate this task.
5 CONCLUSION AND FUTURE WORK
To detect possible clashes before the assembly of cabin components, as wells as to
increase the completeness and accuracy of the planning data, innovative technologies
like AR and 3D scanning are introduced as support tools for the planning of cabin
conversions. For the concept first evaluations are carried out within an aircraft mock-
up at TUHH. The use of combined AR and 3D scanning as presented is a very
promising technology for mobile virtual fit checks and for gathering more information
about the current state of the aircraft cabin. The benefit of the introduced system is its
mobility and thus usability everywhere and by everyone. However, so far, the accuracy
CommandManager
TaskPane
Reduced data
visualization
Full data
Visualization of
clashed area
Clash analysis results Detailed analysis Adapt assembly
Constantin Deneke, Jan Oltmann, Thorsten Schüppstuhl, Dieter Krause
10
of this technology is still limited. But if the accuracy of a scan is not sufficient, a high
precision can be carried out at predefined spots based on the AR application.
Furthermore, the use of the measurement data for quick design changes a plugin for a
CAD software is under development which allows automatic clash detection of
measured and CAD data and facilitate a design space analysis and the model updating
of the planning model. The concept will be further developed and validated in a real
aircraft environment in order to evaluate to what extend and accuracy clashes or design
deviations from the planning base can be detected and updated.
Acknowledgement
This work belongs to the research project “Cabin 4.0” in cooperation with Lufthansa
Technik and is supported by the Federal Ministry for Economic Affairs and Energy as
part of the Federal Aeronautical Research Programme “LuFo V-2”.
6 REFERENCES
[1] Airbus, “Global Networks, Global Citizens: Global Market Forecast 2018-2037”,
(2018).
[2] Boeing, “Current Market Outlook 2017-2036”, (2017).
[3] M. F. Niţǎ, Contributions to aircraft preliminary design and optimization, Dr. Hut,
München (2013).
[4] R. Azuma, Y. Baillot, R. Behringer et al., “Recent advances in augmented reality”,
IEEE Computer Graphics and Applications, Vol. 21, No. 6 (2001), pp. 34–47.
[5] Thomas P. Caudell, David W. Mizell, “Augmented reality: an application of
heads-up display technology to manual manufacturing processes”, (1992).
[6] A. C. Franck Bonard, “Augmented Reality for the factory of the future”, (2016).
[7] K. Pentenrieder, Augmented reality based factory planning, Dissertation (2009).
[8] F. Titov, Technologiegestützte Angebotserstellung für den Umbau komplexer
Investitionsgüter, Dissertation, Technische Universität Hamburg (2016).
[9] M. Tönnis, Augmented reality: Einblicke in die erweiterte Realität, Springer,
Berlin (2010).
[10] V. Raja and K. J. Fernandes, Reverse Engineering: An Industrial Perspective,
Springer, London (2008).
[11] A. Gressin, C. Mallet, J. Demantké et al., “Towards 3D lidar point cloud
registration improvement using optimal neighborhood knowledge”, ISPRS
Journal of Photogrammetry and Remote Sensing, Vol. 79 (2013), pp. 240–251.
[12] C. Heipke, ed., Photogrammetrie und Fernerkundung: Handbuch der Geodäsie,
herausgegeben von Willi Freeden und Reiner Rummel, Springer, Berlin (2017).
[13] J. Liang and J. Gong, “A Sparse Voxel Octree-Based Framework for Computing
Solar Radiation Using 3D City Models”, ISPRS International Journal of Geo-
Information, Vol. 6, No. 4 (2017), p. 106.
