Augmented reality system for aiding engineering design process of machinery systems

Abstract

The paper presents the application of augmented reality for aiding product design and development of machinery systems. Augmented
reality technology integrates an interactive computer-generated word with an interactive real word in such a way that they
appear as one environment. AR technology can enhance a user’s perception of the real world with information that is not actually
part of the scene but is relevant to the user’s present activity. Presented in the AR system is a mode for changing views
of data — especially 3D models — allowing the user to understand the prospective machinery system in a more comprehensive
way, thus making the design process more efficient than the one supported by conventional present-day CAD systems. The presented
prototype system contains an expert system integrated with AR system and allows the delivering of knowledge to the designer
about successive steps of the design process of a mobile robot and practical solutions of realized constructional problems.
An approach concerning AR enables the system user to analyze and verify solutions (represented as 3D models) relative to real
scenes/objects. This approach is advantageous because the real environment around us often provides a vast amount of information
that is difficult to duplicate in a computer. In some cases, the application of an AR system could be an optimal way to verify
developed products.

KeywordsAugmented reality–human-computer interface–CAD–design for customer satisfaction–computer-aided decision making

Full text

M. Januszka, W. Moczulski, Augmented reality system for aiding engineering design process of machinery systems
Published in: Journal of Systems Science and Systems Engineering, 20 (3), Springer, 2011, pp.294-309
AUGMENTED REALITY SYSTEM FOR AIDING ENGINEERING DESIGN
PROCESS OF MACHINERY SYSTEMS

Marcin Januszka
1
Wojciech Moczulski
1
1
Department of Fundamentals of Machinery Design, Silesian University of Technology,
Konarskiego 18A, 44-100 Gliwice, Poland
marcin.januszka@polsl.pl (), wojciech.moczulski@polsl.pl
Abstract
The paper presents the application of augmented reality for aiding product design and development
of machinery systems. Augmented reality technology integrates an interactive computer-generated
word with an interactive real word in such a way that they appear as one environment. AR technology
can enhance a user’s perception of the real world with information that is not actually part of the scene
but is relevant to the user’s present activity. Presented in the AR system is a mode for changing views
of data - especially 3D models – allowing the user to understand the prospective machinery system in a
more comprehensive way, thus making the design process more efficient than the one supported by
conventional present-day CAD systems. The presented prototype system contains an expert system
integrated with AR system and allows the delivering of knowledge to the designer about successive
steps of the design process of a mobile robot and practical solutions of realized constructional
problems. An approach concerning AR enables the system user to analyze and verify solutions
(represented as 3D models) relative to real scenes/objects. This approach is advantageous because the
real environment around us often provides a vast amount of information that is difficult to duplicate in
a computer. In some cases, the application of an AR system could be an optimal way to verify
developed products.
Keywords: augmented reality, human-computer interface, CAD, design for customer satisfaction,
computer-aided decision making

This scientific work is partially financed by the Ministry of Science and Higher Education (Poland) – grant No.
NN502 448339 “The method for aiding design process with the use of Augmented Reality”.
1. Introduction
Technical evolution causes people to design
more and more complex technical objects with
the use of methods and tools from various
disciplines. Nowadays, almost every design
office deals with computer techniques. A large
number of tasks included in the product
development process can be computer aided.
Some problems without computer-aided systems
could be non-solved (Oprzedkiewicz 1993).

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
2 J Syst Sci Syst Eng (2011) 20(3): 294-309
Companies in machinery systems design field
have based their products development process
on Digital Mock-Up (DMU) and virtual
techniques. A typical session of design
evaluation and review is performed with the use
of virtual reality techniques where features of
new products are investigated. The main goal of
this approach is to reduce the needs of Physical
Mock-Up (PMU) building. The expected
benefits of using Virtual Technologies are the
reduction of development time, reduction of
development costs and increasing reliability and
durability of future products for customers
satisfaction. To improve and optimize the design
and development process in modern companies
are adopted more and more often new Virtual
Technologies like virtual and mixed reality (see
(Dunston et al. 2002, Dunston et al. 2000, Nölle
and Klinker 2006, Shin et al. 2005, Wang and
Dunston, 2006)).
Virtual and mixed reality techniques allow
the designer to make proper decisions during the
design process (e.g. conceptual and engineering
design) and to evaluate the reliability of a future
product. It is important problem to adequately
present (visualize) the product (Dunston et al.
2000, Milgram et al. 1994, Wang and Dunston
2006). Thanks to modern visualization systems
it is possible to maximize legibility of product
models (e.g. possibility of visualization in 1:1 or
higher scale). An innovative and effective
solution to help solve these problems is the
application of augmented reality (AR)
technology – variety of mixed reality. In the last
years this innovative technology is used for
aiding designers in an efficient way (Dunston et
al. 2002, Dunston et al. 2000, Milgram et al.
1994). AR involves the superposition of
computer graphics (most often 3D) over real
scenes (predominantly) (fig. 1), viewing through
head-mounted devices (HMD) or handheld
displays (HHD).
The real environment around us provides a
lot of information that is difficult to duplicate
precisely in a computer (Novak Marcincin
2007). For example, when we want to analyse a
designed machine from the point of view of
safety in considering factory layout, we don’t
want to make a difficult virtual 3D model of the
whole factory environment because it can be
costly and time-consuming. AR helps designers
intuitively determine whether the machine in
factory layout meet qualitative indicators, e.g.
safety and aesthetics. Sometimes it is not
Figure 1 Reality-virtuality continuum (Milgram et al. 1994)

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
J Syst Sci Syst Eng
3
necessary to elaborate a virtual 3D model
because of the possibility of using an existing
real object or an environment for much less cost.
AR helps alleviate limitations of VR by placing
the virtual objects side-by-side with real objects.
AR technology can enhance a user’s perception
of the real world with information that is not
actually part of the scene but is relevant to the
user’s present activity (Azuma 1997). It
provides a natural and intuitive means by which
the user can work more efficiently in a real
world environment. An AR-based CAD system
should allow users to move around a virtual
product in a real environment to visualize this
product through the HMDs and interact with this
virtual product.
Several systems trying to apply AR in
industry have been developed. In (Regenbrecht
et al. 2005) the authors present research,
development and deployment of AR systems in
the automotive, aviation and astronautics
industries. In (Navab 2004) the author
summarizes research realized by an industrial
AR (IAR) consortium which supports
augmented reality for development, production,
and servicing in industry (especially in
automobile industry). In (Klinker et al. 2002) the
authors describe a presentation system for
product design called Fata Morgana. The Fata
Morgana system aids the process of modeling
cars.
In this paper the authors concentrate on a
new engineering tool. The paper describes
successive results of research on augmented
reality technology for aiding machinery systems
design and development, which are
a continuation of the research initiated by the
authors in 2005 (Moczulski et al. 2007). The
research presented in this paper system can be
useful in the Systems Engineering Process as an
engineering tool for developing products that
satisfy customer needs. Some parts of the
system can aid in understanding customer needs,
establishing the need for change, discovering
requirements and defining system functions. An
intuitive tool for visualisation helps presenting
ideas and intentions of designers to customers
and inversely. Thanks to that, the product
development process can be customer friendly.
Another part of the system can aid designers in
decision making during the development process
and thanks to that reduce product development
time and costs. The application of augmented
reality systems is a concept to streamline the
Product Development Process (PDP) and its
stages in this area of scientific research.
2. Research Background
Our research concentrates on improving the
design process. The goal of research is to
elaborate on the method and system for more
efficient machinery systems development with
the use of modern techniques of visualization.
The authors identify a new problem in systems
engineering and product development process
domains, that is, the integration of system aiding
decision making with an augmented reality
system. The problem that the authors intend to
study is the elaboration of method collecting and
the representation of data and knowledge for
such a system.
The method should allow the design of more
reliable and durable machinery systems for
customer satisfaction. The system should aid the
designer (or a group of designers) during
conceptual and engineering design (especially in

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
4 J Syst Sci Syst Eng (2011) 20(3): 294-309
the virtual product verification phase). Designers
in the engineering design field notice the ease of
presenting ideas and design is very important
issue.
At the beginning, the authors decided to
elaborate various mechanisms of searching,
collecting, processing of data (especially 3D
models) necessary for the designer during
designing of a new product. An elaboration of
a system for presenting knowledge and data to
the user was the next research problem. The
designer should be able to use data and
knowledge from the system during a design
process (Skarka 2007). For that purpose AR as a
tool to control a dialogue between the designer
and the CAD system and the system aiding
decision making could be used. The system
should allow visualization of data about:
existing constructional solutions, symptoms of
failures and inefficiencies (e.g. exceeded limit of
vibrations, noise, temperature, pressure etc.) in a
previous version of a product, algorithms of
design processes of a selected group of
machinery systems (e.g. mobile robots) or
machinery parts and elements, critical points of
the design process etc. In the case of 3D models
(not only, but especially) should be possible to
display them in any scale (especially 1:1 scale)
with the possibility of viewing from any
perspective in a very intuitive way.
3. The Prototype System for Aiding
Designers
The system presented in the paper belongs to
a group of personal designer’s assistants. A
characteristic feature of the tool is application of
augmented reality techniques. The authors
carried out an implementation of the prototype
system for a design process of mobile robots.
Although, it is also possible to build the system
for any machinery systems.
3.1 Components and Architecture of the
System
The basic component of the implemented
system is a computer with MS-Windows® XP
system installed. The computer runs Dassault
Systemes CATIA V5R19 (modeling software)
and a special elaborated AR application. The
HMD with a small USB camera attached is
connected to a video card of the computer. The
video camera captures video of the real world
and sends it to the computer. HMD allows to see
data from the computer (e.g. 3D models, text
etc.) in real environment surrounding the user
(Januszka and Moczulski 2006). The user wears
the HMD with the video camera attached, so
that when she/he looks at the tracking card with
a special marker through the HMD a virtual
object is seen on it (fig. 2).
A very important and difficult part of the AR
system proposed by the authors is software. The
presented system consists of the following
(fig. 3): ARToolKit tracking libraries, VRML
(Virtual Reality Modeling Language) parser,
expert system (system aiding decision making)
with database, Dassault Systemes CATIA
V5R19 (modeling software), a main application
to integrate all software components and to
realize system functions. The database contains
3D models (in CATPart or VRML formats) and
the others sources of data (drawings,
spreadsheets, diagrams etc.). In the system, all
software components are integrated. If
necessary, software components communicate
with each other and transmit data (e.g. CATIA

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
J Syst Sci Syst Eng
5
Figure 2 System for aiding the designer of machinery systems (components) (Januszka and Moczulski 2010)
system can transmit new data to a database, a
database can transmit existing data to CATIA
system or the system for aiding decision
making).
Figure 3 Organization chart of the system
The system base on the public-domain
augmented reality tracking library called
ARToolKit (from HIT Lab (HitLab 2011)) with
LibVRML97 parser for reading and viewing
VRML files. ARToolKit is a software library
that uses computer vision techniques to precisely
overlay VRML models (3D models, text,
pictures etc.) onto the real world. For that
purpose software uses markers. Each marker
shows a different digitally-encoded pattern on it,
so that a unique identification of each marker is
possible. In the presented conception the
markers are printed on cards. We can compute
the user’s head location as soon as the given
marker is tracked by the optical tracking system.
Finally the main application allows the display
of data superimposed on the real world (exactly
on the card with the marker position and
orientation). The process of video-based marker
detection and overlay of virtual objects by
ARToolKit presents fig. 4.
An important aspect of the presented system
is usability and user-friendlyness. The main
features of the software are controllable by an
easy to use graphical user interface. Various
features were included when implementing the
structure of the GUI, simplicity being the most
important. The user interface consists of
graphical elements such as windows, menus,

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
6 J Syst Sci Syst Eng (2011) 20(3): 294-309
Figure 4 The process of video-based marker detection and overlay of virtual objects (Januszka and Moczulski
2007)
radio buttons, check boxes. The user interface
employs a keyboard to input data (future
research will aim at using another input devices,
eg virtual hands and gesture control). An
elaborated graphical user interface (GUI)
enables, among others, the following: (Januszka
and Moczulski 2010):
 update of knowledge in a
knowledgebase (by a Knowledge
Engineer),
 viewing instructions regarding stages of
a design process of a mobile robot (the
system aids the designer in a design
process) and solutions proposed by an
expert system,
 previewing of data from a database
(tables, drawings, pictures, text, voice
or video information),
 use (by export to CAD system) of
existing 3D models to improve them,
 viewing of results of a designer's work
(e.g. 3D models, results of simulations:
kinematics, structural strength, thermal
strength), in AR mode etc.
All displayed information (especially 3D
models) could be viewed in the real environment
in full 3D mode and 1:1 scale.
3.2 Functions of the System
In the presented conception of the system the
user with HMD on the head sits in front of a
computer. The user looks in the direction of the
card with marker(s) through HMD and virtual
objects are seen onto this card. As virtual objects
the text information, 3D models, pictures,
catalogue tables, drawings from the database or

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
J Syst Sci Syst Eng
7
instructions from the knowledgebase could be
displayed (fig. 5). Information displayed with
the use of AR system are very helpful for the
designer. At the same time the user can design
the mobile robot in CATIA CAD system.
The user can view models of existing robots
or components of robots in AR mode (in 1:1
scale, from any perspective). It is possible to
export/import these parts between modeling
software CATIA V5R19 and AR system. The
designer in the CATIA’s workplane can see the
part and simultaneously this part could be seen
in AR mode (fig. 6). The designer can evaluates
the design and if necessary goes back to the
modeling in order to correct and improve the
details (Klinker et al. 2002). When the design
process is accomplished the user can preview
results of his/her work in AR environment.
The designer can manually manipulate the
model for inspection. It is also possible to export
the models with results of analysis e.g. strength
analysis and the others. Analysis can be
performed with the use of FEM method in
CATIA system. The results of analysis are
presented in the form of animated and
interactive virtual 3D models in AR mode,
instead on a flat computer monitor. Other
designers can sit around a table with a marker
and examine the design of a robot in a three-
dimensional virtual image of the 3D model.
Designers can inspect models of products, walk
around it, and compare it with other models just
like they are used to looking at real (physical)
objects. An inspection is possible from any
viewpoint where each person has their own
viewpoint to the model. They are also free to
interact with the model in real time.
3.3 Designer’s Assistant
One of the goals of the implemented system
was to aid the designer of machinery systems in
designing more reliable and durable mobile
robots for customer satisfaction. For that
purpose the system should deliver helpful
information to the user during designing.
Information in the proposed system should be
viewed in the real environment from any
perspective in a very efficient and intuitive way.
This approach enables the user to analyze and
verify some solution (represented as 3D models)
relative to real scenes/objects. This approach is
advantageous because the real environment
Figure 5 Previewing data from database during designing

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
8 J Syst Sci Syst Eng (2011) 20(3): 294-309
Figure 6 Previewing results of a designer’s work and manually manipulating the model for inspection (Januszka
and Moczulski, 2010)
around us often provides lots of information that
is difficult to duplicate in a computer.
In the presented system knowledge
(necessary during design process) to the
knowledgebase is inscribed in the
knowledgebase thanks to an electronic form.
Design knowledge is acquired from: domain
experts, a professional/scientific literature or
results of research (exploitation, laboratory etc.)
collected in databases (Moczulski 1997).
Initially, the knowledge is expressed verbally.
A
verbal statement is transformed to an expression
that can be executed by the system. Knowledge
to a knowledge base is added, modified and
updated with the use of a knowledge base editor
by a knowledge engineer. Knowledge
(represented in the form of procedures and rules)
and data are used in the design process to
eliminate causes of failures and inefficiencies in
future products. Nevertheless, further research
can be applicable to system methodologies such
as MOCA. An unique approach called MOCA
(Methodology and Tools for Knowledge Based
Engineering) can support procedures to
interview experts, ontological schemas to
organise knowledge and tools for representing
and publishing knowledge across the
organization (Stokes 2001, Bermell-Garcia and
Fan 2008).
Data is collected in the database. The
designer can take advantage of data or
knowledge as often as needed. These data
should be concerned about existing
constructional solutions. Data adequate to solve
problems are displayed in a real environment
around the user while he/she is making
decisions. These data could be used by the user
to design new products or improve existing
constructional solutions in order to increase
reliability.
The discussed system allow for displaying
data in AR mode in the following forms:

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
J Syst Sci Syst Eng
9
 3D models,
 drawing documentation,
 diagrams,
 BOM’s,
 spreadsheets with calculation models,
 catalogue data (text and picture data).
In the database the data is also collected (if
required) in form of sounds.
The expert system delivers to the designer
knowledge about successive steps of the design
process of a mobile robot and practical solutions
of realized constructional problems. The expert
system uses rules to make deductions or choices.
A rule-based system is used to help make
engineering decision making more reliable. The
rule-based system contains a rule base which is a
specific type of knowledge base. The rule base
contains the following example rules:
 IF Robot is small THEN Robot is made
from aluminum (CF: +80%),
 IF Robot is able to move in a varied
terrain THEN Robot has caterpillar
drive (CF: +70%),
 IF Robot is shockproof THEN Robot
has brushless DC electric motor (CF:
+99%),
 IF Drive system is simple THEN Drive
system is four-wheeled (CF: +99%).
A forward chaining method of reasoning when
using inference rules is used. A CLIPS (C
Language Integrated Production System) a
public domain software tool was used for
building the expert system. A CLIPS expert
systems shell is a complete development
environment for building and maintaining
knowledge-based applications. The CLIPS shell
was used in probably first knowledge-based
augmented reality system called IBIS (Feiner et
al. 1993).
A user interface from the expert system shell
is integrated with a main GUI. In simplification,
after entering necessary input parameters (e.g.
an approximate size, speed or application of the
robot), it is possible to choose a constructional
solution proposed by the system (fig. 7). The
proposed solution as a 3D model is exported to
the CATIA’s workplane. The designer develops
a new constructional solution based on the basic
solution proposed by the system.
At each stage of the design process the
designer can preview the results of his/her work
in the AR environment. Moreover the user is
able to verify elaborated solution relative to real
scenes/objects (e.g. a dimensional analysis in
order to check the robot ability to drive between
two obstacles).
The system aids the designer in decision
making with the use of a modern AR
communication human-computer interface.
Prompts inferred by the expert system after user
acceptance are automatically applied in a
developed product.
The system is designed not only to solve
some problems in designing process but it is also
used as application leading the process. A
characteristic of the system is the ability to
deliver to the designer knowledge about
successive steps of the design process of a
mobile robot.
The expert system aids the
designer only in qualitative way (qualitative
visual analysis of design aspects).
Future research are going to aid the designer

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
10 J Syst Sci Syst Eng (2011) 20(3): 294-309
Figure 7 Application of system aiding decision making in the design process
also in more advanced way. For example,
solutions proposed by the system will be able to
automatically influence some parameters of the
virtual 3D model, e.g. choice of a new size of
DC motor will cause a change of motor mounts
etc.
4. Verification
The verification of the system was realized
only in an easy way because of early stage of the
system development. The system evaluation was
carried out only on the base of special forms
(questionnaires) completed by designers after an
experiment. The findings are principally derived
from an empirical study into the perceptions of
using AR technology for mechanical design
activity. Within a framework of a verification of
research, some experiments depend on
realization of a complete design process of a
mobile robot were carried out. Experiments
were realized with the use of presented in this
article system and without. Some experienced
(Group 1) and non-experienced (Group 2)
designers were selected to take part in the
experiment (total 15 designers). One of the goal
of the verification was to confirm that with the
use of the system it was possible to reduce
product development time.
As a result of using the system the designer
has the possibility to choose a faster way to
develop an optimal constructional solution.
Sometimes solution proposed by the system is

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
J Syst Sci Syst Eng
11
also a better solution to the problem than when
the user does not use the system. For example,
the system can give advice that in specific
conditions application of caterpillar drive
instead of four-wheeled drive is worse. Non-
experienced designers have more efficient
access to knowledge, so they are able to finish
the design process faster than designers which
don’t use the presented system. Efficiency of
the system was confirmed, thanks to shortening
development time of an elaborated product
(mobile robot):
 in the case of experienced designers the
development time was shortened about
10%,
 in the case of non-experienced
designers the time was shortened about
20%.
A controlled experiment dependent upon
realization of a design process is prohibitively
difficult or impossible. In the current state of
development of the system, the authors resorted
to natural experiments. The natural experiments
relied solely on observations of the development
of a time variable. Experiments took place in a
natural environment (design office). To the
degree possible, the authors tried to collect data
for the system in such a way that contributions
from all variables was determined, and where
the effects of variation in certain variables
remained approximately constant so that the
effects of other variables could be discerned.
Questionnaires were designed and
administered in order to identify the advantages
derived from AR technology. Questions were
organized as product-related, process-related
and open essay questions. Questionnaires were
designed to be completed based on subjective
experiences and feelings during the experiment.
Summary of the empirical study presents table 1.
In open essay questions respondents
specified another important advantages of the
variations of an existing product. The system
can be used to superimpose modifications on
AR system. One of the advantages of the
presented system is in connection with an
existing real product without needing to model
the entire existing part. The user can compare
Table. 1 Fragmentary results of a statistical survey
Questions
GROUP 1 GROUP 2
Yes
[%]
Y/N
[%]
No
[%]
Y
[%]
Y/N
[%]
N
[%]
Does AR system better facilitate product development
and design decision-making?
75 12,5 12,5 85,7 14,3 0
Does AR system help you to understand and interpret the
inputs of your design collaborators?
87,5 12,5 0 100 0 0
Does AR system better facilitate spatial cognition during
the design process?
87,5 12,5 0 85,7 14,3 0
Does AR system have positive impact on quality of
design output?
62,5 25 12,5 71,4 14,3 14,3
Does AR system improve designs presentations and
therefore have value-added impact?
75 25 0 100 0 0
Does AR systems help in reducing lead-time and costs
without producing physical Mock-Ups?
75 25 0 85,7 14,3 0
Y- Yes, N- No, Y/N- I don’t know

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
12 J Syst Sci Syst Eng (2011) 20(3): 294-309
some elaborated constructional solutions in
relation to real product.
As well, realized experiments confirm that
the application of an AR system in some cases
could be an optimal way to verify developed
products. In an example presented in fig. 8 a
virtual model of a mobile inspection robot was
developed. In a verification phase the designer
intend to prove that the inspection robot is able
to enter to a robot transporter (e.g. verification
of a dimensions of the inspection robot). For
that purpose the designer can use the AR
system. In this case it is not necessary to
elaborate the model of the robot transporter
because it is possible to use a real robot
transporter. We can use the real robot
transporter and virtual model of the inspection
robot.
Detailed verifications will be realized
during further development of the system, and
the authors are going to present the results in
successive publications.
5. Conclusions and Future Work
The development of AR technology in
design (but not only, also in maintenance and
diagnostics) has been initiated by the author
and research team from Department of
Fundamentals of Machinery Design few years
ago (Januszka and Moczulski 2006). The
results of research confirm advantages from
application of AR techniques in a design domain.
Figure 8
An example of verification process for the elaborated inspection robot

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
J Syst Sci Syst Eng
13
In this paper, the authors present a system
which belongs to a group of personal designer’s
assistant. The presented system delivers helpful
information and some constructional solutions to
the user during designing. A characteristic
feature of the tool is the application of
augmented reality techniques enabling the
visualisation of virtual content (3D models,
diagrams, tables, pictures etc.) in real
environment in front of the user. The authors
outline an approach to integrate AR into the
product development process. Implementation
of the prototype system for a design process of
mobile robots was described.
The presented system is in an early stage of
development. However, advantages of a system
based on AR confirm the future possibility of
introducing augmented reality technology in
design engineering enterprises (especially those
of small and medium size).
Using a presented AR system in CAD brings
some advantages. First of all, the user has the
possibility to preview data (especially 3D
models) in real environment, directly around
users at any place, from any perspective (each of
designers have own perspective), and in any
scale. AR also gives the possibility of totally
interacting with the displayed data by
manipulation of position and orientation in a
space around the user. It was proved (see also
(Shin et al. 2005)) that perception of 3D designs
is better when changing views by observer
movement than by model rotation (e.g. with the
use of the standard monitor). The AR mode for
changing views of data - especially 3D models -
allows the user to understand the prospective
system in a more comprehensive way, thus
making the design process more efficient than
the one supported by conventional present-day
CAD systems. The designer has the possibility
for fast and efficient verification of designed
products in order to increase reliability.
Results of research confirm other particular
advantages derived from the presented system
using AR techniques, especially:
 extended efficiency of direct access to
data and knowledge which is necessary
during a design process,
 possibility of aiding decision making
and delivering detailed design
algorithms to improve reliability of
designer machinery systems,
 an improved mechanism of making full
use of existing knowledge and data
(possibility of importing/exporting data
between a knowledgebase/database and
CAD system),
 possibility of presenting data necessary
during a design process (e.g. tables,
results of research, schemes, 3D
models) and results of this process (e.g.
final 3D models) in efficient way, with
possibility of interaction with these
data and viewing from any perspective
and also in any scale (also 1:1 scale).
The results of verification show that designers
agree that an AR system helps them to better
understand and interpret the inputs of a design
process without a physical Mock-Up and help in
reducing product development time and costs.
Natural experiments confirm the reduction of
product development time of approximately
10÷20% in relation to processes without the use
of virtual technologies. The presented system, as
in the case e.g. Ford’s virtual reality cave, is
helping the company's product engineers

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
14 J Syst Sci Syst Eng (2011) 20(3): 294-309
evaluate design before production. Ford’s lab
reduces the need to build physical prototypes
and trims thousands of dollars and several
months off the product development process
(Ford Motor Company 2009). Moreover, in the
opinion of the designers, AR systems have a
positive impact on the quality of design output
(also due to the benefit of the integrated system
aiding in decision making).
Adapting AR to practical uses is connected
with various problems (see reports included in
(Ong and Nee 2004)), especially concerning
hardware technologies. Our experience with the
system has suggested research of problem with
tracking technology (e.g. marker identification)
and the quality of the visualization process that
need to be explored. However, these problems
are raised by other researchers (see (Zhou et al.
2008)) and our study didn’t encounter these
problems. Although, there are still some
problems to solve, but for industrial partners to
be interested in investing into this technology its
possible benefit and its integration into the
whole company has to be visible.
There are several open issues related to the
proposals presented in this paper that need
further study. Future work includes further
system development, user tests, and evaluations.
One of the possible issues which could be
studied is how the knowledge should be
managed in general. Further research can be
applicable to advanced methodologies such as
MOCA. In our opinion, future research should
cause further rationalization of the design
process: shortening product development time,
reducing product development costs.
Next (more technical) issue relates to the use
of VRML files. A standard VRML format
(especially for internet presentations) for AR
system is easy to read but does not support
advanced lighting and texturing (Liarokapis
2007). For this reason, the authors will use
another format for future 3D models, e.g. 3ds.
Moreover, to improve the realism of the AR
scene a fast algorithm for planar shadows and
reflections should be implemented.
As well, an aspect that requires more study is
related to one of the most interesting phases of
the product development process: conceptual
design. In modern approach for conceptual
design it is important to explore (create and edit)
and present ideas in an easy way. For that
purpose the authors are going to apply
augmented reality techniques in conceptual
design. If possible AR should aid the designer in
creating conceptions (sketches, models etc.)
directly in real environment, comparing theirs
and selecting optimal one.
6. Acknowledgment
The authors would like to thank anonymous
reviewers for valuable comments and helpful
suggestions to improve the quality of this paper.
References
 Azuma, R.T. (1997). A Survey of Augmented
Reality. Teleoperators and Virtual
Environments, 6(4): 355-385.
 [Bermell-Garcia, P. & Fan, I. S. (2008).
Practitioner requirements for integrated
Knowledge Based Engineering in PLM.
International Journal of Product Lifecycle
Management, 3(1): 3-20.
 Dunston, P.S. & Wang, X. & Bilinghurst, M.
& Hampson, B. (2002). Mixed reality
benefits for design perception. International

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
J Syst Sci Syst Eng
15
Symposium on Automation and Robotics in
Construction, pp. 191-196, Gaithersburg,
Maryland, September 23-25, 2002, National
Institute of Standards and Technology.
 Dunston, P.S.& Bilinghurst, M. & Luo, Y. &
Hampson, B. (2000). Virtual visualization for
the mechanical trade. International
Symposium on Automation and Robotics in
Construction, pp. 1131-1136, September 18-
20, 2000, National Taiwan University.
 Feiner, S. & MacIntyre, B. & Seligmann, D.
(1993). Knowledge-based Augmented
Reality. Communications of the ACM, 36(7):
53-62.
 Ford Motor Company (2009). Improving
New Product Development Process.
Sustainability Report 2009/10. Available at:
<http://corporate.ford.com/microsites/
sustainability-report-2009-10/economy-
recovery-team-development>. Cited May 25,
2011.
 HITLab at the University of Washington -
ARToolKit Documentation. Available at:
<http://www.hitl.washington.edu/artoolkit/do
cumentation/>. Cited January 24, 2011.
 Januszka, M. & Moczulski, W. (2010).
Augmented reality for machinery systems
design and development. In: Pokojski, J.,
Fukuda, S., Salwinski, J. (eds.), New World
Situation – New Directions in Concurent
Engineering (Advanced Concurent
Engineering Series), pp. 79-86, Springer;
Berlin Heidelberg.
 Januszka, M. & Moczulski, W. (2007).
Machinery design aided by augmented
reality technology. Computer Assisted
Mechanics and Engineering Sciences, 14:
621-630.
 Januszka, M. & Moczulski, W. (2006).
Collaborative augmented reality in CAD
design. Machine Dynamics Problems, 30(3):
124-131.
 Klinker, G. & Dutoit, A. H. & Bauer, M. &
Bayer, J. & Novak, V. & Matzke, D. (2002).
Fata Morgana – A Presentation System for
Product Design. International Symposium on
Mixed and Augmented Reality, pp.76-85,
Darmstadt, Sept. 30 - Oct. 1, 2002, IEEE
Computer Society.
 Liarokapis, F. (2007). An augmented
reality interface for visualizing and
interacting with virtual content. Virtual
Reality, 11: 23-43.
 Milgram, P. & Takemura, H. & Utsumi, A.
& Kishino, F. (1994). Augmented reality: a
class of displays on the reality-virtuality
continuum. Telemanipulator and
Telepresence Technologies, 2351:282-292.
 Moczulski, W. (1997). Methods of
Knowledge Acquisition for the needs of
machinery diagnostics (Monograph, in
Polish). Publishing House of Silesian
University of Technology, Gliwice.
 Moczulski, W. & Panfil, W. & Januszka,
M. & Mikulski, G. (2007). Applications of
augmented reality in machinery design,
maintenance and diagnostics. In: Jablonski,
R., Turkowski, M., Szewczyk, R. (eds.),
Recent Advantages in Mechatronics, pp. 52-
56, Springer-Verlag, Berlin Heidelberg.
 Novak Marcincin, J. (2007). Augmented
virtual reality applications in manufacturing
systems. International Multidisciplinary
Conference. pp. 565-572, Baia Mare
(Romania) . May 17-18, 2007, North
University of Baia Mare.

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
16 J Syst Sci Syst Eng (2011) 20(3): 294-309
 Navab, N. (2004). Developing Killer Apps
for Industrial Augmented Reality. IEEE
Computer Graphics and Applications; 24(3):
16-20.
 Nölle, S. & Klinker, G. (2006). Augmented
Reality as a Comparison Tool in Automotive
Industry. International Symposium on Mixed
and Augmented Reality, pp. 249-250, Santa
Barbara, USA, October 22-25, 2006, IEEE
Computer Society.
 Ong, S.K. & Nee, A.Y.C. (eds.) (2004).
Virtual and Augmented Reality Applications
in Manufacturing. Springer.
 Oprzedkiewicz, J. (1993). Computer-aiding
in reliability of machines (in Polish). WNT,
Warsaw.
 Regenbrecht, H. & Baratoff, G. & Wilke,
W. (2005). Augmented Reality Projects in
the Automotive and Aerospace Industries.
IEEE Computer Graphics and Applications,
25(6): 48-56.
 Shin, D.H. & Dunston, P.S. & Wang, X.
(2005). View changes in Augmented Reality
Computer-Aided-Drawing. ACM
Transactions on Applied Perceptions, 2(1): 1-
14.
 Skarka, W. (2007). Methodology of
knowledge-based engineering (Monograph,
in Polish). Publishing House of Silesian
University of Technology, Gliwice.
 Stokes M. (ed.) (2001). Managing
Engineering Knowledge. MOKA:
Methodology for Knowledge-Based
Engineering Applications. American Society
of Mechanical Engineers, New York.
 Wang, X. & Dunston, P.S. (2006). Potential
of Augmented Reality as an Assistant Viewer
for Computer-Aided Drawing. Journal of
Computing in Civil Engineering, 20(4): 437-
441.
 Zhou, F. & Ben-Lim Duh, H. &
Billinghurst, M. (2008). Trends in
Augmented Reality Tracking, Interaction and
Display: a Review of Ten Years of ISMAR.
International Symposium on Mixed and
Augmented Reality, pp. 193-202,
Cambridge, September 15-18, 2008 IEEE
Computer Society.
Marcin Januszka is currently a Ph.D. student in
Departament of Fundamentals of Machinery
Design at Silesian University of Technology,
Gliwice (Poland). His research interests include
augmented and virtual reality technologies,
methods in computer-aided design, design of
mobile robots. His current research topic is
application of augmented reality in product
development process. He received his M.Sc.
degree in Management and Production
Engineering from the Silesian Univerity of
Technology (Poland) in 2007. He has
participated in several research projects in the
field of mobile robotics and virtual reality.
Wojciech Moczulski is a professor in the
Department of Fundamentals of Machinery
Design at Silesian University of Technology at
Gliwice (Poland). His researches are focused on:
design and operation of the machines, computer
science, particularly in machinery diagnostics
and application of methods and means of
Artificial Intelligence. He is author and co-
author of over 130 publications. He is the holder
of scholarship of Humboldt’s Foundation at
University in Paderborn (Germany). He
managed researches at Wichita State University

Januszka and Moczulski: Augmented reality system for aiding engineering design process of machinery systems
J Syst Sci Syst Eng
17
and University of North Carolina in Charlotte
(USA) and many others. Since 2005 he has been
Associated Editor of “Engineering Applications
of Artificial Intelligence”. Since 2002 he has
been serving as Organizing Committee Chair of
the international Symposium on Methods of
Artificial Intelligence (AI-METH). He is
founder and member of the Central Board of the
Polish Society of Technical Diagnostics.