Content uploaded by Ralf Elbert
Author content
All content in this area was uploaded by Ralf Elbert on Feb 12, 2020
Content may be subject to copyright.
Virtual Reality in Logistics – Opportunities and
Limitations of Planning & Training in Logistics
Prof. Dr. Ralf Elbert, Professor, Chair of Management and Logistics, Technische
Universität Darmstadt, Germany
M. Sc. Tessa Sarnow, Research Associate, Chair of Management and Logistics,
Technische Universität Darmstadt, Germany
M. Sc. Jan-Karl Knigge, Research Associate, Chair of Management and Logistics,
Technische Universität Darmstadt, Germany
1. Introduction
Due to the advances in regards to computation, processing of data, sensors and related
technology, the development of virtual reality (VR) has made notable progress in the last
years. In comparison to the costly and complex setups used in the past (see for example
(Kammergruber, Günthner 2010; Wulz 2008)) to-day’s technological options allow a
user-friendly implementation with easy setup via head-mounted displays (HMDs) with
comparably high processing power. At the same time, the costs are decreasing so that VR
HMD’s are widely spread and liked in the gaming sector and for entertainment purposes
today. Usage in industrial applications is only emerging by and by and mainly realized in
form of pilot projects that identify interesting potentials (Goisbault et al. 2017; Schenk
et al. 2005). With ongoing development of the VR-hardware, the costs are expected to
decrease further, while performance of hardware as well as software will increase. Thus,
application of VR in planning of logistics solutions and training has signicant growing
potentials (Peger, Wehking 2017).
A fact often neglected is, that a virtual model can only rebuild the physical reality up to a
certain level of sensory delity. The term ‘sensory delity’ in the literature also referred
to as immersion describes the degree of equivalence between VR and physical reality in
presentation of information via diverse channels (Bowman, McMahan 2007; Bystrom et
al. 1999). The different factors of a simulation in VR can show different levels of sensory
delity. For example, the sensory delity regarding item weight is rather low, while the
visual presentation of the item might be almost the same as in physical reality and thus
sensory delity therefore is high. The transfer of planning or training output from VR
to physical reality might be aficted with information losses in such extent as sensory
delity is limited (Cummings et al. 2012; Dörner at al. 2013; Jung, Vitzthum 2013; Slater
2010). Information losses, in the following referred to as losses, are all those factors
of physical reality that cannot be modelled in VR, like the item weight in the example
mentioned. Consequently, it can be expected that such losses affect the transferability
D1 New technologies in warehousing 150
of planning and training outcomes. To remain with the example of item weight, one can
imagine that there are differences in moving an item without weight as in VR compared
to moving the item with notable weight in physical reality. Consequently, the picking
times for virtually heavy items is signicantly lower than the picking times for these items
in physical reality. Thus, the loss occurs in form of missing simulatability of the weight
which in turn lowers transferability for this aspect. The research gap is represented by
the lack of empirical research on transferability of planning and training outcome from
VR to physical reality. Related to this is an in-depth investigation of the characteristics
of VR that cause losses leading to limited transferability. The relevance in researching
the transferability is illustrated by the risk of planning errors or lack of training results.
To fully exploit the potentials of VR technology for planning and training in the logistics
context, it is necessary to be aware of the limitations it brings along. Otherwise the
robustness of planning results on one hand and effectivity of training sessions on the
other hand need to be questioned.
This paper contributes to narrow the aforementioned research gap by comparing VR and
physical reality in regards to a model’s sensory delity and identifying potential losses
resulting from deviations in the simulation. Concluding, the resulting transferability of
planning and training outcomes is derived. The respective research questions (RQ) are:
RQ 1. Which factors of physical planning and training can be rebuilt in VR with
high sensory delity?
RQ 2. What are the losses between a virtual planning solution or training and its
physical counterpart?
RQ 3. How is transferability of planning and training outcomes from a VR simulation
to physical reality affected by the identied losses?
VirtualrealityPhysical reality
Aspect 1
Aspect 2
Aspect 3
Aspect 4
Physical reality
Aspect 1
Aspect 2
Aspect 4
Aspect 1
Aspect 2
Aspect 4
RQ 1RQ 2RQ 3
Figure 1: Relation between research questions (own illustration)
A graphical overview of the structure and relation of the RQs, is given in gure 1. In the
rst step, answering RQ 1, the abilities of VR to present different aspects of a specic
scenario is analyzed. Closely linked to this, in a second step answering RQ 2 focuses on
D1 New technologies in warehousing 151
identifying the losses that occur when transferring physical reality content to VR. With the
analysis of the subsequent re-transfer of VR content to physical reality RQ 3 is answered.
The remainder of this paper is structured as follows. After introducing the conceptual
framework used to organize the analysis of sensory delity, VR is described. Subsequent
the research propositions (P) are derived and tested by means of an empirical study. The
study’s results are evaluated in regards to the research propositions. Consequently, the
mouldability and transferability of VR experiences into a physical reality logistics context
is assessed and thus the research questions are answered.
2. Conceptual Framework
In order to structure the following analysis of a VR model’s sensory delity, a framework
to comprehensively describe a VR is developed. A main part of this framework is based
on the approach by Blümel (2011). When contrasting VR and physical reality, he rst
differs between the categories content and learning actions. Both are then operationalized
into several factors and consequences for use of VR and physical reality in learning are
derived for each of the factors. The category content includes the four factors complexity,
dynamics, networkedness and transparency, whereas the category learning actions
consists of the three factors reversibility, cost-dependence and time-dependence. As this
paper aims to focus not only on training but merely on usability of VR for planning and
on training in logistics context, the framework is extended using additional literature. The
categories interaction (Bowman et al. 2004) and user, with the corresponding factors
environment-dependence and presence (Kotranza, Lok 2008; Magnenat-Thalmann et al.
2004; Slater, Usoh, Steed 1994; Slater 2003), which are involved often when approaching
VR, represent a viable extension of the framework (e.g. Beckhaus, Lindeman 2011;
Slater 2010). Additionally, the category content is extended by the factors adaption and
control. This decision was made as the review of literature showed the high importance
of VR’s characteristic to be highly adaptable (Kaiser, Schatsky 2017; Bowman, McMahan
2007; Dombrowski, Vollrath 2008) while presenting extraordinary controllability at the
same time (Blümel 2011; Bowman, McMahan 2007). Furthermore a renaming of the
category learning actions into technology seems reasonable. Under the viewpoint that
this category holds the integrative role between the user and the content and hence
has the task to transfer certain contents the ve human senses are added as factors to
this category, namely haptic (Iwata et al. 2003, 2004; Koizumi et al. 2011; Lindeman,
Beckhaus 2009; Moon, Kim 2004), visual (Beckhaus, Lindeman 2011), audio (Wenzel
1992), olfactory (Haselhoff, Beckhaus 2006; Yamada et al. 2006; Yanagida et al. 2004)
and gustatory (Narumi et al. 2011; Ranasinghe et al. 2011). The main factors are then
content, technology, user and interaction as can be seen in gure 2. These factors are
used in the framework as they aggregate the commonly used characteristics of VR and
allow for a structured approach in describing a VR.
D1 New technologies in warehousing 152
Interact ion
Environment-
dependence
User
Pr esence
Content
Complexity
Tran sparency
Dynamics
Adaption
Technology
Time-depen dence
Cost -dependence
Reversibility
Visual
Haptic
SensoryFidelity
Control
Networked-
ness
Olfactory
Gust atory
Audio
Figure 2: Factors describing a VR (own illustration)
The overall impact of content as a factor in VR simulations is highlighted by Beckhaus,
Lindeman (2011), emphasizing the “technical” characteristic of content, and Whitton
(2003). The factor of technology is closely linked to the content as it plays a transferring
role to enable the user to interact with the simulated reality. It is therefore often mentioned
together with content but shall be considered separately in this framework to underline
its role as intermediary. As presence describes the response of the individual user it
is consequently not suitable to describe and compare the abilities of VR in contrast to
physical reality objectively (Bowman, McMahan 2007). This means that interaction, which
is strongly affected by the user, also is limited in regards to objective assessment. Since
the primary aim for now is an objective assessment of VR via the evaluation of sensory
delity, the categories user and interaction are looked at for the sake of completeness
while the focus lays on the factors content and technology.
When describing a physical reality, the rst category is the content, with the related
factors complexity, dynamics, adaption, transparency, control and networkedness.
Complexity describes the possibility of regulating the scope of stimuli placed on the user.
VR can contain highly complex situations but never up to the same degree of complexity
as a physical reality (Bowman, McMahan 2007; Peger, Wehking 2017). This can be
advantageous when VR is used for training of intricate tasks where successive increase
of complexity is appropriate. By dynamics the inuence of time progression within the
VR is labeled (Blümel 2011). In planning it might be benecial to slow down or speed
up the simulation in order to draw conclusions from the observed development either
in detail or on a superordinate level. The next factor is adaption. It relates to the effort
needed to adjust the reality to one’s needs. In VR the possibilities for adaptive measures
is relatively extended compared to physical reality. Furthermore, the expenditure for
adaptions is quite low as only a change in the virtual model is required (Kaiser, Schatsky
2017; Bowman, McMahan 2007). When taking the layout planning of a warehouse as
an example it will become clear that changing the position of all shelves physically is a
D1 New technologies in warehousing 153
far greater effort than doing this virtually. This increase in exibility and reduced effort
comes in form of cost as well as time saving (Dombrowski, Vollrath 2008). Furthermore,
VR is able to create a scenario that cannot be recreated in physical reality like weather
phenomena. A further factor belonging to the content of a scenario is the transparency. It
characterizes the option to enhance accessibility for example by using explosion views of
mechanical components or visualizing schematic process ows in the VR (Blümel 2011).
Due to the mouldability of a VR, the transparency can be increased or decreased at will.
Thus, understanding of a scenario can be enhanced signicantly (Blümel 2011). The
next factor is control. While inuence in physical reality is limited in many ways, a VR
offers great possibilities to exercise control (Blümel 2011; Bowman, McMahan 2007). An
example could be the simulation of a bursting dam. By control of the scenarios dynamics
engineers are able to understand the dynamics inside the construction and learn from it
(Zhu et al. 2016). Last factor to be discussed is networkedness. Due to information and
communication networks a linkage between several sites can be established to let users
interact with each other or work together in the VR as needed for a given purpose. This
is called networkedness. In case of networkedness, a scenario can be shared among
several users in a VR so that a joint experience results (Blümel 2011; Blümel 2013;
Peger, Wehking 2017). The application examples include meetings as well as training
of cooperative tasks. One of the restrictions that come with VR meetings is the lack of
options to visualize facial expression and body language. This complicates chairing such
a meeting (Hammerschmid 2017; Kaiser, Schatsky 2017).
The second category is given by the technology, transferring the content. It can be
differentiated into the factors reversibility, cost-dependence, time-dependence, haptic,
visual, audio, olfactory and gustatory. Under this category the factor reversibility is of
major importance. It describes the possibility to always undo actions carried out in VR
without serious consequences (Blümel 2011). Therefor it is benecial for the assessment
of VR versus physical reality. The cost-dependence is an ambivalent domain as the
advantageousness of VR in contrast to a physical reality is case dependent. This is
because for example in training with VR the development costs for the model are rather
high but using the model is almost free of charge. Hence, conditional on the number of
users, complexity for simulation of a scenario in VR and resulting cost for hardware, one
of the realities offers cost advantages. In general, costs for scenarios used repeatedly
and/or representing complex situations are lower in VR over time. In contrast, modelling
of a less complex scenario in VR might be unnecessary as setup would take more time
and costs would be higher than using physical reality (Peger, Wehking 2017, Blümel
2011; Bowman, McMahan 2007; Hammerschmid 2017). Time-dependence, in contrast,
is hardly existing in VR as the technology is at the user’s disposal at every time, thereby
increasing exibility and availability (Blümel 2011). As the experience of a scenario in VR
is only dependent on the availability of the technical equipment, the usage is independent
of time. Furthermore, virtual scenarios can be repeated without limitation (Blümel
2011; Chan et al. 2011; Bowman, McMahan 2007; Chan et al. 2011). The next factor is
D1 New technologies in warehousing 154
haptic. As the interaction with the virtual environment is realized via controllers, there
is only very limited possibility to feel for example the surface of an item. Similar holds
for the perception of forces. As VR is characterized by being only virtually perceivable,
there is no primarily impact of physical forces, which can lead to great differences in a
scenarios outcome and is seen especially critical in training setups (b-reel 2016; Lopes
et al. 2017). The visual factor includes all impressions the user perceives with his eyes.
Besides the pure displaying of a scenario further technical performance indicators have
to be taken into account. For example the eld of view, is limited in VR and can inuence
the user’s situation awareness as well as rendering or stereoscopy. The implementation
of auditory content can be realized in VR in the same way as in physical reality via
speakers or headphones. When focus is laid on presentation of natural soundscape one
might at rst glance asses VR less capable than physical reality. In fact the effort to
create the audio input for a scenario articially is the same in both realities as audio
content is independent of VR technology. The factor of olfactory input concerns perception
of smells. By implementation of air cannons aiming at the user in VR and shooting
scented air (Yanagida et al. 2004) or wearing mobile olfactory displays (Yamada et
al. 2006; Haselhoff,Beckhaus 2016) virtually strolling through an oriental market and
smelling the exotic scent of the spices becomes possible. Nonetheless, the effort is not
negligible. Creation of articial gustatory impressions are also under research. Examples
are a substitution with olfactory-visual input (Narumi et al. 2011), tongue stimulation
via electric pulses and temperature (Ranasinghe et al. 2012) or auditory substitution
(Koizumi et al. 2011). However, as gustatory sensation composes of many aspects as
structure, smell and taste, it is difcult to simulate virtually.
The third category of the framework is the interaction, which includes the factor
environment-dependence and plays an integrative role between technology and user. It
has two points to be mentioned. The rst one is the physical relation to the surrounding
area that can never be cut off. We can experience very dangerous scenarios in VR without
the slightest risk for physical welfare but there is also danger coming with this. The threat
is arising from the combination of acting in and perceiving the VR primarily while being
physically bound to the physical reality whose perception is only limited. The consequence
is often not less than an entanglement in the cable and a bump (Peger, Wehking 2017).
The second point is the emotional safety coming with VR. This is especially valuable
for training of scenarios that put a high pressure on the user as for example in phobia
therapy (Bowman, McMahan 2007; Kaiser, Schatsky 2017).
The last category, user, in this context is described by the factor of presence. As said
before, this domain cannot be measured and compared, as it is a subjective assessment
of the scenario. Bowman and McMahan (2017) state that these assessments can differ
between users of the same VR as well as among one user at different sessions in a VR.
Nonetheless, presence does have strong inuence on the performance of a VR. When
dening the level of presence in a physical reality as 100%, the level can only be below
100% for experiencing a VR, as one can hardly imagine to feel ‘more present’ in VR than
D1 New technologies in warehousing 155
ever in physical reality (Bowman, McMahan 2007; Peger, Wehking 2017; Whittinghill
2015).
Although the paper at hand focuses only on the factors representing the sensory delity a
framework was established with the purpose to create a basis for comprehensive analysis
and description of a VR including all relevant factors. Due to this the categories of user
as well as interaction were introduced and the categories of content and technology were
extended according to the current state of research.
3. Virtual Reality in Logistics Planning and Training
The use cases for VR HMDs are numerous and include simulation of operational processes
as well as creation of training environments, marketing and sales issues or design and
analysis of layouts. Research in VR is especially strong concerning product development
(Blümel 2013; Ottosson 2002) or factory and process planning (Peger, Wehking 2017;
Pappas et al. 2006). Also applications for diverse specic training purposes is highly
popular. Examples go from maintenance of power lines (García et al. 2016), the often
mentioned example of medicine (e.g. Izard et al. 2018), human-robot collaboration
(Matsas, Vosniakos 2017) and machining tasks (Nathanael et al. 2016) to karate training
(Zhang et al. 2018). Vaughan et al. (2016) give an overview of current VR training
systems differentiating the ve sections of medicine, industry/ commercial, collaboration,
serious games and rehabilitation. In the following, the focus is on planning and training
purposes in the context of logistics.
With the help of VR, a situation can be presented in a highly immersive scenario (Bowman,
McMahan 2007). Thereby, the freedom of designing this environment is enormous,
allowing a level of abstractness or detailedness as needed for the issue at hands. By
using sensors and motion capturing technologies, not only the created environment can
be perceived but also active participation of the user while interacting with virtual objects
is realized (Chan et al. 2011). Especially in the context of warehouse management, a
cost-intense setup and limited availability of resources do not allow for intensive training
and collection of empirical data upfront to planning. To overcome the limitations VR can
be used (Reif, Walch 2008). At the same time, the degree of control lays far beyond
the one achieved in a physical environment (Pehlivanis et al. 2004). Consequently, VR
technology presents a promising opportunity for planning and training purposes, given
that restrictions are taken into account and specic requirements are met.
The underlying assumption for believing that outcomes of VR planning and training can
be transferred to physical reality is that the logistics system in VR is representing the
physical environment in an adequate way (Reif, Walch 2008). However, VR and physical
reality can and do differ so that there are limitations that cannot be overcome by
simulation. One of the most obvious counterexamples is the impossibility to model item
D1 New technologies in warehousing 156
weights as well as resulting forces (b-reel 2016). At the same time, there are scenarios
not realizable in physical reality at reasonable expense e.g. intense training of NASA’s
astronauts (Grush 2017). The existence of such differences should not be an obstacle
but kept in mind when using simulations in VR.
For a rst orientation three scientic papers applying VR for path navigation in nuclear
plants, planning of aneurysm surgeries and training of human-robot collaboration are
assessed by means of the developed framework. The selection is based on the idea to
analyze at least one paper from the eld of planning and one from the eld of training.
Moreover, only papers from the last two years were considered to guarantee comparable
level of technological development. Furthermore, the paper describing the research had
to include comprehensive description of the VR model and its capabilities in order to make
an analysis via the framework possible. Finally industrial applications were preferred to
applications for the end-customer market as for example karate training (Zhang et al.
2018).
The rst chosen paper by Chao et al. (2017) develops a model that combines path
navigation using Dijkstra’s algorithm in combination with a VR environment that presents
a nuclear facility. In this application complexity is very low as obstacles are modelled
only with basic geometric shapes. The factor of dynamics is only indirectly applicable via
the calculation of radiation dose with the help of simulated walking speed. For adaption
the sensory delity is rather high as there are several obstacle setups and scenarios
realized and evaluated. Transparency is given to a medium degree as the outcomes of
the simulation runs are visualized in map views and can be compared among the different
scenarios. The factor of control is assessed with maximum sensory delity due to the
unlimited range of inuence on the model at any time. In contrast to this networkedness
shows no sensory delity at all as it is not applicable in this case. The strength of the
model at hand can be seen when checking the reversibility. It is high as each scenario is
evaluated comprehensively while at the same time preventing actual radiation exposure
for humans, so no harmful consequences arise. For cost- as well as time-dependence
the assessment is rather high. Even, there is no statement on costs for this model,
an evaluation of paths in nuclear plant would not be possible in physical reality due to
harmfulness, which can be considered equal to extraordinary high costs. Furthermore,
the simulation of different paths through the nuclear plant can be done independent
from working hours as can every computational simulation once started. Haptic, audio,
olfactory and gustatory factors are not applicable for this model. The visual delity is
medium as the map view is rather abstract and not realistic. Thus, sensory delity is
quite low. Still, this does not mean, it is a bad model as becomes clear when looking
at environment-dependence and presence and thereby at the model’s goal. It is a non-
immersive scenario, which combines a planning algorithm with VR, so no user can feel
presence and consequently no environment-dependence can occur. The use of technical
data and parameters allow to objectively determine the minimum dose path through a
D1 New technologies in warehousing 157
nuclear plant. Consequently the more subjectively related categories of interaction and
user are negligible and the effort is minimal to not existing.
The second paper deals with planning of surgical procedures in VR (Kockro et al. 2016).
The complexity is rather high as several imaging methods are used to automatically build
the models on a patient-specic level. The different structures and tissues are visualized.
The factor of dynamics is not applicable in this case, whereas adaption is rather high due
to the patient-specicity. Transparency shows medium sensory delity as the visualization
only includes such tissues that are relevant for the surgery. Some goes for control as the
model is generated automatically based on the image recordings and can be adjusted
only up to a limited degree to avoid bias. Again, networkedness is not applicable. The
factor of reversibility is high due to the fact that the model allows planning and training
of a surgery to unlimited extent. For cost-dependence there is no statement in the paper
but the VR model is assessed to realize rather high advantages in this term as it can
improve surgical success and is more cost-effective than using rubber models. Also the
time-dependence is rather high due to the independence from production times of rubber
models and automated model generation. Haptic delity was slightly improved by using
controllers instead of a computer mouse in order to allow natural hand movements. Still,
no structures and forces are modelled, which leads to medium assessment. The visual
presentation is high as it is based on original images. The factors of audio, olfactory
and gustatory are not applicable. Similar to the rst example, environment-dependence
and presence of the user play a minor role as the focus of the model’s application is on
objective planning and not individual evaluation of a virtual environment.
The last paper by Matsas and Vosniakos (2017) is about the design of a system using VR
to train human-robot collaboration. In this case complexity is rather high as the model
consists of a complete shop oor environment with additional areas presented via video
and detailed presentation of the robot arm and workspace. A reduction of complexity is
always possible by reduction of visual and auditory input. The factor of dynamics is not
directly applicable but only through consideration of the robots reaction to the user’s
action as form of dynamic interaction. Adaption is not considered in this example although
it should be possible. Assessment of transparency is medium. This is due to the spherical
visualization of the robot’s workspace, which is beyond reality but enhances transparency.
The control is at maximum as all equipment parameters and its arrangement on the shop
oor can be changed at will. Networkedness, again, is not applicable. Contrary to this,
reversibility is high. In order to gain training effects, the task is completed repeatedly.
Consequences from collisions with the virtual robot are limited to the educational effect.
As training on a physical human-robot interaction workspace blocks this workspace for
value-adding processes, the training of such processes in VR is advantageous, which leads
to a rather high assessment of the cost-dependence factor. Time-dependence is medium
as coordination between trainee and coach is necessary only. The haptic, olfactory and
gustatory factors are not applicable. Visual as well as audio delity is rather high as 3D
models of existing robot arms or equipment and operating noise from different sources
D1 New technologies in warehousing 158
are implemented in the model. Due to the limited space requirements that comes along
with a manufacturing task as the one modelled in this example the factor environment-
dependence is assessed as rather high. All necessary movements can be realized without
limitation or fear to hit a wall. To enhance users’ presence an avatar is synchronized to
the users movements which allows for a rather high assessment.
A concluding overview of the considered models’ sensory delity for the different factors
of the framework can be seen in table 1. For some factors there is a high match between
the three models whereas other factors differ. This is due to the differences in the
respective goal and the approach to reach this goal. One can see, that there are factors
that are very hardly to be modelled in general. Some might be negligible for a specic
application. Thus, in the current paper an analysis of differences between physical reality
and VR model as well as conclusions for the usability of VR for planning and training in
the context of logistics are developed.
(Chao et al.
2017)
(Kockro et al.
2016)
(Matsas,
Vosniakos
2017)
Content
Complexity ◔ ◕ ◕
Dynamics ◔ ○ ◔
Adaption ◕ ◕ ○
Transparency ◑ ◑ ◑
Control ● ◑ ●
Networkedness ○ ○ ○
Technology
Reversibility ● ● ●
Cost-dependence ◕ ◕ ◕
Time-dependence ◕ ◕ ◑
Haptic ○ ◑ ○
Visual ◑ ◕ ◕
Audio ○ ○ ◕
Olfactory ○ ○ ○
Gustatory ○ ○ ○
Interaction Environment-dependence ○ ○ ◕
User Presence ○ ○ ◕
Table 1: Qualitative assessment of models’ sensory delity (from high (●) to low (○))
D1 New technologies in warehousing 159
Based on the structure of the developed framework and the relation between its different
components given by the four categories, dependencies between those categories result.
Those dependencies as well as the research questions lead to the following research
propositions:
P 1: The higher the level of sensory delity of VR within the scope of VR
technology, the higher the maximal possible overall sensory delity is.
P 2a: The higher the level of sensory delity of VR within the scope of VR content,
the fewer losses occur.
P 2b: The higher the level of sensory delity of VR within the scope of VR
technology, the fewer losses occur.
P 3: The fewer losses occur due to deviations in sensory delity, the better the
results for transferability are.
Implicitly it can be seen, that the level of sensory delity of technology limits the
possibilities to transfer content. By this the technology establishes a threshold above
which no sensory delity in content is passed to the next category of interaction but pulled
down to the technology’s level of sensory delity (P 1). Furthermore, higher sensory
delity in one category is assumed to lead to a reduction of losses in this category. This
became apparent for example during the analysis of the VR model developed by Chao
et al. (2017). The objects in the simulation were modeled rather rough and thus with
low complexity. Obviously, a cylinder serves quite well as approximation to a human
shape in this application but nonetheless there a severe losses induced by this accepted
reduction of sensory delity (P 2a and P 2b). Finally, the line of argument of this research
is reected by the assumption that resulting transferability of a VR simulation increases
with a decrease in losses (P 3).
4. Experimental Design
These propositions are assessed by means of experiments, which is derived from similar
studies described in the literature. See for example Janeh et al. (2010), Pereira et al.
(2016) or Reif, Walch (2008) for similar numbers of participants. Reif, Walch (2008) also
used a setup akin to the one chosen for the study at hands. The setup for this study
is a twofold training setup in manual order picking in form of multi-order picking with
pick-by-voice via the Wizard-of-Oz method. The experimental design was validated in
workshops with practitioners from the eld of warehouse management. According to the
feedback from the workshop participants, the experimental setup was adjusted in an
iterative process, leading to the setup used for the paper at hand. Furthermore, pretests
were conducted in order to identify the appropriate number and coverage of experiments.
Both measures helped to improve sensory delity of the model and closeness to reality.
D1 New technologies in warehousing 160
Relevant impact of the workshops determined the experimental process described in the
following to create a realistic stress level for the experiment’s participants. One setup is
realized in physical reality while the other one copies this setup as accurately as possible
in VR with the aim of maximizing the overall sensory delity. The 18 participants were
acquired among students in the university environment via lectures and eld trips. The
study took place in November 2017. The participants were asked to fulll four runs
consisting of 16 orders each. In each run, four different customer orders were fullled
with one to nine items to be picked per order. Half of the group serving as control group
was asked to do this in physical reality only, while the participants of the other group
change from VR to physical reality after two runs, simulating a transfer of trained content
from VR to physical reality. The proceeding of the experiments is illustrated in gure 3.
Physical RealityVirtualReality
Physical Reality
Run 1
(16picks)
Run 2
(16picks)
Run 1
(16picks)
Run 2
(16picks)
Run 3
(16picks)
Run 4
(16picks)
Run 3
(16picks)
Run 4
(16picks)
Figure 3: Experimental process (own illustration)
The setup consists of one cart with one bin for each of the four simultaneously fullled
orders and the rack to be picked from. There are 18 picking locations arranged in three
levels containing the items. Neutral card boxes represent the items to avoid effects that
might be caused by different sizes, colors and weight of the items. Figure 4 shows the
virtual model and the physical representation of the experiment.
Collected data included personal information like age, sex and previous experience with
order picking and VR as well as an evaluation of the perceived workload by application
of the NASA Task Load Index (Hart and Staveland 1988; Hart 2006). To measure
performance, the experiments were recorded and picking time, errors (picking location/
order/ amount of items) and handling (drop of items) was evaluated.
This setup is chosen to gain insight into possibilities for simulation of a physical reality
in VR with highest possible sensory delity. Furthermore, it allows comparison of the
participants’ performance in VR and physical reality in order to identify losses (e.g. longer
picking times, more picking errors, bad handling) occurring during transfer from VR to
physical reality. By matching the observed losses to the previously identied aspects of
reality, conclusions regarding the research propositions can be drawn. An analysis of the
transferability concludes the research.
D1 New technologies in warehousing 161
Figure 4: Experimental setup in VR and physical reality (own gures)
5. Results
As the aim of the research at hand is the examination of the relation between sensory
delity, losses and transferability between VR and physical reality, the model used in the
experiment had to be modelled with highest possible sensory delity. As sensory delity
is dened by the degree of equivalence between both realities, a high sensory delity can
also be reached when adjusting the physical reality. With this in mind, a model for the
aforementioned experiment was developed and shall be assessed in the same manner
as the three VR models mentioned before, also including occurring losses. Subsequently,
resulting transferability is assessed by means of the experiments.
5.1. Empirical Findings
The two categories whose realization in VR are the main drivers for overall sensory
delity are the content and the technology. As stated in previous paragraphs, each of
them can be divided into several factors which in turn allow a certain level of sensory
delity. The following section provides an analysis of each factor’s sensory delity given
in table 2 as well as resulting losses followed by a summary of the resulting sensory
delity per category.
To begin with the category of content, the factor of complexity is the rst in this category.
From gure 4 one can see, that the model as well as the surrounding is kept simple to
allow participants in the experiments to focus solely on the task of order picking. In the
physical reality this could only be replicated to a limited degree. Supporting actions were
the smooth illumination, elimination of interfering sources and the use of plain white
boxes. Due to the thoroughly chosen level of complexity, the stimuli for the participants
D1 New technologies in warehousing 162
were reduced successfully. Nonetheless, the model shows some difference to its physical
counterpart leading to a medium assessment of its sensory delity for this factor. Losses
occurring at this point are for example the visualization of the surrounding room. The
next factor is dynamics. As our example of order picking is not a highly dynamic process,
the only dynamic aspect is represented by gravitation. The model was implemented in
a way, that the boxes followed physical laws as falling to the ground or bumping against
each other. The resulting assessment of the model thus is rather high as practically no
loss exists. The factor of adaption relates to the adaptability of the model and physical
setup, respectively. In the model a change is easily realizable by changing settings in
the software. To reach this exibility a shelf was chosen that allows several heights of
the levels and is light weighted to be rearranged easily. Also the bins and boxes can be
changed arbitrarily. Losses exist not in the manner that VR cannot represent a physical
condition but due to the virtually unlimited adaptability of the VR the other way round.
Therefore, the resulting assessment is rather high. Transparency is a factor that does
not play an important role in the chosen scenario of order picking and in fact can hardly
be applied in the model at hand. Contrary to this, control is assessed with high sensory
delity. The reason for this is the experimental setup in a laboratory. By using a laboratory
surrounding, far more inuencing factors can be controlled to a better degree than in a
eld study. Thereby level of control in physical reality is enhanced to a similar degree
as in VR and losses are limited. Since there is no cooperation or interaction between
several participants in the experiments, the factor of networkedness is not applicable.
In the second category, technology, the rst factor is reversibility. For the model in VR
as well as in physical reality this is reached to full extent and without losses as every
action can be reversed without consequences for the participants or involved equipment.
Picking a wrong item does not lead to a faulty delivery and dropping boxes has no
signicant effect. The next factor describes the cost-dependence. A task like order picking
is characterized by it repetitiveness and thus VR can make use of its inherent superiority
for repeated use in contrast to physical reality, in which every run causes costs. The
experimental setup in physical reality consists of the racks, carts, bins and boxes which
make no notable investments necessary but enables the implementation of a training
environment. Thus, the operational processes in a real warehouse would not have to be
interrupted for training sessions and the equivalence between VR and physical reality
is very high as losses are negligible. As stated before, the time-dependence of a VR
model is technically not existing. Due to the application in an experiment a dependency
is introduced, as coordination of time slots between investigator and participants had to
be realized. This holds true for VR as well as for physical reality, leading to a rather high
assessment of this factor. The factor of haptic can only be modelled to limited extent as
losses are represented by the necessity to use controllers, which inhibit sensing of surface
structures for example. Furthermore the weight of items could not be modelled. In this
case we made use of the possibility to adjust physical reality to VR and chose weightless
card boxes as items to be picked from the shelf. Also the participants were asked to
D1 New technologies in warehousing 163
take only one box in each hand in physical reality to get closer to the VR, where each
controller can only stick to one box at a time. By doing so the degree of equivalence was
increased, losses limited and consequently sensory delity can be assessed as medium.
A very high assessment is given to the visual factor. The main strength of VR lays in
displaying information visually. By designing the shelves, bins, boxes and cart in VR based
on the visual representation of the objects in physical reality, equivalence between both
realities and thus sensory delity is very high as losses occur scarcely. Same goes for
the audio factor. By an exact alignment of the speakers in both realities and usage of the
same voice and syntax for the picking orders, the sensory delity could be increased to
the maximum, allowing reduction of losses to zero. As gustatory and olfactory input are
not included in the application both factors are not applicable.
The categories user and interaction mentioned in the framework do not have a direct
effect on sensory delity but nonetheless implicitly inuence the result of a VR experience
severely. Thus, they shall be reected upon, too. Within the category of interaction the
only factor is the environment-dependency. As the experiments were accompanied by
two investigators of which one ensured a tangle free process by managing the cable
of the HMD in VR, the dependency could be reduced and thereby adjusted to the
level achievable in physical reality. Consequently, the assessment is rather high. The
participants’ evaluation of presence was not queried explicitly. Due to the fact that most
of them got used to the VR very fast and literally immersed into it, the factor of presence
is assessed as medium. As an extension of the current research, an analysis of the NASA
Task Load Index is in progress and will be used to compare perceived workload in VR
and the physical setup. Thereby an extension of the state of knowledge is achievable,
that goes in the direction of the human factor rather than observing purely technical
properties of a VR.
In the category of content especially controllability is found to have great sensory
delity, followed by dynamics and adaption. Transparency and networkedness were not
applicable. This leads to a medium assessment of the content’s sensory delity for the
developed model in this research. It can be seen that strengths of VR under the category
of technology are mainly related to presentation of visual and audio content whereas
haptic input cannot be modelled. Reversibility, cost- and time-dependence show very high
sensory delity and are only opposed by the non-applicability of gustatory and olfactory
factor. Overall the category of technology can be assessed as rather high. Interaction as
well as presence are limited and thus only at a medium level.
D1 New technologies in warehousing 164
Content
Complexity ◑
Dynamics ◕
Adaption ◕
Transparency ○
Control ●
Networkedness ○
Technology
Reversibility ●
Cost-dependence ◕
Time-dependence◕
Haptic ◑
Visual ●
Audio ●
Olfactory ○
Gustatory ○
Interaction Environment-
dependence ◕
User Presence ◑
Table 2: Qualitative assessment of developed model’s sensory delity
(from high (●) to low (○))
Concluding, the category of technology shows best results when it comes to overall
sensory delity. But also the content can be represented with great equivalence between
VR and physical reality. Losses can be identied in particular in regards to forces and
feel, as they show lowest sensory delity or rather none at all. In case of the training
model the most common reason for losses are the impossibility to model weight and
resistance of items in the VR. In the experiment, this becomes visible for example
because fatigue could be observed only to a very limited extent as the participants
only moved the relatively low weight of the controllers. Furthermore, as due to the
technological restrictions only a visual representation of the shelf was possible in VR,
the participants were able to pick “through” the shelf and boxes presenting storage
bays. Another example is the aforementioned fact that only two items could be picked
at once in VR. For orders, asking for one or two items the sensory delity of technology
was rather high whereas orders asking for more than two items automatically lead to a
decrease in sensory delity and thus to losses.
D1 New technologies in warehousing 165
5.2. Discussion of the research propositions
Based on the previous insights on sensory delity and losses, and extended by the results
of the experiments, the research propositions can be discussed and nally be conrmed
or have to be rejected. For the review of the rst three propositions the aforementioned
assessment of the model via the developed framework is to be used. The last proposition
can be addressed with the experiments performed.
The rst proposition (P 1) presents the category of technology as the bottleneck limiting
overall sensory delity through its function as transferring unit between content and user.
This proposition can be conrmed on the basis of the experience gained while modelling
the VR for the experiments. One of the most obvious examples to prove this, is given
when looking at the factors dynamics within the category of content and the factor haptic
in the category of technology. As said before, the boxes obey physical laws and fall to
the ground which is in line with high sensory delity for dynamics. But the consequence
to be expected, namely a hurting foot, when the box hits it, cannot be observed. The
reason lays in the limited sensory delity of haptic perceivable information. Thereby
the technology pulls resulting sensory delity down and represents an upper bound for
overall sensory delity.
The second (P 2a) and third (P 2b) proposition tackle the occurrence of losses due to
limitations of sensory delity in the category of content and technology, respectively.
As can be seen from the analysis and assessment of the model a decrease in sensory
delity comes along with a loss as both strongly relate to each other and mutually dene
each other. This holds for the category of content (P 2a) as the examples of adaptability
and complexity show. Whereas adaptability presents the positive case with high sensory
delity and no losses, complexity outlines the negative case in which reduced sensory
delity causes losses. Due to this, P 2a and P 2b can be conrmed. In the category of
technology (P 2b) an example supporting the proposition is given by the factors visual and
audio, which reach maximum sensory delity and no losses can be observed. Contrary
to this the haptic factor cannot be modelled with high sensory delity resulting in severe
losses.
For evaluation of the fourth research proposition (P 3) it is necessary to take into account
the experiments’ results and their interpretation in terms of transferability. Based on a
thoroughly analysis and statistical evaluation of the participants’ picking performance, it
can be seen, that picking times in VR and physical environment don’t differ signicantly.
Furthermore it could be shown, that participants receiving training in VR were able to
transfer the experience from the training runs in VR to the physical reality leading to
shorter picking times when rst confronted with the physical setup compared to the rst
runs of the participants not trained in VR. Nonetheless, training in VR did not lead to the
same learning effects as a training in physical reality, which implies a slight inferiority of
VR due to lack of efcient transferability. This is in line with the last proposition, which
traces lack of transferability back to the losses caused by reduced sensory delity. The
D1 New technologies in warehousing 166
aforementioned extension of the research by means of an analysis of the NASA Task Load
Index should allow for a further investigation of the transferability as it broadens the
view from the objective measurement of picking times to the user oriented evaluation
of perceived workload.
Concluding, it is to say, that transferability is primarily given as long as the aim of a
training setup for example is to familiarize with a situation, understanding processes or
getting used to procedures. When it comes to training of assembly tasks requiring ne
motor skills, transferability will be very limited due to the lack of physical feedback in
VR. However, in a planning context the transferability of VR solutions into physical reality
is expected to work well as nearly no losses were identied.
6. Conclusion
In this paper, a framework for assessment of sensory delity of a VR simulation examining
four different categories has been developed and applied to several examples. Furthermore
a VR model for the training of manual order picking has been built and assessed by means
of the developed framework. Additionally, the model has been compared to a physical
representation of the training setup in order to allow further assessment of the models’
capabilities, namely transferability.
The results of the models’ sensory delity analysis show the strengths of VR mainly in
visual representation, control and reversibility. Presentation of information on a haptic,
olfactory or gustatory level is hardly possible. With the complete analysis given in section
6.1 the rst research question can be answered. Looking at the second research question,
the losses are closely related to the identied shortcomings of sensory delity for the
different factors describing a model. They are to be determined individually for each
model in the same manner as sensory delity. General ndings are strong interrelation
of sensory delity and losses. One step further, the losses’ impact on transferability of
VR planning and training outcomes is analyzed. Thereby research question three can
be answered in the following way: Losses in one factor of a VR inuence the resulting
transferability to physical reality in this factor negatively. But it has to be mentioned,
that for example for planning purposes VR has the character of an insurance, which
reduces the risk of planning errors. This purpose can only be fullled when those factors
relevant for a specic planning objective show high sensory delity. This means, that
low sensory delity as in the case of complexity in the model by Chao et al. (2017)
for planning of walking paths in a nuclear facility are acceptable for factors of minor
relevance. Connecting to this, an important statement from Matsas, Vosniakos (2017)
is, that realism can be neglected in favor of other goals pursued in a VR simulation.
This implies, that an assessment of a models sensory delity is only the rst step.
Subsequently, a weighting of the different factors based on the focus of the respective
model has to be considered. Thereby the rst step in assessment of a model’s suitability
D1 New technologies in warehousing 167
is done via evaluation of a model’s sensory delity. In the next step the t between
realized sensory delity and requirements based on application of the model has to be
checked. Consequently, one could argue that high sensory delity reduces losses and
thus enhances transferability, but low sensory delity and consequently high losses don’t
explicitly lead to low transferability as long as the factor showing low sensory delity is
not of major importance for the modelling task.
Practical implications of this research are directed towards an attentive use of VR in
order to avoid wrong conclusions as results of VR planning outcomes or inappropriate
use for training purposes. Besides this exposure of limitations, some advice can be given
to overcome individual limitations and increase sensory delity. Limitations of the study
and its results are for example the non-consideration of the inuence posed by the user’s
presence. Furthermore, in-depth research on the inuence of different VR technologies
besides HMDs as for example the so-called Cave could enhance the knowledge base. We
highly encourage empirical research to deepen and broaden the level of knowledge in the
eld of VR application for planning and training in logistics contexts. Especially further
research on the mentioned weighting of the factors in order to increase t between
required and supported level of sensory delity is of interest.
7. References
Beckhaus, S. & Lindeman, R. W. (2011): Experiential Fidelity: Leveraging the Mind to
Improve the VR Experience. Pages 39-49 in Virtual Realities. Springer, Vienna.
Beckhaus, S., & Lindeman, R. W. (2011): Experiential Fidelity: Leveraging the Mind to
Improve the VR Experience. In Virtual Realities (pp. 39-49). Springer, Vienna.
Blümel, E. (2011): Virtual Reality Based Technology Platforms for Development,
Testing and Training. Proceedings of Annual International Conference Virtual and
Augmented Reality in Education (VARE) 2011: 1-16.
Blümel, E. (2013): Global Challenges and Innovative Technologies Geared Toward
New Markets: Prospects for Virtual and Augmented Reality. Procedia Computer
Science, 25: 4-13.
Bowman, D., Kruijff, E., LaViola Jr, J. J. & Poupyrev, I. P. (2004): 3D User interfaces:
theory and practice, CourseSmart eTextbook. Addison-Wesley, Boston.
Bowman, D. A., McMahan, R. P. (2007): Virtual reality: how much immersion is
enough? Computer 40 (7): 36-43.
Burdea, G. C. (2000): Haptics issues in virtual environments. Computer Graphics
International 2000. Proceedings: 295-302.
D1 New technologies in warehousing 168
Buttussi, F., & Chittaro, L. (2018): Effects of different types of virtual reality display
on presence and learning in a safety training scenario. IEEE transactions on
visualization and computer graphics, 24 (2): 1063-1076.
Bystrom, K. E., Bareld, W. & Hendrix, C. (1999): A conceptual model of the sense
of presence in virtual environments. Presence: Teleoperators and Virtual
Environments 8 (2): 241-244.
Chan, J. C., Leung, H., Tang, J. K. & Komura, T. (2011): A virtual reality dance
training system using motion capture technology. IEEE Transactions on Learning
Technologies 4 (2): 187-195.
Culbertson, H., Delgado, J. J. L. & Kuchenbecker, K. J. (2014): One hundred data-
driven haptic texture models and open-source methods for rendering on 3D
objects. Haptics Symposium (HAPTICS) 2014: 319-325.
Cummings, J. J. Bailenson, J. N. & Fidler, M. J. (2012): How Immersive is Enough?
A Foundation for a Meta-analysis of the Effect of Immersive Technology on
Measured Presence. Proceedings of the International Society for Presence
Research Annual Conference.
Deutsch, M., Ebert, D., von der Gracht, H. & Lichtenau, P. (2016): Neue Dimensionen
der Realität. Executive Summary zur Studie der Potenziale von Virtual und
Augmented Reality in Unternehmen. KPMG Wirtschaftsprüfgesellschaft.
Dombrowski, U. & Vollrath, H. (2008): Einsatz von Virtual Reality in der Lagerplanung.
Ein Vorgehensmodell für den Einsatz von Virtual Reality zur Lageroptimierung.
PPS Management 13 (4): 25-28.
Dörner, R., Geiger, C., Oppermann, L. & Paelke, V. (2013): Interaktionen in Virtuellen
Welten. Pages 157-194 in Virtual und Augmented Reality (VR/AR) – Grundlagen
und Methoden der Virtuellen und Augmentierten Realität (R. Dörner, W. Broll, P.
Grimm & B. Jung, Eds.) Springer Vieweg, Berlin Heidelberg.
Elbert, R. & Müller, J. P. (in press): The Impact of item weight on travel times in
picker-to-parts order picking: An agent-based simulation approach. Proceedings
of the 2017 Winter Simulation Conference.
García, A. A., Bobadilla, I. G., Figueroa, G. A., Ramírez, M. P., & Román, J. M. (2016):
Virtual reality training system for maintenance and operation of high-voltage
overhead power lines. Virtual Reality, 20 (1): 27-40.
Goisbault, I., Kerrad, L. & Vignon Marès, M.-C. (2017): Der Duft aus der Waschküche.
Planung & Analyse 5 (2017): 50-51.
Grush, L. (2017): Walking through space in NASA’s virtual reality lab. The Verge.
Available at https://www.theverge.com/2017/8/22/16178138/nasa-virtual-
reality-lab-mars-rover-simulator, last access: 14.12.2017.
D1 New technologies in warehousing 169
Hart S. G., & Staveland L. E. (1988): Development of NASA-TLX (Task Load Index):
Results of empirical and theoretical research. In Hancock P. A., Meshkati N.
(Eds.), Human mental workload, pp. 139-183. North Holland Press, Amsterdam.
Hart, S.G. (2006). Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of
the Human Factors and Ergonomics Society Annual Meeting, 50(9), pp. 904–908.
Haselhoff, S. & Beckhaus, S. (2016): Benutzerindividuelle, tragbare Geruchsausgabe
in virtuellen Umgebungen. Pages 83-94 in Virtuelle und Erweiterte Realität, the
3rd Workshop of the GI working group VR/AR.
Iwata, H., Yano, H., Uemura, T. & Moriya, T. (2003). Food simulator. ICAT.
Iwata, H., Yano, H., Uemura, T., & Moriya, T. (2004): Food simulator: A haptic
interface for biting. Virtual Reality 2004. Proceedings: 51-57.
Izard, S. G., Juanes, J. A., Peñalvo, F. J. G., Estella, J. M. G., Ledesma, M. J. S.,
& Ruisoto, P. (2018): Virtual Reality as an Educational and Training Tool for
Medicine. Journal of medical systems, 42 (3): 50.
Janeh, O., Langehn, E., Steinicke, F., Bruder, G., Gulberti, A. & Poetter-Nerger, M.
(2010): Walking in Virtual Reality: Effects of Manipulated Visual Self-Motion on
Walking Biomechanics. ACM Transactions on Applied Perception, 2 (3): 1-15.
Jung, B. & Vitzthum, A. (2013): Virtuelle Welten. Pages 65-96 in Virtual und
Augmented Reality (VR/AR) – Grundlagen und Methoden der Virtuellen und
Augmentierten Realität (R. Dörner, W. Broll, P. Grimm & B. Jung, Eds.) Springer
Vieweg, Berlin Heidelberg.
Kammergruber, F. & Günthner, W. A. (2010): Einsatz von Virtual Reality zur
Logistikplanung. ZWF Zeitschrift für wirtschaftlichen Fabrikbetrieb 105 (12):
11191122.
Koizumi, N., Tanaka, H., Uema, Y. & Inami, M. (2011): Chewing jockey: augmented
food texture by using sound based on the cross-modal effect. Proceedings
of the 8th International Conference on Advances in Computer Entertainment
Technology: 21.
Kotranza, A. & Lok, B. (2008): Virtual human + tangible interface = mixed reality
human. An initial exploration with a virtual breast exam patient. Virtual Reality
Conference 2008 (VR’08): 99-106.
Lindeman, R. W. & Beckhaus, S. (2009): Crafting memorable VR experiences using
experiential delity. Proceedings of the 16th ACM Symposium on Virtual Reality
Software and Technology: 187-190.
D1 New technologies in warehousing 170
Lopes, P., You, S., Cheng, L. P., Marwecki, S. & Baudisch, P. (2017). Providing
haptics to walls & heavy objects in virtual reality by means of electrical muscle
stimulation. Proceedings of the 2017 CHI Conference on Human Factors in
Computing Systems: 1471-1482.
Magnenat-Thalmann, N., Thalmann, D., Papagiannakis, G., Glardon, P., Joslin, C. &
Kim, H. (2004): Real-Time Virtual Characters for VR/AR Applications. CGI 2004
(No. VRLAB-CONF-2007-016).
Matsas, E., & Vosniakos, G. C. (2017): Design of a virtual reality training system for
human–robot collaboration in manufacturing tasks. International Journal on
Interactive Design and Manufacturing (IJIDeM), 11 (2): 139-153.
Moon, T. & Kim, G. J. (2004): Design and evaluation of a wind display for virtual
reality. Proceedings of the ACM symposium on Virtual reality software and
technology: 122-128.
Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T. & Hirose, M. (2011): Augmented
reality avors: gustatory display based on edible marker and cross-modal
interaction. Proceedings of the SIGCHI conference on human factors in
computing systems: 93-102.
Nathanael, D., Mosialos, S., Vosniakos, G. C., & Tsagkas, V. (2016): Development and
evaluation of a virtual reality training system based on cognitive task analysis:
The case of CNC tool length offsetting. Human Factors and Ergonomics in
Manufacturing & Service Industries, 26 (1): 52-67.
Ottosson, S. (2002): Virtual reality in the product development process. Journal of
Engineering Design 13 (2): 159-172.
Pantelidis, V. S. (2009): Reasons to use virtual reality in education and training
courses and a model to determine when to use virtual reality. Themes in Science
and Technology Education 2: 59-70.
Pappas, M., Karabatsou, V., Mavrikios, D. & Chryssolouris, G. (2006): Development
of a web-based collaboration platform for manufacturing product and process
design evaluation using virtual reality techniques. International Journal of
Computer Integrated Manufacturing 19 (8): 805-814.
Pehlivanis, K., Papagianni, M. & Styliadis, A. (2004): Virtual Reality & Logistics.
Proceedings of the International Conference on Theory and Applications of
Mathematics and Informatics: 377-384.
Pereira, A., Lee, G. A., Almeida, E. & Billinghurst, M. (2016): A Study in Virtual
Navigation Cues for Forklift Operators. 2016 XVIII Symposium on Virtual and
Augmented Reality: 95-99.
D1 New technologies in warehousing 171
Pfeiffer, M., Dünte, T., Schneegass, S., Alt, F. & Rohs, M. (2015). Cruise control for
pedestrians: Controlling walking direction using electrical muscle stimulation.
Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing
Systems: 2505-2514.
Peger, D. & Wehking, K.-H. (2017): VR-basierte Planung logistischer Systeme:
Entwicklung von Einsatzszenarien und Inbetriebnahme einer Versuchsumgebung.
Logistics Journal: Proceedings 10 (2017).
Ranasinghe, N., Karunanayaka, K., Cheok, A. D., Fernando, O. N. N., Nii, H.
& Gopalakrishnakone, P. (2011): Digital taste and smell communication.
Proceedings of the 6th international conference on body area networks: 78-84.
Reif, R. & Walch, D. (2008): Augmented & Virtual Reality applications in the eld of
logistics. The Visual Computer 24 (11): 987-994.
Schenk, M., Straßburger, S. & Kissner, H. (2005): Combining virtual reality and
assembly simulation for production planning and worker qualication. Proc.
of International Conference on Changeable, Agile, Re-congurable and Virtual
Production.
Slater, M. (2003): A Note on Presence Terminology. Available at: http://www0.cs.ucl.
ac.uk/research/vr/Projects/Presencia/ConsortiumPublications/ucl_cs_papers/
presence-terminology.htm, last access: 12.03.2018.
Slater, M. (2009): Place illusion and plausibility can lead to realistic behaviour in
immersive virtual environments. Philosophical Transactions of the Royal Society
B: Biological Sciences 364 (1535): 3549-3557.
Slater, M., Usoh, M. & Steed, A. (1994): Depth of presence in virtual environments.
Presence: Teleoperators & Virtual Environments, 3 (2): 130-144.
Vaughan, N., Gabrys, B., & Dubey, V. N. (2016): An overview of self-adaptive
technologies within virtual reality training. Computer Science Review, 22: 65-87.
Wenzel, E. M. (1992): Localization in virtual acoustic displays. Presence: Teleoperators
& Virtual Environments 1 (1): 80-107.
Whittinghill, D. M., Ziegler, B. & Moore, J. (2015): Nasum Virtualis: A Simple
Technique for Reducing Simulator Sickness. Available at: http://www.purdue.
edu/newsroom/releases/2015/Q1/virtual-nose-may-reduce-simulator-sickness-
in-video-games.html, last access: 08.12.2017.
Whitton, M. C. (2003): Making virtual environments compelling. Communications of
the ACM 46 (7): 40-47.
Wulz, J. R. (2008): Menschintegrierte Simulation in der Logistik mit Hilfe der Virtuellen
Realität. Dissertation. Lehrstuhl für Fördertechnik Materialuss Logistik,
Technische Universität München, Munich.
D1 New technologies in warehousing 172
Yamada, T., Yokoyama, S., Tanikawa, T., Hirota, K. & Hirose, M. (2006): Wearable
olfactory display: Using odor in outdoor environment. Virtual Reality Conference
2006: 199-206.
Yanagida, Y., Kawato, S., Noma, H., Tomono, A. & Tesutani, N. (2004): Projection
based olfactory display with nose tracking. Virtual Reality 2004. Proceedings: 43-
50.
Zhang, L., Brunnett, G., Petri, K., Danneberg, M., Masik, S., Bandow, N., & Witte, K.
(2018): KaraKter: An Autonomously Interacting Karate Kumite Character for VR-
based Training and Research. Computers & Graphics.
Zhu, J., Zhang, H., Yang, X., Yin, L., Li, Y., Hu, Y., & Zhang, X. (2016): A collaborative
virtual geographic environment for emergency dam-break simulation and risk
analysis. Journal of spatial science 61 (1): 133-155.
D1 New technologies in warehousing 173
