Available via license: CC BY 4.0
Content may be subject to copyright.
AN INTERACTIVE LOGISTICS CENTRE INFORMATION INTEGRATION SYSTEM
USING VIRTUAL REALITY
S. Hong 1, B. Mao 1, *
College of Information Engineering, Collaborative Innovation Centre for Modern Grain Circulation and Safety, Jiangsu Key
Laboratory of Grain Big-data Mining, Nanjing University of Finance & Economics, Nanjing, 210023, China - hjmrezl@outlook.com,
maoboo@gmail.com
Commission III Urban Sensing and Mobility
KEY WORDS: Logistics Centre, Virtual Reality, Video Surveillance, Eye Tracking, Logistics Traceability, Semantically Labelled
Data
ABSTRACT:
The logistics industry plays a very important role in the operation of modern cities. Meanwhile, the development of logistics industry
has derived various problems that are urgent to be solved, such as the safety of logistics products. This paper combines the study of
logistics industry traceability and logistics centre environment safety supervision with virtual reality technology, creates an
interactive logistics centre information integration system. The proposed system utilizes the immerse characteristic of virtual reality,
to simulate the real logistics centre scene distinctly, which can make operation staff conduct safety supervision training at any time
without regional restrictions. On the one hand, a large number of sensor data can be used to simulate a variety of disaster emergency
situations. On the other hand, collecting personnel operation data, to analyse the improper operation, which can improve the training
efficiency greatly.
* Corresponding author
National Key Technologies Research and Development Program of
China under Grant 2015BAD18B02
1. INTRODUCTION
The rapid development of information technology in modern
society has accelerated the tendency of e-commerce to lead the
economic transformation and development (Sila, 2013; Bask et
al. 2012). Logistics is one of the most important factors in
realizing and sustainable development of e-commerce (Yu et al.
2016). It is playing an increasingly important role in modern
city operations. The development of the logistics industry
produces various information such as video monitoring,
environment sensing and accurate positioning to effectively
monitor, and configure logistics resources (Kong et al. 2017).
Therefore, more and more surveillance devices are equipped in
the logistics centre to monitor temperature, humidity, video
information, number of inventory and etc. in real time.
To integrate the various data for better understanding, this paper
combines with virtual reality technology which can simulate
real scene very vividly, create a VR based system to visualize
and interact the monitoring information from the logistics centre.
In this proposed system, environment information (temperature,
humidity, CO2 and others) is visualized in the VR platform by
volume rendering with visual variable mapping algorithms. The
video surveillance data is analysed using object detection, then
the detected objects such as people, vehicles or goods are
modelled and visualized in the VR environment. Furthermore,
when a user checks the logistics centre data with our VR
platform, we record all the interaction of the user with eye
tracking equipment.
The rest of this paper is structured as follows: in Section 2, we
described some related works of our study. In Section 3, we
interpreted our system framework and give user’s data
acquisition, which is implemented in Section 4. Finally, Section
5 concludes and analyses the study.
2. RELATED WORK
2.1 Logistic Traceability
With the awakening of people's safety awareness, the demand
for product safety is increasing, and the transparency of product
information has become the mainstream of current market
development. The application of traceability technology in
product safety is essential. The traceability technology has been
applied successfully in various food safety supply chains and
has played an irreplaceable role (Xu et al. 2006). The system
proposed by this paper is based on the logistics traceability. In
logistics, traceability refers to the capability for tracing goods
along the distribution chain on a batch number or series number
basis. Traceability is an important aspect for example in the
automotive industry, where it makes recalls possible, or in the
food industry where it contributes to food safety (Li et al. 2007).
The international standards organization EPCglobal under GS1
has ratified the EPCglobal Network standards (especially the
EPC Information Services EPCIS standard) which codify the
syntax and semantics for supply chain events and the secure
method for selectively sharing supply chain events with trading
partners (Qiao et al. 2007). These standards for traceability have
been used in successful deployments in many industries and
there is now a wide range of products that are certified as being
compatible with these standards.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018
ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China
This contribution has been peer-reviewed.
https://doi.org/10.5194/isprs-archives-XLII-3-523-2018 | © Authors 2018. CC BY 4.0 License.
523
2.2 Virtual Reality
Virtual reality, is a high-level computer interface based on
immersion, interactivity and conception, the comprehensive
utilization of computer graphics, simulation technology,
multimedia technology, artificial intelligence technology,
computer network technology, parallel processing technology
and sensor technology, simulating human visual, auditory,
tactile and other sensory organ functions, that can make people
immersed in a virtual scene generated by computer, interact
with it in real time by language, gestures and other natural ways,
which creates a human-oriented multi-dimensional information
space.
Virtual reality technology has virtuality beyond reality, and the
core equipment of the system still is the computer. Image
display devices are the key peripherals used to create the
stereoscopic visual effect, commonly used are 3D projectors
and helmet displays etc. Currently, frequently used helmet
displays are Oculus Rift, HTC Vive, and SONY PlayStation VR.
The above three virtual reality helmets are integrated head
tracking, position tracking system, there is little difference in
hardware with only minor differences between them. Oculus
Rift and HTC Vive are equipped with a monocular resolution of
1080 * 1200 pixels OLED display, the visual angle of 110
degrees, the refresh rate of 90Hz, but the Oculus Rift screen
ratio of 16: 9, HTC Vive is 9: 5. Relatively speaking, the Sony
PlayStation VR has a slightly lower screen resolution of 960 *
1080 pixels and visual angle of 100 degrees, but the refresh rate
reaches up to 120Hz. This paper uses the HTC Vive helmet.
Virtual environment represents a controllable alternative to field
studies conducted in the real world. (Bertrand et al. 2013)
conducted a path-finding study based on the virtual reality
system. (Helmut et al. 2016) combined the virtual reality
environment with a mobile eye tracking device, proposed a
novel navigation evaluation system. There is no research on the
grain logistics centre in the virtual reality environment at
present. In this study, we combine VR helmet with eye tracking
module in virtual reality environment to realize an interactive
logistics centre information integration system, which greatly
reduces the research cost and enhances the significance of
research on tracing the source of logistics in VR.
3. SYSTEM FRAMEWORK
We implement a basic test platform for a grain logistics centre,
as shown in Figure 1.
Figure 1. VR Based Logistic Centre Information Integration
System
3.1 Framework Interpretation
From above system framework of this paper, we can conclude
that the whole study mainly divided into three parts. Firstly, we
simulate to acquire the grain logistics centre various sensors
data in the scene, such as virtual camera photography,
surveillance videos, the carbon dioxide concentration sensor,
oxygen concentration sensor, air temperature and humidity
sensor. Meanwhile, user’s behaviour data such as User training
data, User operating data can also be obtained by HTC Vive,
and Eye Tracking Module. The data obtained above will be
stored in a relational database. At last, we use deep learning
method to analyse the data, by reading the data from relational
database, then use deep learning algorithm to train the data and
build two predict models, one of the models can be used to
predict user’s future action, it’s convenient for executives
customizing the training program for users, another model is
used to find the grain logistics centre abnormal monitoring.
3.2 Data Collection
There are many ways to get points in 3D world coordinates
system. (Munn et al. 2008) proposed a method where the 3D
Point of Regard (POR) is estimated from a portable monocular
video-based eye tracker applying computer vision techniques to
get 3D structure and motion from video sequences. (Pirri et al.
2011) proposed a model-based approach for 3D gaze estimation
for wearable multicamera devices. The methods described
above are relatively complicated. In this paper, we get user’s
gaze points by the collision points between Steam VR ray and
these observed objects, this method is simple and can guarantee
the success rate of data acquisition. The obtained data is written
to disk through the C# file stream, as shown in Figure 2.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018
ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China
This contribution has been peer-reviewed.
https://doi.org/10.5194/isprs-archives-XLII-3-523-2018 | © Authors 2018. CC BY 4.0 License.
524
Figure 2. Using C# stream to write users’ data
4. IMPLEMENTATION OF SYSTEM
To implement this paper proposed system, in the primary step,
this demo system introduces real logistics centre scenes (Figure
3.) into OBJ format model according to the proportion of 1:1,
and import it into Unity 3D software, then combines with Steam
VR technology to display the logistics centre scene in the HTC
Vive virtual reality helmet, as shown in Figure 4.
Figure 3. Logistics Centre Scene
Figure 4. HTC Vive Virtual Reality Helmet
Then, using the Steam VR 1.0 tracking technology and HTC
Vive handles to realize panoramic roaming (Zhang et al. 2016).
In the scene, we can use the Unity 3D camera to render and
simulate the real monitoring function to collect the monitoring
data according to the actual situation of the logistics centre
(Figure 5.). Meanwhile, using the HTC Vive handle interactive
function to simulate the outburst safety misadventure such as a
fire accident by reducing the previous safety incidents related to
logistics warehousing, thereby, can realize the education and
training of the logistics centre managers for safety accidents.
This system also can record the personnel's operating data
during the training process. Thus, we can find the monitoring
anomaly by analysing the acquired user data through a deep
learning algorithm, to realize the function of user behaviour
analysis, which has a practical significance for the training of
warehousing managers in the logistics centre.
Figure 5. Roaming in Logistics Centre
5. CONCLUSION AND ANALYSIS
The proposed VR based framework of this paper has some
innovative significance, using the immersion of virtual reality,
users can immerse themselves in the real logistics centre scene
without leaving the house, and can repeat the training of
logistics centre safety supervision, and the importance of
framework is that not only surveillance data is integrated and
visualized for better understanding, but also the user
interactivities are recorded which is essential to generate the
semantically labelled data automatically. The semantic labelled
information could be further used to train the data mining
models such as deep learning network for automatic logistic
centre semantic monitoring and alerting system (Rimienė et al.
2007). For example, abnormal detection is important to the
safety of logistic centre, but it is difficult to define what is
abnormal. With the help of the proposed system, we can
monitor the interaction between user and surveillance data from
that the objective opinions to a phenomenon in a logistic centre
from the users are recorded. These objective opinions including
watching time, the way of checking, zoom in/out and etc. will
be further classified into normal or abnormal reactions which
build a bridge between surveillance data and its semantic
meaning automatically.
ACKNOWLEDGEMENTS
This paper is supported by National Key Technologies Research
and Development Program of China under Grant
2015BAD18B02.
REFERENCES
Sila, I., 2013. Factors affecting the adoption of B2B E-
commerce technologies. Electronic Commerce Research, 13(2),
pp. 199-236.
Bask, A., Lipponen, M., and Tinnilä, M., 2012. E-commerce
logistics: a literature research review and topics for future
research. International Journal of E-Services and Mobile
Applications, 4(3), pp. 1-22.
Yu, Y., Wang, X., Zhong, R.Y., and Huang, G.Q., 2016. E-
commerce Logistics in Supply Chain Management: Practice
Perspective. Procedia Cirp, 52, pp. 179-185.
Kong, X.T.R., and Luo, H., 2017. Reconfigurable express
logistics centre: a simulation study. IFAC-Papers Online, 50(1),
pp. 11770-11775.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018
ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China
This contribution has been peer-reviewed.
https://doi.org/10.5194/isprs-archives-XLII-3-523-2018 | © Authors 2018. CC BY 4.0 License.
525
Xu, S.W., Li, Z.Q., and Lim, Z., 2006. Study on China Food
Safety Information Sharing and Public Management System.
China Agriculture Press, Beijing. pp. 31-34.
Li, G.M., Huang, L.P., and Zhan, J.C., 2007. RFID Application
in Food Safety Traceability. In: Logistics and Material Handing,
Vol. 12, pp. 85-87.
Qiao, J., Han, Y., Li, and D. L., 2007. Importance and
Restricting Factor Analysis on Implementing Food Safety
Traceability in China. Guide to Chinese Poultry, 43(6), pp. 10-
12.
Bertrand, J., Ebrahimi, E., Wachter, A., Luo, J., Babu, S. V.,
and Duchowski, A. T., 2013. Visual attention to wayfinding
aids in virtual environments. In: Proceedings of the 5th Joint
Virtual Reality Conference, pp. 9-16.
Helmut, S. F., Volker, S., and Stefan, S., 2016. Evaluation of
indoor guidance systems using eye tracking in an immersive
virtual environment. Spatial Cognition and Computation, 17(1-
2), pp. 163-183.
Munn, M. S., and Pelz, J. B., 2008. 3D point-of-regard, position
and head orientation from a portable monocular video-based
eye tracker. In: Proceedings of the Symposium on Eye Tracking
Research & Applications, pp. 181-188.
Pirri, F., Pizzoli, M., and Rudi, A., 2011. A general method for
the point of regard estimation in 3D space. In: IEEE Conference
on Computer Vision and Pattern Recognition. IEEE, Vol. 32,
pp. 921-928.
Zhang, K.F., Zhao, X., and Xie, Y.L., 2016. VR and AR develop
actual combat. China Machine Press, Beijing. pp. 239- 240.
Rimienė, K., and Grundey, D., 2007. Logistics Centre Concept
through Evolution and Definition, Engineering Economics, 31
(4), pp. 9.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018
ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China
This contribution has been peer-reviewed.
https://doi.org/10.5194/isprs-archives-XLII-3-523-2018 | © Authors 2018. CC BY 4.0 License.
526
