Immersive Visualizations Using Augmented Reality and Virtual Reality

Abstract

Immersive visualizations refer to a novel way of representing data and information through spatial computing technologies such as augmented and virtual reality. It aims to enhance the user perspective for enhanced insights from data and information to assist in the various processes including but not limited to decision making, enhanced learning, higher precision, and cost savings.

Various applications of AR and VR provide better visualizations over traditional methods resulting in better decision making, cost savings for organizations, enhanced experiences for consumers, and many more. Immersive products provide an edge over the competition for businesses and enhance brand value, better stakeholder communication, and higher client satisfaction. At times it also facilitates cost reduction in specific scenarios. Though AR and VR as technology are beneficial and advancing at a rapid pace, there are some short-term challenges in implementing immersive visualizations. To overcome these challenges and make immersive visualizations effective, solution providers need to design products and services which provide the appropriate level of presence, interactivity, and immersion based on application. The human-centered design needs to be adapted for more human senses as AR and VR can make use of visual, aural, and spatial dimensions.

Full text

I
Immersive Visualizations
Using Augmented Reality and
Virtual Reality
Madhusudan Rao
1
and Manoj Dawarwadikar
2
1
NTT Data Services Pvt. Ltd, Bangalore, India
2
SP Jain School of Global Management, Sydney,
Bangalore, India
Synonyms
Augmented reality;Data visualization;Immersive
technologies;Information visualization;Virtual
reality
Definition
Immersive visualizations refer to a novel way of
representing data and information through spatial
computing technologies such as augmented and
virtual reality. It aims to enhance the user perspec-
tive for enhanced insights from data and informa-
tion to assist in the various processes including but
not limited to decision making, enhanced learn-
ing, higher precision, and cost savings.
Introduction
Immersive technologies such as augmented real-
ity and virtual reality have a transformational
effect on how complex data and information is
visualized. This trend has led to several applica-
tions across industries to adopt immersive visual-
izations. The reality-virtuality continuum has
served as a reference framework for classifying
different immersive technologies. The continuum
was a continuous scale ranging between the
completely virtual and completely real environ-
ment with visualization and immersion as the
primary influencers (Milgram et al. 1995) (Fig. 1).
In this context, augmented reality, typically
called as AR, is a technology that superimposes
digital content on a user’s view of the real physical
world. It provides a composite view through a
device such as a smartphone, a tablet, or a smart-
glass. The AR has core capabilities of information
visualization, guiding the user and making the
environment interactive through mediums such
as gestures and voice (Porter and Heppelmann
2017). Latest developments in smartphone tech-
nologies such as sensor-based “simultaneous
localization and mapping”(SLAM), 3D render-
ing, and camera capabilities have made aug-
mented reality accessible to most consumers
with better applications (Pangilinan et al. 2019).
Virtual reality, typically called as VR, is a
computerized simulation of a new environment
with visuals and interactions through a head-
mounted device and controls. Vision, audio, and
haptic feedback create experiences that feel like
real. Virtual reality replaces physical reality with
computer-generated content and adds the core
capability of simulation in addition to AR
© Springer Nature Switzerland AG 2020
N. Lee (ed.), Encyclopedia of Computer Graphics and Games,
https://doi.org/10.1007/978-3-319-08234-9_395-1

(Porter and Heppelmann 2017). VR spectrum pro-
vides several usage patterns based on immersion
and interactivity. Web browser-based non-
immersive, noninteractive simple applications
such as consumption of 360-degree photos and
videos are at the lower end of the spectrum. At the
same time, head-mounted device (HMD) based
experiences with controllers and 6° of freedom
provide complete immersion and real-time inter-
active experiences. The hardware requirements
for these experiences vary based on their com-
plexity and application (Pangilinan et al. 2019).
Mixed reality, typically known as MR, uses a
combination of augmented and virtual reality to
create engaging experiences. The extended real-
ity, typically known as XR, is an umbrella term
that covers augmented, virtual, and mixed reality.
It also encompasses other supporting technologies
such as AI (Artificial Intelligence), 5G, IoT
(Internet of Things) to create engaging and inter-
active applications (Pangilinan et al. 2019).
AR and VR have wide-ranging applications
across industries. Depending on the need for inter-
activity, immersion, and visualization, the appli-
cations make use of AR or VR appropriately.
Enterprise sectors such as education, manufactur-
ing, healthcare, military, real estate have seen
early adoption of AR and VR for applications
such as simulated walkthroughs, remote surgeries,
combat training, employee training, and more
(Carmigniani et al. 2011; Sicaru et al. 2017). Con-
sumer segments such as retail, e-commerce, edu-
cation, gaming, entertainment, and tourism also
make use of AR and VR for applications such as
virtual try-on for products (Bonetti et al. 2018,
interactive books, interactive advertisements,
immersive videos, virtual tours of famous places
(Huang et al. 2016), and 3D interactive lessons
Kosa and Uysal 2019).
Since the advent of Internet and mobile
penetration, the amount of data and information
processed in every enterprise and consumer appli-
cation have grown exponentially. This situation
poses significant challenges for users to make use
of data and information for activities such as deci-
sion making, gaining insights, learning effec-
tively, and collaborating. One of the critical
elements that makes data accessible and usable
for users is the visualization of it. The primary
objective of data and information visualization is
to facilitate learning and decision-making of users
across the personal and organizational spectrum
(Schintler and McNeely 2020).
As the complexity and volume of the informa-
tion and data are increasing exponentially, the
traditional methods of 2D and 3D visualizations
on flat screens of computers or phones are not
sufficient enough. Hence, emerging technologies
such as AR and VR are being explored by
researchers and businesses.
Need for Immersive Visualization
There are various reasons data visualization is
increasingly becoming more critical in recent
times. With growing complexity and volume of
data, aggregation and creating insightful or
delightful experiences is a significant challenge.
The purpose of visualization varies based on
applications, and some of the objectives are facil-
itating higher learning, speedier decision making,
and keeping the user at the center of data and
insights. This process must seamlessly work for
general as well as professional users (Schintler
and McNeely 2020).
Decision-making process analyses data to
extract patterns and facilitate the discovery of
knowledge or insights. Reducing the complexity
of data to discover trends and anomalies lead to
accelerated and more accurate decision making.
Immersive Visualizations Using Augmented Reality and Virtual Reality, Fig. 1 Reality-virtuality continuum as a
function of immersion (Milgram et al. 1995)
2 Immersive Visualizations Using Augmented Reality and Virtual Reality

Visual data mining techniques use human visual
perception to gain insights from patterns.
Immersive technologies assist significantly in
this entire process (Moloney et al. 2018).
Scientific visualizations play an essential role
in facilitating learning and information sharing
among researchers, businesses, and government
organizations. Applications such as stocks analy-
sis or sports analytics are usually multi-
dimensional and hence difficult to visualize on
existing visualization technologies such as 2D or
3D. Entertainment and gaming applications
require greater interactivity to enjoy the overall
experience. Industrial applications, such as
healthcare training or enterprise training, require
a sense of physical presence to be more productive
(Sicat et al. 2018). Due to these several complex
situations, immersive visualizations become a
tool of preference for personal and enterprise use.
Augmented and virtual reality have been
extended from the original continuum to the
multidimensional realm. Now it covers a sense
of presence, level of interactivity, and the mecha-
nism to achieve these through the embodiment of
suitable devices (Flavián et al. 2019). In other
words, the primary characteristics of immersive
technologies such as fidelity with spatial, visual,
and aural senses of humans make it perfectly
suitable for complex data and information visual-
ization (Moloney et al. 2018).
Applications of AR and VR for Immersive
Visualizations
This section presents a summary of applications
from various industries which use immersive
visualizations through AR and VR to facilitate
decision making, learning, or experiences
effectively.
Retail
Virtual Try-On of clothing or accessories through
augmented reality is picking up among consumers
as well as retailers. An in-store augmented mirror
setup provides users with a digital catalogue to
try-on for clothing and accessories. The proposed
AR application gives an enhanced shopping
experience compared to the physical model lead-
ing to higher customer satisfaction due to the
speedier decision-making process for shopping.
As a result, physical retail stores can provide
greater choice to users similar to online methods
and augment it with a sense of a physical presence
(Bonetti et al. 2018).
A study was conducted that ingested data from
neuro-scientific tools such as tracking eye move-
ments, storing navigation, and selecting the
brands in a virtual supermarket, performing data
analysis, and thereby gaining insights about con-
sumer choices, customer experience, and shop-
ping behavior in a store. Qualitative analysis was
also used to compare the choices of consumers
and the subsequent outcomes and sales. The over-
all suggestions correlated high attention devoted
to a brand and slow eye movements (between
brands), to additional brand purchases within the
category. The outcome of the results meant that
that the additional brand choices drive the time
buyers spent on the first choice. Hence, less time
available for the first selection leads to additional
purchases within the product category and
increased sales (Bigne et al. 2015). This is a
good use-case of how a VR environment was
used to identify retail consumers’behaviors to
improve product placements and, hence, increase
sales.
Manufacturing/Production
The training and manufacturing process of assem-
bling desktop computers is enhanced with AR to
convert instructions manual into a step-by-step
visual guide. The proposed AR app is a simplified
process to follow for new employees without los-
ing the task’s context. The clarity of instructions
leads to reduced errors and lesser training costs
incurred by the organizations. Thus, productivity
is higher, and product quality is also enhanced
(Osborne and Mavers 2019).
By combining the manufacturing industry with
VR technology, enterprises can remain competi-
tive by utilizing quality inputs in new product
developments. In manufacturing enterprises, by
its visualization and immersive experience, VR
helps in aggregating relevant information and
enables faster decision-making during the product
Immersive Visualizations Using Augmented Reality and Virtual Reality 3

development processes. A thorough analysis
shows the increasing use of VR technologies and
further research being conducted to increase the
practical use of VR technologies (Choi
et al. 2015).
Education
A virtual tour of heritage locations provides
immersive history and cultural experiences to stu-
dents. In one of the examples, a 360-degree virtual
simulation is created at the historical center of the
city of Rethymno in Greece to teach ancient his-
tory and culture. A virtual reali ty application gives
learners a higher attention span due to a virtual
environment free of distractions. The additional
sense of physical presence helps to facilitate
greater understanding and engagement to the
learners. The level of interactivity fostered
through the VR application evokes higher interest
in learners than just consuming content over the
screen (Argyriou et al. 2020).
Real Estate
4D CAD models and immersive visualizations are
used in structural steel framing operations to
showcase the design and construction process to
all stakeholders for approvals and updates.
Through effective visualizations, construction
processes and operations are streamlined pro-
cesses. The outcome means the decision-making
process is effective due to lesser ambiguity
reduced errors and greater participation and
engagement of all stakeholders (Kamat
et al. 2010).
Healthcare
Patient care and training in the poststroke phase
are one of the healthcare use cases using
immersive technologies. The rehabilitation of
body-part functioning (such as hands) is enhanced
through AR assisted devices for common tasks
such as reaching out and grasping objects. The
approach involving AR helps give haptic feed-
back, and the interactivity helps in accelerating
the patients’rehabilitation process. The process is
highly scalable due to the re-use of the virtual
environment resulting in cost saving for patient
care. As AR systems are easy to set up, the manual
operations effort is reduced (Xun et al. 2005)
(Fig. 2).
The surgical workplace is transforming with
the help of emerging technologies like VR and
AR. VR applications play an important role in
healthcare by enhancing the medical use of data
such as anatomy, intraoperative surgery, or post-
operative rehabilitation. This process is acceler-
ated by an increase in the availability and speed of
VR hardware and a reduction in price (Khor et al.
2016) (Fig. 3).
Tourism
Tourist destination marketing is using virtual real-
ity to spread interesting information among cus-
tomers. Creating interactive and informative
virtual worlds to market tourist destinations is
attracting customers. This method of using
immersive visualizations provides higher con-
sumer engagement and more credible marketing
efforts resulting in less ambiguity for customers.
Hence, marketers receive a higher satisfaction rate
from their consumers as the immersive visualiza-
tion of destination gives a more realistic look and
feel of what to expect. The outcome is that the
customers make their decisions faster (Huang
et al. 2016; Huang and Liao 2015).
Augmented reality applications are enhancing
the overall tourism experience by providing more
context-specific information in a museum. The
tourists could use their smartphone camera and
augmented reality application to know interesting
information such as history and facts about each
article presented in a museum. This experience
significantly adds value to the tourist’s knowledge
in a very engaging way without losing the context
and language barriers in a foreign location (Yung
and Catheryn 2019).
Wellness
Virtual reality and interactive, immersive games
are being used to increase mindfulness. Its appli-
cations include therapy for pain management and
mental wellbeing.The VR approach offers con-
textual visualizations leading to lower mental
stress and perceived pain. The immersive
approach avoids distractions from the real-world
to assist in the healing process. This ultimately
4 Immersive Visualizations Using Augmented Reality and Virtual Reality

leads to higher satisfaction among patients and
speedier recovery (Kosa and Uysal 2018).
Journalism
Interactive prints and immersive storytelling are
revolutionizing the journalism industry.
Augmenting the print media formats with digital
content through AR and immersive storytelling
makes news more effective and engaging. The
immersive storytelling approach makes for more
effective storytelling (via interactive media). The
digital augmentation enhances the value of
existing media such as print to make it more
relevant for the users. It enables the users to get
an easier transition from the existing medium to a
new medium without losing context. Some of the
limitations in this approach are lack of hardware
availability, lack of tools for content creation, and
lack of awareness among consumers (Pavlik and
Bridges 2013).
Immersive storytelling is also experimented
through virtual reality by many leading broadcast-
ing and publishing companies such as The
New York Times to deliver a first-person account
of refugees during displacement due to crisis.
These short experiences put a consumer in the
middle of the action through 360-degree videos
and evoke a strong sense of reality and empathy.
The use of VR transforms the news from an
account of data and information to an experience
that heightened the learning as well as entertain-
ment of the consumer. One of the limitations in
this approach is shorter usage span of virtual real-
ity due to headset issues, hence short, and bite-
sized documentaries are popularly used for such
experiences (Sirkkunen and Uskali 2016).
Immersive Visualizations
Using Augmented
Reality and Virtual
Reality,
Fig. 2 Rehabilitation
process of the stroke
patients using AR apps.
(Xun et al. 2005)
Immersive Visualizations
Using Augmented
Reality and Virtual
Reality, Fig. 3 Surgeons
using Google Glass in
Operating theatres. (Khor
et al. 2016)
Immersive Visualizations Using Augmented Reality and Virtual Reality 5

Defense
Military training uses virtual reality to simulate
battleground and teach strategic tactics through
interactive training for the forces. The VR simu-
lations enhance both the navigation and coordina-
tion capabilities among soldiers. This method
provides a more accurate representation of war
scenarios and difficult situations than usual
mediums due to its immersive environment and
interactivity. The immersive simulations provide
scalability and flexibility in training content
(Livingston et al. 2011) (Fig. 4).
Augmented reality is used as a handy tool to
disseminate real-time context-aware information
in a military operation. This objective can be
achieved through a head-mounted AR glass or a
handheld device such as a smartphone. The usage
of AR also helps in getting remote assistance or
recreation of a specific environment, such as a
demolished structure. It helps understand minor
and vital details about the actual environment and
make rapid decisions in combat situations. Navi-
gation information is also delivered on an AR
headset to guide the personnel making the expe-
rience seamless (Dodevska et al. 2018).
Limitations
There are various technical as well as business
challenges in implementing immersive visualiza-
tions using AR and VR. VR experiences are lim-
ited by the quality of the headset, quality of
software applications, and interactivity issues.
Standalone wireless headset suffers due to pro-
cessing capability, weight, field-of-view, and bat-
tery life. High-end PC based VR headsets are
difficult to set up and expensive. If not designed
well, software applications can deliver a bad expe-
rience to users, including motion sickness. Inter-
activity is currently limited to the use of
controllers and hand gestures, which are still
evolving. Augmented reality experiences on AR
glasses are limited by the design, size, weight,
battery, and field of view of the headset.
Smartphone-based AR experiences are widely
used now. However, they do not provide rich,
immersive experiences and interactivity. They
suffer from issues such as dependency on a spe-
cific smartphone model, GPU capabilities of
smartphone and smartphone hardware such as
camera and display. Business limitations such as
awareness about the technology, cost of adopting
the technology, and consumer readiness are being
researched for AR and VR implementation. Lack
of maturity of the eco-system, such as availability
of suitable hardware, software, and content, is also
Immersive Visualizations Using Augmented Reality and Virtual Reality, Fig. 4 Military training simulation
example, using Virtual Reality
6 Immersive Visualizations Using Augmented Reality and Virtual Reality

a challenge (Chandra and Kumar 2018;
Leovaridis and Bahnă2017; Porter and
Heppelmann 2017).
Conclusion
Various applications of AR and VR provide better
visualizations over traditional methods resulting
in better decision making, cost savings for orga-
nizations, enhanced experiences for consumers,
and many more. Immersive products provide an
edge over the competition for businesses and
enhance brand value, better stakeholder commu-
nication, and higher client satisfaction. At times it
also facilitates cost reduction in specific scenarios.
Though AR and VR as technology are beneficial
and advancing at a rapid pace, there are some
short-term challenges in implementing immersive
visualizations. To overcome these challenges and
make immersive visualizations effective, solution
providers need to design products and services
which provide the appropriate level of presence,
interactivity, and immersion based on application.
The human-centered design needs to be adapted
for more human senses as AR and VR can make
use of visual, aural, and spatial dimensions.
Cross-References
▶Mindfulness, Virtual Reality, and Video Games
References
Argyriou, L., Economou, D., Bouki, V.: Design methodol-
ogy for 360° immersive video applications: the case
study of a cultural heritage virtual tour. Pers. Ubiquit.
Comput. (2020). https://doi.org/10.1007/s00779-020-
01373-8
Bigne, E., Llinares, C., Torrecilla Moreno, C.: Elapsed
time on first buying triggers brand choices within a
category: a virtual reality-based study. J. Bus. Res. 69,
(2015). https://doi.org/10.1016/j.jbusres.2015.10.119
Bonetti, F., Warnaby, G., Quinn, L.: Augmented reality and
virtual reality in physical and online retailing: a review,
synthesis and research agenda. In: Jung, T., Claudia,
M. (eds.) Augmented Reality and Virtual Reality,
pp. 119–132. Springer, Cham (2018)
Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P.,
Damiani, E., Ivkovic, M.: Augmented reality technol-
ogies, systems and applications. Multimed. Tools Appl.
51(1), 341–377 (2011)
Chandra, S., Kumar, K.N.: Exploring factors influencing
organizational adoption of augmented reality in
e-commerce: empirical analysis using technology--
organization-environment model. J. Electron. Commer.
Res. 19(3), (2018)
Choi, S., Jung, K., Do Noh, S.: Virtual reality applications
in manufacturing industries: past research, present find-
ings, and future directions. Concurr. Eng. 23, (2015).
https://doi.org/10.1177/1063293X14568814
Dodevska, Z., Mihic, M., Manasijevic, S.: The role of
augmented reality in defensive activities, (2018)
Flavián, C., Ibáñez-Sánchez, S., Orús, C.: The impact of
virtual, augmented and mixed reality technologies on
the customer experience. J. Bus. Res. 100, 547–560
(2019)
Huang, T.L., Liao, S.: A model of acceptance of
augmented-reality interactive technology: the moderat-
ing role of cognitive innovativeness. Electron.
Commer. Res. 15(2), 269–295 (2015)
Huang, Y.C., Backman, K.F., Backman, S.J., Chang, L.L.:
Exploring the implications of virtual reality technology
in tourism marketing: an integrated research frame-
work. Int. J. Tour. Res. 18(2), 116–128 (2016)
Kamat, V., Golparvar-Fard, M., Martinez, J., Peña-Mora,
F., Fischer, M., Savarese, S.: CEC: research in visual-
ization techniques for field construction. J Const Eng
Manag. 137,(2010).https://doi.org/10.1061/(ASCE)
CO.1943-7862.0000262
Khor, W., Baker, B., Amin, K., Chan, A., Patel, K., Wong,
J.: Augmented and virtual reality in surgery-the digital
surgical environment: applications, limitations and
legal pitfalls. Ann Translat Med. 4, 454–454 (2016).
https://doi.org/10.21037/atm.2016.12.23
Kosa, M., Uysal, A., et al. Acceptance of Virtual Reality
Games: A Multi-Theory Approach. International Jour-
nal of Gaming and Computer-Mediated Simulations
(IJGCMS), 12(1), 43–70 (2020). https://doi.org/
10.4018/IJGCMS.2020010103
Leovaridis, C., Bahnă, M.: Aspects regarding virtual real-
ity as innovation in creative industries. Rev Romana
Soc. 28, (2017)
Livingston, M., Rosenblum, L., Brown, D., Schmidt, G.,
Julier, S., Baillot, Y., Swan, J., Ai, Z., Maassel, P.:
Military Applications of Augmented Reality. Springer,
New York (2011). https://doi.org/10.1007/978-1-4614-
0064-6_31
Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Aug-
mented reality: a class of displays on the reality-
virtuality continuum. Telemanipulator Telepresence
Technologies. 2351, 282–292 (1995). https://doi.org/
10.1117/12.197321
Moloney, J., Spehar, B., Globa, A., et al.: The affordance of
virtual reality to enable the sensory representation of
multi-dimensional data for immersive analytics: from
Immersive Visualizations Using Augmented Reality and Virtual Reality 7

experience to insight. J Big Data. 5, 53 (2018). https://
doi.org/10.1186/s40537-018-0158-z
Osborne, M., Mavers, S.: Integrating augmented reality in
training and industrial applications. In: 2019 Eighth
International Conference on Educational Innovation
through Technology (EITT). IEEE, Biloxi (2019).
https://doi.org/10.1109/EITT.2019.00035
Pangilinan, E., Lukas, S., Mohan, V.: Creating Augmented
and Virtual Realities: Theory and Practice for Next-
Generation Spatial Computing. O’Reilly Media, Inc,
Sebastopol (2019)
Pavlik, J.V., Bridges, F.: The emergence of augmented
reality (AR) as a storytelling medium in journalism.
J Comm Monog. 15(1), 4–59 (2013)
Porter, M.E., Heppelmann, J.E.: Why every organization
needs an augmented reality strategy. Harv. Bus. Rev.
95(6), 46–57 (2017)
Schintler, A.L., McNeely, C.L.: Encyclopedia of Big Data.
Springer, Cham (2020)
Sicaru, I.A., Ciocianu, C.G., Boiangiu, C.A.: A survey on
augmented reality. J Inf Syst Oper Manag. 11(2),
(2017)
Sicat, R., Li, J., Choi, J., Cordeil, M., Jeong, W.K., Bach,
B., Pfister, H.: Dxr: a toolkit for building immersive
data visualizations. IEEE Trans. Vis. Comput. Graph.
25(1), 715–725 (2018)
Sirkkunen, E., Uskali, T.: Journalism in virtual reality:
opportunities and future research challenges. In: Pro-
ceedings of the 20th International Academic Mindtrek
Conference (2016)
Xun, L., Kline, T., Fischer, H.C., Stubblefield, K.A., Ken-
yon, R.V., Kamper, D.G.: Integration of augmented
reality and assistive devices for post-stroke hand open-
ing rehabilitation. In: 2005 IEEE Engineering in Med-
icine and Biology 27th Annual Conference, Shanghai
(2005)
Yung, R., Catheryn, K.-L.: New realities: a systematic
literature review on virtual reality and augmented real-
ity in tourism research. Curr. Issue Tour. 22(17),
2056–2081 (2019)
8 Immersive Visualizations Using Augmented Reality and Virtual Reality