Conference PaperPDF Available

Using augmented reality for visualizing complex graphs in three dimensions

Authors:

Abstract and Figures

In this paper we explore the effect of using augmented reality (AR) for three-dimensional graph link analysis. Two experiments were conducted. The first was designed to compare a tangible AR interface to a desktop-based interface. Different modes of viewing network graphs were presented using a variety of interfaces. The results of the first experiment show that a tangible AR interface is well suited to link analysis. The second experiment was designed to test the effect of stereographic viewing on graph comprehension. The results show that stereographic viewing has little effect on comprehension and performance. These experiments add support to the work of Ware and Frank, whose studies showed that depth and motion cues provide huge gains in spatial comprehension and accuracy in link analysis.
Content may be subject to copyright.
ISMAR 2003
Using Augmented Reality for Visualizing Complex Graphs in Three Dimensions
Daniel Belcher
1
, Mark Billinghurst
1, 2
, SE Hayes
1
, and Randy Stiles
3
1
Human Interface Technology Lab, University of Washington, Seattle
2
Human Interface Technology Lab (New Zealand), University of Canterbury, Christchurch, NZ
3
Lockheed Martin Corporation
{
1
danb,
2
grof}@hitl.washington.edu, beephree@u.washington.edu,
3
randy.stiles@lmco.com
Abstract
In this paper we explore the effect of using Augmented
Reality for three-dimensional graph link analysis. Two
experiments were conducted. The first was designed to
compare a tangible AR interface to a desktop-based
interface. Different modes of viewing network graphs
were presented using a variety of interfaces. The results
of the first experiment lend evidence to the view that a
tangible AR interface is well suited to link analysis. The
second experiment was designed to test the effect of
stereographic viewing on graph comprehension. The
results show that stereographic viewing has little effect on
comprehension and performance. These experiments add
support to the work of Ware and Frank, whose studies
showed that depth and motion cues provide huge gains in
spatial comprehension and accuracy in link analysis.
1. Introduction
In two studies, Colin Ware’s group showed that kinetic
depth cues – and to a limited extent stereo depth cues –
can greatly increase comprehension of complex graph
structures. An improvement of 300% in size of the
connected graph was observed, when compared to the
same graph displayed in 2D [9]. Subsequent work by
Ware and Rose [10] has shown that the mode of
manipulation is also a crucial element in the mapping
between the input device and virtual object. Interfaces
that use modes of manipulation that are closer to those
found in everyday physical interaction, and that have
simple spatial logic are what we call tangible interfaces.
They find that tangible interfaces provide increased
comprehension of graph structure.
Augmented Reality (AR) typically involves the overlay of
virtual imagery on the real world. In this paper we explore
how AR interfaces can be used to view complex linked
graph structures. We believe that AR techniques could be
beneficial for several reasons, including:
A large virtual display of graphical links in AR
Increased comprehension of complex link
analysis graphs
Spatial recall of link analysis graphs
For many applications, there is a need to understand
connectivity between linked nodes displayed as a graph.
This is often called link analysis. Ware and Frank showed
that the interface could play a crucial role in accurate and
rapid link analysis. Tangible interfaces provide a
platform upon which to build useful applications that have
improved graph comprehension, but the question remains:
does this effect transfer to Augmented Reality? We
hypothesize that the same increase in comprehension for
connected graphs occurs for Augmented Reality
information displays as for screen-based 3D interfaces.
The use of a tangible object for an input device may
provide an additional positive effect. Finally, we
hypothesize that an AR interface with a stereoscopic
display system will perform as well as a similar 3D on-
screen system using stereographic glasses.
To test these hypotheses we conducted two experiments.
In the first, we were concerned with comparing a 2D On-
screen viewing condition, to 3D on-screen and
Augmented Reality viewing conditions. Participants were
shown randomly generated graphs, at different levels of
complexity, and asked questions about the connectivity of
selected nodes. This study will help us determine the
suitability of AR for link analysis.
In our second experiment, we were concerned with the
effects of stereo on link analysis. Ware and Frank [9]
showed little positive effect for stereo when compared
with that of kinetic depth effects. Though we believe
facility of movement and a simplified interface will
provide the most gain, it is incumbent upon us to test
stereo effects in our own interface. Therefore, our second
experiment will compare On-screen interfaces, both
monoscopic and stereoscopic, to the corresponding mono-
and stereo-AR interfaces.
Although several groups have used AR interfaces for
scientific [3] and mathematical [4] visualization, there
have been no user studies conducted yet comparing
performance with these interfaces to screen-based
systems. If our research shows that AR interfaces
perform as well, or better than screen-based interfaces on
graph analysis tasks then this may have significant
implications for future AR visualisation interfaces.
1
ISMAR 2003
2. Experiment 1: Modes of Viewing
The question we were asking in this experiment was
“How different is an AR interface than a 2D – or 3D – on-
screen interface for path tracing in a graph?” In order to
get a quantitative evaluation, we have adopted (and
adapted) the task used in Ware and Franck [9]: path
tracing in an interconnected graph. The independent
variable was the viewing condition. The dependent
variables were percent error and response time.
Three viewing conditions (Figure 1) were employed:
1. 2D On-screen interface: The 3D graph was
shown on screen projected onto a 2D plane with
a black background. The viewer was allowed to
zoom toward and away from the projection plane
using the right mouse button. There was no
rotation of the model.
2. 3D On-screen interface: The 3D graph was
shown on acreen atop a white disk placed to
mimic the tracking disc in the AR condition.
The subject used the left-mouse button to spin
the viewpoint relative to the 3D graph in a
trackball-like fashion. The right mouse button
zoomed the viewpoint in and out.
3. Augmented Reality interface: The subject
manipulated a real disk while wearing a Head
Mounted Display with a small video camera
attached to it. The disk had a square marker on it
that was used by computer vision tracking
software [1] to track the users viewpoint and
overlay the graphic on the disc. The tracking
marker was then occluded with a smaller virtual
disc that covered the marker itself. When the
subject rotated or tilted the disc, the graph was
rotated or tilted accordingly.
We predict that the tangible AR interface will perform as
well as the 3D on-screen interface, both in terms of
accuracy (or minimal percent error) and response time.
2.1. Equipment
The hardware used in this study consisted of one Dell
Dimension 4500 Pentium 4 with a PNY Nvidia Quadro4
XGL Series graphics card. For the Augmented Reality
condition, an ELMO mini-camera was mounted on a
SONY Glasstron LDI-D100B Head Mounted Display
(SVGA 800x600, 30-degree field of view). A
Hauppauge! WinTV GO 190 video capture card was used
to capture the NTSC signal from the video camera. The
subjects manually manipulated a cardboard disc with a
tracking marker positioned in the center. Figure 2 shows
a user during the AR condition. In the two onscreen
conditions, a two-button mouse was used to manipulate
the orientation and position of the graph displayed on a
Dell Ultrascan SVGA monitor.
Figure 2: The experimental setup. The AR Condition
2.2. Participants
16 subjects took part in this experiment, ranging between
20 and 48 years old, with the average age being 25. The
female to male ratio was 5:11. Two of the subjects were
left-handed; one ambidextrous; and the remaining thirteen
were right handed.
1: 2D On-screen condition 2: 3D On-screen condition 3: AR condition
Figure 1: Experiment 1 Conditions correspond to interfaces.
2
ISMAR 2003
2.3. Experimental Procedure
The task was to decide whether there was a path of length
two connection between two highlighted nodes in a
randomly generated graph. On each trial there was either
a path of length two – from one highlighted node to
another with only one intervening node between – or no
such path (Figure 3).
Figure 3: No Connected Path
The 3D graphs displayed consisted of a number of small
nodes arranged randomly in a simulated 17cm³ volume.
In order to generate such graphs, the nodes were divided
into three equally sized groups. Two such groups were
the “potentially highlighted” nodes and the third group
consisted of the “intermediate nodes.” Each potentially
highlighted node was connected via arcs to two different
nodes in the intermediate group. For n nodes, this
produced a total of (4/3 * n) connecting links. All the
nodes were placed randomly in the simulated volume.
Five levels of graph complexity were used:
Level Num. of Nodes Num. of Links
1 21 28
2 36 48
3 48 64
4 63 84
5 75 100
This resulted in 15 graph level to condition combinations.
There were 10 trials per level and all were performed in
each condition, resulting in 150 trials per experimental
session.
Highlighted nodes were drawn in bright red, and the
unhighlighted (intermediate) nodes were drawn in gray.
Lighting was applied to all nodes in the graph. Each node
was set to be 0.4 cm on each side. The connecting arcs
were drawn in blue with two-pixel lines. The background
color was black in all conditions (see Figure 3).
Upon arrival at the lab, the subjects were given a set of
simple written instructions and were allowed two minutes
to practice in each condition and ten practice trials.
Subjects were instructed to “answer as accurately and
quickly as possible” and to respond with a verbal “yes” or
no”. The experimenter recorded the subject’s answer by
pressing either ‘y’ or ‘n’ on the keyboard. The orders of
the conditions, as well as the order of the size of the
graphs presented, were blocked and counterbalanced
across all subjects. Prior to each trial, subjects were told
which graph level to expect. During each trial, subject
were given as much time as necessary to respond. The
response time and response validity were recorded.
2.4. Results
The error data for this experiment is summarized in
Figure 4. The x-axis represents the graph complexity, as
reflected in the number of nodes. The y-axis represents
the mean percent error. As can be seen, the percentage of
errors in the 2D On-screen condition is greater than that
of the 3D On-screen condition and the AR condition.
These results are virtually identical to, and consistent
with, those found in [9]. An analysis of variance revealed
a significant main effect of condition on mean percent
error, F(2,12) = 11.021, p < 0.05. An ANOVA revealed a
significant difference between the 3D and 2D conditions
for mean error with F(1,8) = 16.707, p < 0.05 and a
significant difference between 2D and AR, with F(1,8) =
15.103, p < 0.05. An ANOVA revealed no significant
difference between the 3D and AR conditions.
Figure 4: Mean Percent Error for Experiment 1
3
ISMAR 2003
Figure 5 summarizes the time data in this experiment. It
shows a series of curves roughly separated by one order
of magnitude from each other. The x-axis represents the
number of nodes. The y-axis represents the mean
response time in seconds. Surprisingly, the AR condition
remains the most time-consuming of the conditions across
all levels of graph size. Furthermore, the 2D condition
took the least amount of time across all levels of
complexity. An analysis of variance revealed a
significant main effect of condition on response time with
F(2,12) = 3.975, p < 0.05 and a significant difference
between the AR and 2D conditions, F(1,8) = 7.986, p <
0.05.
Figure 5: Mean Response Time for Experiment 1
2.5. User Feedback
After the experiment the subjects were asked a number of
questions. First they were asked to “rank the three
interfaces in terms of overall ease of use,” with 1 being
assigned to the easiest condition, and 3 to the most
difficult. Figure 6 shows that users preferred the 3D On-
screen and AR conditions over the 2D condition. An
analysis of variance revealed a significant main effect of
condition on ranking, F
(2,45)
= 7.34, p < 0.01.
The second statement was: “please rank the three
interfaces/conditions in the order in which you preferred
the information displayed,” with 1 being the condition
most preferred, and 3 being the least (see figure 6). An
analysis of variance revealed a significant main effect of
condition on the information display preference ranking,
F
(2,21)
= 14.38, p < 0.01.
Figure 6: Avge. Rankings of Ease of Use and
Information Preference
The next questions was to “rank the three
interfaces/conditions in terms of ease of physical
manipulation of the position and orientation of the
graphic” with a score of 1 being the easiest and 3 being
the most difficult (see figure 7). An analysis of variance
revealed no significant main effect of condition on the
physical manipulation ranking.
The final ranking was phrased as: “rank the three
interfaces in terms of how well you believe you performed
the given task” with 1 the best, and 3 the worst (see figure
7). An analysis of variance revealed a significant main
effect of condition on perceived performance ranking,
F
(2,45)
= 35.1, p < 0.01. The majority of subjects believed
they performed best in the 3D On-screen condition, and
many participants thought they performed worst in the 2D
On-screen condition.
Figure 7: Avge. Rankings of Ease of Manipulation and
How Well Task Performed
4
ISMAR 2003
2.6. Discussion
On the survey, 15 of the 16 respondents ranked the
conditions identically for the “overall ease of use” and
“ease of physical manipulation scores.” This may
indicate the potential of AR lies in its usability and
tangibility. This is supported by the rankings between the
AR and the 3D condition, which are strikingly similar.
One of the most common complaints in the AR condition
was the marker tracking. In order to have effective AR
tracking the tracking marker needed to be in view at all
times. This was not possible because a virtual disk
covered the marker. Thus the virtual graph model
sometimes flicked in and out of view.
Subjects were also asked to describe the strategy they
used The most common was a “process of elimination”.
Respondents would focus on one of the highlighted nodes
and then trace a path to the immediately connecting nodes
to see if their arcs angled back toward the other
highlighted node. The most common variation was to
look at the angle of the arcs leaving both highlighted
nodes in an attempt to determine if there was a possible
viewpoint from which these lines intersect.
3. Experiment 2: Stereo vs. Mono
Experiment 2 explored the effect of stereopsis. The task
remains unchanged: path tracing in an interconnected
graph. As in the previous experiment, the independent
variable was the viewing condition. The dependent
variables were percent error and response time. Four
viewing conditions were employed:
1/ On-screen Mono: This was the same as 3D On-
screen interface in Experiment 1. However, the
background was gray and a texture map of the
tracking marker used in the AR condition was
attached to the virtual tracking disk.
2/ On-screen Stereo: Same as condition 1, except in
stereo. The correct view of the graph was generated
for each eye position and continuously updated. The
subject wore a pair of LCD shutter glasses.
3/ Augmented Reality Mono: This interface was the
same as the AR interface in Experiment 1, with the
tracking marker no longer occluded.
4/ Augmented Reality Stereo: Same as the preceding
condition, except in stereo. Two cameras (instead of
one) were mounted on the HMD and the AR image
was presented using quad-buffered stereo.
In light of the findings of Ware and Frank [7], we predict
that stereo will add little positive effect to performance,
both in terms of accuracy and response time.
3.1. Equipment
As in Experiment 1, the experimental display system for
the Augmented Reality conditions consisted of a SONY
Glasstron LDI-100B HMD. However, in this experiment,
two ELMO mini-NTSC cameras were mounted on the
HMD. A second video capture card was installed to
handle the second video stream. The Sony Glasstron
LDI-100B supports quad-buffered stereo. The same
tracking card and pattern were used. The StereoGraphics
CrystalEyes 3D LCD shutter glasses, in synch with the
emitter and the monitor provided the stereo effect in the
Onscreen Stereo condition. The vertical refresh rate of
the monitor was set to 120 Hz, with each eye receiving an
update at 60 Hz (Figure 8).
3.2. Participants
Participants in Experiment 2 consisted of 16 subjects,
ranging between 18 and 48 years old, with an average age
of 25. The female to male ratio was 8:8. Only one of the
subjects was left-handed; the remaining 15 were right
handed. One of the subjects had previously used an AR
interface, but none had previously used a stereographic
display system. All were familiar with the use of a
mouse
.
Figure 8: The On-screen stereo setup.
3.3. Experimental Procedure
The experimental procedure in Experiment 2 was
virtually unchanged from that of the preceding
experiment with the same task. However, due to the
increased number of conditions, and in order to complete
the required number of trials in the allotted experimental
time-slot, we were forced to lower the number of levels of
graph complexity from 5 to 4. The range of complexity –
5
ISMAR 2003
from 21 to 75 – remained the same. The levels of graph
complexity were as follows:
Level Num. of Nodes Num. of Links
1 21 28
2 39 52
3 57 76
4 75 100
This resulted in 16 graph level to condition (interface)
combinations. There were 10 trials per level and all
levels were performed with each interface. This resulted
in a total of 160 trials per experimental session.
The color and lighting of the nodes and arcs remains
unchanged from Experiment 1. However, in order to
reduce the ghosting affects associated with stereo, the
background color was changed to gray in all conditions.
Subjects wore the shutter glasses in both the On-screen
stereo and On-screen mono conditions.
As previously mentioned, the tracking marker in the AR
conditions was no longer occluded and a texture map of
the square and symbol was placed in the On-screen
conditions. The decision to do this was at the request of
Experiment 1’s subjects (see section 2.5)
Upon arrival at the lab, the subjects were given a set of
simple written instructions outlining the task. Before
beginning the experiment proper, subjects were allowed
two minutes to practice in each condition and ten practice
trials to make sure they understood the task. Subjects
were instructed toanswer as accurately and quickly as
possible” and to respond with a verbal “yes” or “no”. The
experimenter recorded the subject’s answer by pressing
either ‘y’ or ‘n’ on the keyboard. The orders of the
conditions, as well as the order of the size of the graphs
presented, were blocked and counterbalanced across all
subjects. Prior to each block of trials, subjects were told
which graph level to expect. During each trials, subject
were given as much time as necessary to respond. The
response time and response validity were recorded. All
participants were given as many breaks as they required
and the overall experiment duration averaged one hour
and fifteen minutes.
3.4. Results
The error data for this experiment is summarized in
Figure 9. The x-axis represents the graph complexity, as
reflected in the number of nodes. The y-axis represents
the mean percent error. As expected the mean error
increases with the number of nodes, but there is little
difference in error percentage across conditions for a
given graph complexity. An analysis of variance for this
data reveals no significant main effects or differences
between conditions.
Figure 9: Mean Percent Error for Experiment 2.
Figure 10 summarizes the time data in Experiment 2. It
shows a series of curves almost indistinguishable. The x-
axis represents the number of nodes, from 21 to 75. The
y-axis represents the mean response time as measured in
seconds. An analysis of variance for this data reveals no
significant effects or difference between conditions.
Figure 10: Mean Response Time for Experiment 2.
6
ISMAR 2003
3.5. User Feedback
As in Experiment 1, the participants were asked for their
feedback on the experimental conditions. The
quantitative results for this survey consisted of two
ranking questions. In the first ranking, subjects were
asked to “rank the four interfaces in terms of overall ease
of use,” with 1 being the easiest, and 4 the most difficult.
The responses are summarized in Figure 11. The four
conditions tested are on the x-axis, while the average
ranking are displayed on the y-axis.
Figure 11: Avge. Rankings of Ease of Use
Figure 11 shows that subjects felt there was little
difference in ease of use among the four conditions. It
should be noted that those subjects who did not notice any
particular difference between the AR Mono and the AR
Stereo or between the On-screen Mono and the On-screen
Stereo conditions decided to evenly rank the two
respective conditions. So a subject who did not
differentiate between AR Stereo and AR Mono simply
gave the same ranking to both. This resulted in more than
first (1) and second (2) place rankings for each condition
that third (3) or fourth (4) rankings. An analysis of
variance revealed no significant main effect of condition
on ranking.
The second question was: “please rank the three
interfaces in terms of how well you believe you performed
the given task” with 1 being the best performance, and 4
being the worst performance. The responses to this
ranking are summarized in Figure 12.
Figure 12 shows that the majority of subject believed they
performed best in the AR conditions. It should be noted
that those subjects who did not notice any particular
difference between the AR Mono and the AR Stereo or
between the On-screen Mono and the On-screen Stereo
conditions decided to evenly rank the two respective
conditions. So a subject who did not differentiate
between AR Stereo and AR Mono simply gave the same
ranking to both. It is clear from Figure 12 that a majority
of participants thought they performed worst in the On-
screen mono condition. Subjects that reported ranking the
AR Stereo condition as their best performance (with a
ranking of 1) claimed to have done so because “they
could better distinguish between the lines (arcs).” An
analysis of variance revealed no significant main effect of
condition on ranking.
Figure 12: Avge. Rankings of Perceived Performance
3.6. Discussion
The search strategies used were virtually identical to those
reported in Experiment 1. Most of the subjects reported
using the “process of elimination” strategy, after a small
amount of exploration. Many of the subjects (10 of 16)
reported to have “just looked for the intermediary node”
in the On-screen Stereo and AR Stereo conditions in
which the node count was low (21 or 39).
One of the limitations of stereo video-capture based
Augmented Reality is the fixed camera angles. The two
cameras were mounted 5.8 cm apart (the average inter-
ocular distance) on the top of the HMD, at such an angle
(1.5 degrees) as to focus on an area roughly 85 cm away
(slightly less than the average arm length). However, the
focal length of the cameras remains fixed no matter how
near or far the objects in the field-of-view. This becomes
a problem when the user wishes to zoom in close to the
tracking-card, as the camera angles do not change with
the distance of the object. This limitation is not inherent
in screen-based quad-buffered stereo (On-screen stereo
condition) because the OpenGL buffer rendering code
7
ISMAR 2003
dynamically changes the focal length based upon the
location of the 3D object.
4. Conclusion
In this paper we reproduced the classic experiments of
Ware and Franck [9] to evaluate the usefulness of using
Augmented Reality for visualizing complex three-
dimensional node-graph representations. In their work
they found that kinetic depth cues were more important
than stereo cues in performing link analysis on complex
graphs.
In this paper we have performed two experiments. In the
first we compared link understanding in 2D and 3D
screen conditions to an AR interface. Subjects took longer
in the AR condition than the two screen-based conditions,
but produced as few errors as the 3D screen condition and
significantly less than the 2D screen case. In subjective
rankings of Ease of Use, Information Display Preference,
Ease of Manipulation and Perceived Performance,
subjects felt that the AR condition was equivalent to the
3D screen condition and significantly better than the 2D
screen case.
The second experiment explored the effect of adding
stereo cues and compared accuracy and timing results
across four conditions (stereo and non-stereo screen
interfaces and stereo and non-stereo AR). Although
performance got worse as the number of graph nodes
increased, there was no difference in performance
between these conditions at a given graph complexity.
The major difference between the 2D screen condition
and the 3D and AR conditions is in the ability to rotate the
model. Thus these results support those of Ware and
Franck, namely that graph understanding is significantly
improved by the support for kinetic depth cues. The lack
of performance and accuracy differences between the
stereo and non-stereo conditions similarly highlight the
dominance of kinetic over stereo cues for this task.
In experiment one, users performed as accurately with the
AR interface as with the 3D screen interface and felt it
was just as good to use. This implies that AR interfaces
could be an effective way to visualize abstract
information such as interconnected graphs.
One key advantage of AR interfaces is the support for a
tangible interaction metaphor, with its direct mapping
between a real-world physical object and virtual object.
The use of a tangible AR metaphor enables users to easily
manipulate the graph content and view it from any
perspective, improving their comprehension.
Even though the users performed as accurately in the AR
interfaces they took more time than in the 2D and 3D
screen conditions. This may have been because of the
perceptual qualities of the HMD compared the screen.
Although the screen and HMD had the same resolution,
they both had different Fields of View and color and
contrast properties.
Although these results are interesting, there are just the
beginning. In the future we want to explore other types of
graph visualization and comprehension tasks and compare
between screen-based and AR visualization. We also want
to explore a wider range of manipulation and interaction
techniques for AR interfaces for scientific visualization.
5. Acknowledgements
The primary source of support for this research came
from the Lockheed-Martin Gi2Vis Grant. Thanks to
Hannah Slay of the WCL at the University of South
Australia for her help and insight. Thanks to Kiyoshi
Kiyokawa of Osaka University for his help, criticism and
encouragement. We also thank Konrad Schroder of the
HIT Lab Seattle for much needed technical support.
Finally, thanks go to the lab members at the HIT Lab
Seattle and the HIT Lab New Zealand for providing such
an amazing collaborative atmosphere.
6. References
[1] ARToolKit 2001. ARToolKit website:
http://www.hitl.washington.edu/artoolkit.
[2] K. Arthur, K.S. Booth, and C. Ware. “Evaluating
Human Performance for Fishtank Virtual Reality.” ACM
Transactions on Information Systems, 11(3), 216-266,
1993.
[3] A. Fuhrmann, H. Löffelmann, D. Schmalstieg
“Collaborative Augmented Reality: Exploring Dynamical
Systems” IEEE Visualization 1997, pp. 459-462,
November 1997.
[4] H. Kaufmann, D. Schmalstieg, M. Wagner.
“Construct3D: A Virtual Reality Application for
Mathematics and Geometry Education” Education and
Information Technologies 5:4, special issue on "Virtual
Reality", pp. 263-276, 2000.
[5] G. Parker, G. Franck, and C. Ware. “Visualization of
Large Nested Graphs in 3D: Navigation and Interaction.”
Journal of Visual Languages and Computing, 9 299-317,
1998.
[6] R.L. Sollenberger, and P. Milgram. “A comparative
Study of Rotational and Stereoscopic Computer Graphic
Depth Cues.” Proceedings of the Human Factors Society
Annual Meeting, 1452-1456, 1991.
[7] R.L. Sollenberger, and P. Milgram. “The effects of
Stereoscopic and Rotational Displays in a Three-
Dimensional Path-Tracing Task.” Human Factors, 35 (3)
483-500, 1993.
[8] C. Ware, K. Arthur, and K.S. Booth. “Fishtank
Virtual Reality.” INTERCHI’93 Technical Paper
Proceedings, 37-42, 1993.
8
ISMAR 2003
[9] C. Ware, and G. Franck. “Evaluating Stereo and
Motion Cues for Visualizing Information Nets in Three
Dimensions.” ACM Transactions on Graphics, 15(2)
121-139, 1996.
[10] C. Ware, and J. Rose. “Rotating Virtual Objects with
Real Handles.” ACM Transactions on CHI, 6(2) 162-180,
1999.
9
... Combining AR and a node-link diagram can provide an immersive and ubiquitous experience that engages users to explore, embed richer information and content that extends the boundaries of the graph, and deal with data sets that have a strong connection to the physical objects or space. Belcher et al. [20] also identified three potential benefits of an AR node-link diagram, including increased comprehension, larger display, and enhanced spatial recall of the graph. ...
... Their study results showed that a 3D graph enhanced accuracy and spatial comprehension for abstract data analysis with the depth and motion cues it provided. Belcher et al. [20] compared the effect of using AR for graph link analysis to a desktop interface. According to their study, a tangible AR interface was better suited in the graph exploration than the desktop interface, while the stereo-graphic viewing had little effect on comprehension and task performance. ...
... One possible reason for this is that participants spent more time in AR. Another possible explanation is that the spatial layouts and embodied movements could enhance the recall and comprehension of the graph, which was also mentioned in previous works (e.g., [20]). ...
Article
Full-text available
Recent research in the area of immersive analytics demonstrated the utility of augmented reality for data analysis. However, there is a lack of research on how to facilitate engaging, embodied, and interactive AR graph visualization. In this paper, we explored the design space for combining the capabilities of AR with node-link diagrams to create immersive data visualization. We first systematically described the design rationale and the design process of the mobile based AR graph including the layout, interactions, and aesthetics. Then, we validated the AR concept by conducting a user study with 36 participants to examine users’ behaviors with an AR graph and a 2D graph. The results of our study showed the feasibility of using an AR graph to present data relations and also introduced interaction challenges in terms of the effectiveness and usability with mobile devices. Third, we iterated the AR graph by implementing embodied interactions with hand gestures and addressing the connection between the physical objects and the digital graph. This study is the first step in our research, aiming to guide the design of the application of immersive AR data visualization in the future.
... 2. State of the Art: Abstract 3D Visualizations in Immersive Environments [389] [199] [143] [209] 2. State of the Art: Abstract 3D Visualizations in Immersive Environments Visualization Technique References Node-Link Graph [18,26,75,98,115,149,209,213,236,349,354,360,398,399,428] Scatterplots [6,9,26,47,71,73,74,96,111,116,121,208,221,260,261,265,266,301,311,341,364,377,403,421] Parallel Coordinates [26,47,48,73,74,311] Glyphs, Icons & Symbols [19,377,428] Geographic [15,47,152,158,323,341,389,422] Volume [143,168,236,255,341,354,377] Flow [16,158,166,181,255] Other [47,71,149,198,199,221,227,265,311,332,341,405] # papers Our analysis of these visualization techniques shows an unbalanced distribution, with a clear focus on two types of techniques under investigation: node-link graph and scatterplot, which comprise 39 out of 48papers. As can be seen in Figure 2.3, there has been a sustained interest in these techniques for many years, while the investigation of geographic visualizations has recently experienced a boom. ...
... References Clustering / Classification [6,48,116,208,266,301] Anomaly Detection [6,116,266,403] Pattern Analysis [143,260,301,403] Visual Search [6,18,75,96,98,152,158,213,221,266,301,332,360,364,377,389,398,399,403,422,428] Overview & Details on Demand [15,16,19,121,149,152,158,168,181,221,236,255,332,364,377,405,421,428] Comparative Analysis [166,198,199,227,236,422] Data Enrichment [301,311,349] Not specified [26,47,71,73,74,149,209,261,265,311,323,328,341] # papers year Figure 2.10: Literature review. Distribution of analysis tasks over time. ...
... Especially in exploratory analysis scenarios visual search tasks are often deployed. Many papers describe visual search tasks in volume visualizations [96], geographic visualizations [152,389], scatterplot visualizations [6,266,301,364,377], flow visualizations [422], and network visualizations [18,75,98,213,360,398,399]. While in most papers the task is described as a generic way to explore data in order to build new hypotheses, in several works specific tasks were described and deployed in user studies. ...
... In view of the special setup and the relatively small size of the random networks used, the results must be checked for more general evidence. Belcher et al. [BBHS03] investigated the use of AR for the analysis of complex networks and provided a user study based on artificially created networks. They reproduced the classical experiments by Ware and Franck [WF96] and compared AR with 2D and 3D screen settings using simple node-link visualizations. ...
... Besides scatterplots, graph visualizations are also quite popular for comparing different media. For instance, Belcher et al. [BBHS03] compared the medium screen with an AR HMD for path tracing performance on graph visualizations. The focus was on evaluating the benefit of stereo-vision for path tracing in 3D graphs, which were displayed as 2D projections on a screen, in 3D on a screen, and in 3D in AR. ...
... Slay et al. [SPVT01] used the DSTO InVision system to evaluate different interaction modes in AR. Belcher et al. [BBHS03] used a SONY Glasstron LDI-100B HMD in combination with an ELMO mini camera and a cardboard disc with tracking markers to visualize manipulable virtual elements. ...
Article
Full-text available
After a long period of scepticism, more and more publications describe basic research but also practical approaches to how abstract data can be presented in immersive environments for effective and efficient data understanding. Central aspects of this important research question in immersive analytics research are concerned with the use of 3D for visualization, the embedding in the immersive space, the combination with spatial data, suitable interaction paradigms and the evaluation of use cases. We provide a characterization that facilitates the comparison and categorization of published works and present a survey of publications that gives an overview of the state of the art, current trends, and gaps and challenges in current research.
... Path tracing has also been used to compare monoscopic and stereoscopic viewing of structures resembling angiograms [48], [49]. Studies comparing 2D and 3D layouts for path tracing within networks are reported in [10], [50], [51]. Two of these [50], [51] used networks with random layouts, making them less relevant to real visualizations. ...
... Studies comparing 2D and 3D layouts for path tracing within networks are reported in [10], [50], [51]. Two of these [50], [51] used networks with random layouts, making them less relevant to real visualizations. The most recent [10], summarized below, is also the most carefully designed. ...
Preprint
It is common to advise against using 3D to visualize abstract data such as networks, however Ware and Mitchell's 2008 study showed that path tracing in a network is less error prone in 3D than in 2D. It is unclear, however, if 3D retains its advantage when the 2D presentation of a network is improved using edge-routing, and when simple interaction techniques for exploring the network are available. We address this with two studies of path tracing under new conditions. The first study was preregistered, involved 34 users, and compared 2D and 3D layouts that the user could rotate and move in virtual reality with a handheld controller. Error rates were lower in 3D than in 2D, despite the use of edge-routing in 2D and the use of mouse-driven interactive highlighting of edges. The second study involved 12 users and investigated data physicalization, comparing 3D layouts in virtual reality versus physical 3D printouts of networks augmented with a Microsoft HoloLens headset. No difference was found in error rate, but users performed a variety of actions with their fingers in the physical condition which can inform new interaction techniques.
... Since the first introduction of AR technologies, researchers have first continued to improve AR hardware, algorithms (e. g., tracking algorithms), and then the user interfaces, with several communities showing interest in augmenting physical reality with virtual contents. AR has been applied in many disciplines, including medicine [23,232], education [5,210], archaeology [22,28], games [211], remote expert guidance [92,162,200], industry [63,68,126,204], crisis response [196,220], information visualization [25,60,159,227], and now scientific visualization (Chapter 3). Over the same period, smartphone applications (e. g., the Pokémon Go game) have increased the public's knowledge of, and interest in, Augmented Reality. ...
... Immersive Analytics (IA) is at the junction of MR Environments and Visual Analytics [269] ( Figure 1.3). AR can be a medium to support users in their analytical tasks [25,91,185], collaborative or not. Such systems need high computational power for both simulations and rendering to provide quick feedback. ...
Thesis
I studied the benefits and limitations of Augmented Reality (AR) Head-Mounted Displays (AR-HMDs) for collaborative 3D data exploration. Prior of conducting any projects, I saw in AR-HMDs benefits concerning their immersive features: AR-HMDs merge the interactive, visualization, collaborative, and users' physical spaces together. Multiple collaborators can then see and interact directly with 3D visuals anchored within the users' physical space. AR-HMDs usually rely on stereoscopic 3D displays which provide additional depth cues compared to 2D screens, supporting users at understanding 3D datasets better. As AR-HMDs allow users to see each other within the workspace, seamless switches between discussion and exploration phases are possible. Interacting within those visualizations allow for fast and intuitive 3D direct interactions, which yields cues about one's intentions to others, e.g., moving an object by grabbing it is a strong cue about what a person intends to do with that object. Those cues are important for everyone to understand what is currently going on. Finally, by not occluding the users' physical space, usual but important tools such as billboards and workstations performing simulations are still easily accessible within this environment without wearing off the headsets. That being said, and while AR-HMDs are being studied for decades, their computing power before the recent release of the HoloLens in 2016 was not enough for an efficient exploration of 3D data such as ocean datasets. Moreover, previous researchers were more interested in how to make AR possible as opposed to how to use AR. Then, despite all those qualities one may think prior of working with AR-HMDs, there were almost no work that discusses the exploration of such 3D datasets. Moreover AR-HMDs are not suitable for 2D input which are however commonly used with usual explorative tools such as ParaView or CAD software, where users such as scientists and engineers are already efficient with. I then theorize in what situations are AR-HMDs preferable. They seem preferable when the purpose is to share insights with multiple collaborators and to explore patterns together, and where explorative tools can be minimal compared to what workstations provide as most of the prior work and simulations can be done before hand. I am thus combining AR-HMDs with multi-touch tablets, where I use AR-HMDs to merge the visualizations, some 3D interactions, and the collaborative spaces within the users' physical space, and I use the tablets for 2D input and usual Graphical User Interfaces that most software provides (e.g., buttons and menus). I then studied low-level interactions necessary for data exploration which concern the selection of points and regions inside datasets using this new hybrid system. The techniques my co-authors and I have chosen possess different level of directness that we investigated. As this PhD aims at studying AR-HMDs within collaborative environments, I also studied their capacities to adapt the visual to each collaborator for a given anchored 3D object. This is similar to the relaxed "What-You-See-Is-What-I-See" that allows, e.g., multiple users to see different parts of a shared document that remote users can edit simultaneously. Finally, I am currently (i.e., is not finished by the time I am writing this PhD) studying the use of this new system for the collaborative 3D data exploration of ocean datasets that my collaborators at Helmholtz-Zentrum Geesthacht, Germany, are working on. This PhD provides a state of the art of AR used within collaborative environments. It also gives insights about the impacts of 3D interaction directness for 3D data exploration. This PhD finally gives designers insights about the use of AR for collaborative scientific data exploration, with a focus on oceanography.
... While the full potential of the nascent VR and AR technologies has yet to be realized [286], the use of these technologies for network analysis offers a number of benefits. Three-dimensional (3D) VR representations of networks are reported to provide numerous benefits when compared with classic 2D networks [287], including facilitating accurate distance assessments between nodes [288], path finding among large numbers of nodes [289,290], community detection in complex graphs [291,292], and facilitating the accuracy of users' mental models [293]. VR and AR platforms also lend themselves to collaborative work [294][295][296][297], including the creation of a 3D layout of a large Twitter network that provides collaborative interactions [298]. ...
Article
Full-text available
Understanding the complex process of information spread in online social networks (OSNs) enables the efficient maximization/minimization of the spread of useful/harmful information. Users assume various roles based on their behaviors while engaging with information in these OSNs. Recent reviews on information spread in OSNs have focused on algorithms and challenges for modeling the local node-to-node cascading paths of viral information. However, they neglected to analyze non-viral information with low reach size that can also spread globally beyond OSN edges (links) via non-neighbors through, for example, pushed information via content recommendation algorithms. Previous reviews have also not fully considered user roles in the spread of information. To address these gaps, we: (i) provide a comprehensive survey of the latest studies on role-aware information spread in OSNs, also addressing the different temporal spreading patterns of viral and non-viral information; (ii) survey modeling approaches that consider structural, non-structural, and hybrid features, and provide a taxonomy of these approaches; (iii) review software platforms for the analysis and visualization of role-aware information spread in OSNs; and (iv) describe how information spread models enable useful applications in OSNs such as detecting influential users. We conclude by highlighting future research directions for studying information spread in OSNs, accounting for dynamic user roles.
... Some studies indicate that AR has a positive impact on students' motivation (Kaufmann & Dünser, 2007;Liu, Tan & Chu, 2009); this could be attributed to the creativity of AR educational experiences, which leads participants to fully engage in the learning process (Yilmaz, Kucuk & Goktas, 2017). On the other hand, some studies point out that some participants have reported usability issues and rated AR difficult to use (Belcher, Billinghurst, Hayes & Stiles, 2003;Kaufmann & Dünser, 2007). ...
Article
Full-text available
To exploit the potential of immersive network analytics for engaging and effective exploration, we promote the metaphor of "egocentrism", where data depiction and interaction are adapted to the perspective of the user within a 3D network. Egocentrism has the potential to overcome some of the inherent downsides of virtual environments, e.g., visual clutter and cyber-sickness. To investigate the effect of this metaphor on immersive network exploration, we designed and evaluated interfaces of varying degrees of egocentrism. In a user study, we evaluated the effect of these interfaces on visual search tasks, efficiency of network traversal, spatial orientation, as well as cyber-sickness. Results show that a simple egocentric interface considerably improves visual search efficiency and navigation performance, yet does not decrease spatial orientation or increase cyber-sickness. An occlusion-free Ego-Bubble view of the neighborhood only marginally improves the user's performance. We tie our findings together in an open online tool for egocentric network exploration, providing actionable insights on the benefits of the egocentric network exploration metaphor.
Article
Full-text available
Among the existing platforms, the mobile platform provides most of the augmented reality experiences for users. Augmented reality solutions have been applied in several contexts due to technological advances in this area. However, many of these advances are still out of reach of more massive access by regular users due to access limitations or the cost of these new technologies. After conducting a literature review considering the application of data visualization in augmented reality environments, we have identified an evolution of equipment used in this context with a decrease of low-cost proposals. Hence, the works related to these two areas showed high use of technologies not yet accessible to the vast majority of users. Thus, we propose a low-cost augmented reality application that can be used on mobile devices for data visualization. We developed a hybrid proposal of an augmented reality prototype to interacting with cards through the mobile device to support the process of creating data visualization. The tangible cards have a design that helps the user in the visualization creation process, indicating each card’s role in the interaction process. The proposed application supports a multi-user environment, allowing multiple users to interact simultaneously with data visualizations in the AR environment. The visualizations are generated on a dedicated server for this function, providing a set of seven different types of visualization. To validate the developed prototype we performed two tests, the first being a stress assessment on the server that generates data visualizations to analyze the proposed application’s scalability. The second was a remote usability test with twelve participants, to evaluate the interaction and design of the cards, and the viability of this solution for augmented reality applications.
Conference Paper
Full-text available
The defining characteristics of what we call “Fish Tank Virtual Reality” are a stereo image of a three dimensional (3D) scene viewed on a monitor using a perspective projection coupled to the head position of the observer. We discuss some of the relative merits of this mode of viewing as compared to head mounted stereo displays. In addition, we report the experimental investigation of the following variables: 1) whether or not the perspective view is coupled to the actual viewpoint of the observer, 2) whether stereopsis is employed. Experiment 1 involved the subjective comparison of pairs of viewing conditions and the results suggest that head coupling may be more important than stereo in yielding a strong impression of three dimensionality. Experiment 2 involved subjects tracing a path from a leaf of a 3D tree to the correct root (there were two trees intermeshed). The error rates ranged from 22% in the pictorial display, to 1.3% in the head coupled stereo display. The error rates for stereo alone and head coupling alone were 14.7% and 3.2% respectively. We conclude that head coupling is probably more important than stereo in D visualization and that head coupling and stereo combined provide an important enhancement to monitor based computer graphics.
Article
Full-text available
A series of three experiments investigated the effectiveness of stereoscopic and rotational display techniques for the purpose of establishing human factors guidelines for the design of three-dimensional (3D) displays. In the described experiments, depth perception was evaluated by examining accuracy in a 3D path-tracing task, with stimulus displays resembling the structure of cerebral angiograms. The first experiment allowed subjects to control rotation in dynamic displays. The results indicated that performance improved using either technique relative to viewing two-dimensional (2D) displays. However, rotational displays were superior to stereoscopic displays, and performance was best when both techniques were combined. The second experiment compared subject-controlled rotation with observation of continuously rotating displays at different rates of rotation. Performance declined at faster rotation rates; however, there were no advantages of subject-controlled rotation. In the third experiment, performance in rotational displays was no better than that in stereoscopic displays enhanced with multiple static viewing angles. However, performance was always best when both 3D techniques were jointly implemented. The results are discussed in terms of the visual information available using either 3D display technique and are related to the weighted additive model of depth perception.
Article
Full-text available
This paper concerns the benefits of presenting abstract data in 3D. Two experiments show that motion cues combined with stereo viewing can substantially increase the size of the graph that can be perceived. The first experiment was designed to provide quantitative measurements of how much more (or less) can be understood in 3D than in 2D. The 3D display used was configured so that the image on the monitor was coupled to the user's actual eye positions (and it was updated in real-time as the user moved) as well as being in stereo. Thus the effect was like a local "virtual reality" display located in the vicinity of the computer monitor. The results from this study show that head-coupled stereo viewing can increase the size of an abstract graph that can be understood by a factor of three; using stereo alone provided an increase by a factor of 1.6 and head coupling alone produced an increase by a factor of 2.2. The second experiment examined a variety of motion cues provided by head coupl...
Article
Full-text available
Construct3D is a three dimensional geometric construction tool based on the collaborative augmented reality system "Studierstube". Our setup uses a stereoscopic head mounted display (HMD) and the Personal Interaction Panel (PIP) -- a two-handed 3D interaction tool that simplifies 3D model interaction. Means of application in mathematics and geometry education at high school as well as university level are being discussed. A pilot study summarizes the strengths and possible extensions of our system. Anecdotal evidence supports our claim that the use of Construct3D is easy to learn and encourages experimentation with geometric constructions. Keywords: Virtual reality, geometry education, enhancing spatial abilities, userinterface design, 3D modelling Introduction Spatial abilities present an important component of human intelligence. The term spatial abilities covers five components, spatial perception, spatial visualization, mental rotations, spatial relations and spatial or...
Article
Full-text available
Times for virtual object rotations reported in the literature are of the order of ten seconds or more and this is far longer than it takes to manually orient a "real" object, such as a cup. This is a report of a series of experiments designed to investigate the reasons for this difference and to help design interfaces for object manipulation. The results suggest that two major factors are important. Having the hand physically in the same location as the virtual object being manipulated is one. The other is based on whether the object is being rotated to a new, randomly determined orientation, or is always rotated to the same position. Making the object held in the hand have the same physical shape as the object being visually manipulated was not found to be a significant factor. The results are discussed in the context of interactive virtual environments. Categories and Subject Descriptors: H.1.2 [Models and Principles]: User/Machine Systems -- human factors; I.3.6 [Computer Graphics]:...
Article
人間にとって自然で直感的なインタフェースを設計するためのデザインコンセプトとして,Tangible User Interfaceをあげることができる.このコンセプトに基づき設計された拡張現実感システムを我々はTangible Augmented Reality Interface と呼び,そのプロトタイプシステムをいくつか試作してきた.本論文では,このTangible Augmented Reality Interface について,その特徴,有効性を説明する.また,具体的なプロトタイプシステムとして「Magic Book」および「Magic Paddle」を紹介する.Magic Book は,仮想3次元絵本であり,ユーザは実際の本の上に,仮想的な3次元オブジェクトを見ることができる.さらに,そのシーンの中に入り込むことも可能である.また,この仮想世界の中で,複数のユーザがコミュニケーションをとることもできる.Magic Paddleは,仮想のミニチュア家具のレイアウトシステムで,紙製のパドル(へら状の物体)で,仮想の家具に対する操作ができる.直感的なインタフェースが実現されており,容易に3次元オブジェクトの操作ができる.
The present research investigated the effectiveness of rotational and stereoscopic display techniques as applied to practical applications in computer graphics. In the described experiments, depth perception was evaluated by examining accuracy in a 3D path-tracing task, with stimulus displays resembling the structure of cerebral angiograms. In the first experiment, task performance was determined without 3D enhancements, with either technique used alone and with both techniques combined. The results indicated that performance improved using either technique, however, performance with rotational displays was superior to stereoscopic displays, and performance was best when both techniques were combined. The results of a second experiment revealed that rotational displays were no better than stereoscopic displays enhanced with multiple static viewing angles. The experimental findings are discussed in terms of the visual information available using either 3D display technique as well as with respect to the hardware requirements of both display systems. The results are also related to the weighted additive model of depth perception.
Conference Paper
We present collaborative scientific visualization in STUDIERSTUBE. STUDIERSTUBE is an augmented reality system that has several advantages over conventional desktop and other virtual reality environments, including true stereoscopy, 3D-interaction, individual viewpoints and customized views for multiple users, unhindered natural collaboration and low cost. We demonstrate the application of this concept for the interaction of multiple users and illustrate it with several visualizations of dynamical systems in DynSys3D, a visualization system running on top of AVS.
Article
In this paper we present collaborative scientific visualization in STUDIERSTUBE. STUDIERSTUBE is an augmented reality system that has several advantages over conventional desktop and other virtual reality environments, including true stereoscopy, 3D-interaction, individual viewpoints and customized views for multiple users, unhindered natural collaboration and low cost. We demonstrate the application of this concept for the interaction of multiple users and illustrate it with several visualizations of dynamical systems in DynSys3D, a visualization system running on top of AVS. CR Categories and Subject Descriptors: I.3.1 [Computer graphics]: Three-Dimensional Displays, I.3.7 [Computer graphics]: Virtual Reality Keywords: virtual environments, scientific visualization, dynamical systems, augmented reality 1 INTRODUCTION As a highly interdisciplinary field, scientific visualization frequently requires experts with different background to cooperate closely. Many valuable insights only...
Article
Most systems for visualizing large information structures use 2D graphics to view networks of nodes and arcs that represent data. To understand large structures it is often necessary to show both small-scale and large-scale structure. This has been called the problem of focus and context. Distortion, rapid zooming, elision, and multiple windows are all techniques that have been developed to provide both focus and context within single representations. We review these techniques and argue that 3D visualization has a number of advantages. A system called NestedVision3D (NV3D) will be presented that has been developed to investigate the use of 3D visualization for understanding the structure of large computer programs. NV3D is a system for visualizing large nested graphs using interactive 3D graphics. It has been tested with graphs containing more than 35,000 nodes and 100,000 relationships. We describe NV3D and its design philosophy. Basic navigation is facilitated by a set of 3D widget...