Content uploaded by David Baum
Author content
All content in this area was uploaded by David Baum on Sep 28, 2017
Content may be subject to copyright.
GETAVIZ: Generating Structural, Behavioral, and
Evolutionary Views of Software Systems for
Empirical Evaluation
David Baum∗, Jan Schilbach∗, Pascal Kovacs∗, Ulrich Eisenecker∗, and Richard M¨
uller∗
∗Information Systems Institute
Leipzig University,
{baum, schilbach, kovacs, eisenecker, rmueller}@wifa.uni-leipzig.de
Abstract—Software visualizations are used to support stake-
holders in software engineering activities like development,
project management, and maintenance. The respective tasks
determine which aspects of software, i.e., structural, behavioral
and/or evolutionary information, need to be visualized. To pro-
mote the usage of software visualizations they have to optimally
support the needs of the respective stakeholder for the specific
task at hand. Therefore, we see the necessity to create innovative
visualizations and to optimize existing ones. In order to achieve
this, it is necessary to empirically evaluate the different visual-
izations and their variants. In this paper, we present GETAVIZ as
a toolset to support these processes, i.e., designing visualizations,
generating task- and role specific visualizations, and conducting
empirical evaluations. The toolset implements the concept of
generative and model-driven software visualization and makes it
possible to generate different visualizations for all three aspects
of software. Its strength lies in its adaptability, so that new
visualizations and variations of existing ones can be implemented
easily. In addition to the generator this toolset contains several
extractors for different programming languages, a browser-based
user interface for viewing and interacting with visualizations, and
an evaluation server to facilitate the execution of local and remote
experiments. The paper illustrates the capabilities of GE TAVIZ
and it discusses plans for its further development.
Index Terms—3D, evaluation process, empirical evaluation,
software visualization, user interface, visualization metaphor
I. INTRODUCTION
Software visualization research has produced many propos-
als to support and improve software engineering activities and
tasks. However, there is still a big gap between research and
practice. The majority of the research results is not used by
practitioners [1], [2]. The reasons for this are manifold. We
believe, one reason for this situation is missing knowledge,
when to use which visualization.
To gain this missing knowledge, M¨
uller et al. [3] adapted the
work of Munzner et al. [4], [5] to the field of software visual-
ization and proposed a theoretical framework for the evaluation
of software visualizations. The framework takes the different
factors user,task,software artifact,representation,navigation
& interaction, and implementation into account. It supports
researchers in designing and planning empirical evaluations.
To conduct empirical studies efficiently, a tailorable evaluation
process as well as an extensive tool coverage for this process
are necessary. Thus, we extend earlier work on a user-centered
approach for optimizing information visualizations concerning
readability, comprehensibility, and user satisfaction [6].
In this paper we present GETAVIZ, a toolset underpinning
the theoretical framework for the evaluation of software vi-
sualization [3] with the tools to conduct these evaluations. It
provides tool support for the complete design and evaluation
process of software visualizations, including the automatic
generation of role- and task-specific visualizations.
II. GETAVIZ
The main objective of the presented toolset is to conduct
empirical evaluations locally and remotely in an efficient and
reproducible way for different visualization metaphors as well
as their variants. The complete process starting from designing
a task- and role-specific software visualization up to its em-
pirical evaluation has to be supported. To accomplish this, we
have developed multiple tools and integrated them in a toolset
as depicted in Fig. 1. It consists of 1) extractors for different
programming languages and version control systems, 2) a
generative and model-driven software visualization generator,
3) a browser-based user interface for viewing and interacting
with a visualization, and 4) an evaluation server for simplifying
and standardizing empirical evaluations. The whole toolset is
demonstrated in an additional screencast.1Next, we describe
the individual components of the toolset.
III. EXT RAC TOR S
The generator takes one of three language independent
models as input, thus the whole generation process is language
independent. The FAMIX 3.0 meta-model family consists
of three meta-models originally designed for software anal-
ysis [7]: FAMIX as basis for modeling structural aspects,
DYNAMIX for adding behavioral aspects [8], and HISMO for
adding evolutionary aspects of software systems [9]. All meta-
models are suited for object-oriented as well as structured
programming languages.
To generate the actual models we use language dependent
extractors. Currently, there are FAMIX extractors for Java,
Ruby, ABAP, and C], DYNAMIX extractors for Java and
Ruby, and HISMO extractors for git and SVN.
1https://youtu.be/OOEQNhyTrLo
Fig. 1. Overview of the toolset and its implementation techniques
IV. GEN ER ATOR
The software visualization generator supports the language
independent visualization in 2D and 3D of all three aspects of
software systems, i.e., structure, behavior and evolution [10].
It is designed to be highly flexible and easily extensible, e.g.,
to re-implement visualizations from different researchers to
integrate them into an unified process of empirical evaluation.
GETAVIZ already supports multiple visualization metaphors,
variants per metaphor, and output formats as described in the
following sections.
A. Supported Metaphors and Variants
Currently, GETAVIZ supports four major visualization
metaphors, named Recursive Disk (RD),City,Plant, and
Multisphere Metaphor (cf. Fig. 2). These metaphors were
implemented as part of a research project comparing software
visualizations in 2D, 3D, and virtual reality. Since there is an
ongoing debate about the usefulness and necessity of the third
dimension in information visualization [11]–[13], we think it
is important to support 2D and 3D equally.
We reimplemented the City metaphor from Wettel and
Lanza [14] (cf. Fig. 2(a)) in order to evaluate the RD metaphor
(cf. Fig. 2(b)) against this established visualization. The Plant
metaphor (cf. Fig. 2(c)) is a real-world metaphor visualizing
the same information as a flower bed, where the stamens
and petals represent methods and attributes of a class. The
Multisphere metaphor (cf. Fig. 2(d)), in contrast, is an abstract
metaphor depicting structural information as nested spheres.
Fig. 2 shows one variant for each metaphor. Every metaphor
can be altered in several ways, e.g., by adding information to
the visualization or removing information from it as shown for
the RD metaphor in Fig. 3 or by using different animations,
layout algorithms, or visual channels, such as color, shape,
tilt, and size [15] as shown in Fig. 4 for the City metaphor.
Next, we describe the RD metaphor and some variants to
show the adaptability and completeness of our visualization
approach.
1) RD Structure Visualization: The RD metaphor displays
all important structural aspects of software system including
packages, classes, methods, and attributes [16]. The glyphs
used by the metaphor are nested disks, rings, and segments.
The nesting represents the containment relations of the visu-
alized software entities.
In this variant, the RD metaphor is two-dimensional. Hence,
the third dimension can be used to convey further information
about the entities represented by the glyphs. One possible
variant is depicted in Fig. 3(a). Here, the height of the methods
shows the cyclomatic complexity of the methods.
2) RD Behavior Visualization: The RD metaphor can also
be used to visualize the behavior of a software system [17].
In Fig. 3(b) the glyphs positioned above the base are method
calls of a trace. The position of the glyphs shows the starting
time of the call in the trace and their height is determined by
its duration.
3) RD Evolution Visualization: Also the third aspect of
software, evolution, can be visualized as a variant of the
(a) City metaphor (b) RD metaphor (c) Plant metaphor (d) Multisphere metaphor
Fig. 2. Selected metaphors supported by GE TAVIZ
(a) Structural view with RD metaphor: Cy-
clomatic complexity as height of methods
(b) Behavioral view with RD metaphor:
Call duration as height of methods
(c) Evolutionary view with RD
metaphor: Different versions of the
software system
Fig. 3. Selected variants of the RD metaphor supported by GE TAVIZ, visualizing different aspects of software systems
RD metaphor [17]. The base of the visualization in Fig. 3(c)
is a maximal layout. It shows all elements occurring in the
depicted evolution period of the software system. The size of
the glyphs in the base layout is determined by the maximal
size of the represented entities. The versions of the visualized
software system are positioned above the base layout. Only
the entities occurring in the respective version of the software
system are shown in the visualization. The size of the glyphs
of the versions is determined by the size of the element in the
depicted version.
B. Implementation
We extend the earlier work of M¨
uller et al. [18], who intro-
duced the generative and model-driven approach to cope with
the presented requirements when creating highly customized
and ready-to-use software visualizations. By separating the
visualization process into multiple model transformations a
high variability of platforms can be implemented for different
metaphors. The generator is based on this implementation
approach.
The generation process consists of three steps, each one
takes a model as input and outputs the transformed model. The
implementation is based on the Eclipse Modeling Framework,
especially Xtext2and Xtend3. The Xtext framework is used to
describe the meta-models. Further, the framework is capable
of parsing text files that follow these meta-models and create
a model from it. At the moment, parsing the input files is
the bottleneck with regard to scalability. The size of the
model is limited as Xtext loads the complete model into RAM
at once, similar to a DOM parser. Currently, we are able
to visualize the Apache Spring Framework which consists
of approximately 2 million lines of code (LOC), but not
much larger models. Once this step is completed, Xtend is
used to perform the model transformations. Xtend is a JVM
programming language and especially designed for this use-
case. Overall there are three transformation steps that are
described in the following.
2https://eclipse.org/Xtext/
3https://eclipse.org/xtend/
(a) City floor variant: methods as
floors, attributes on top of the build-
ings
(b) City panel variant: methods as
cuboids, attributes as cylinders
(c) City bricks variant: methods and
attributes as bricks of the buildings
Fig. 4. Selected variants of the City metaphor supported by GE TAVIZ
1) Source to Model: In this step a metaphor-specific model
is created from the source model, e.g., the FAMIX model.
Thereby the mapping between code entities and visual entities
takes place. For instance, for each class in the source model
a disk in the RD model is created.
2) Model to Model: The second step enriches the metaphor-
specific model from the first step with further information. For
instance, layout information cannot be calculated during the
first step because information about the complete model is
required to calculate positions. The resulting model contains
every information to visualize the software system.
3) Model to Target: Within the last step the metaphor-
specific model from the previous step is used to generate the
final output. Every entity of the metaphor-specific model is
transformed into an X3D or X3DOM representation.
V. US ER INTER FACE
To support the different use cases of end user application,
conducting studies, and testing features for every generated
visualization a highly configurable and generic browser-based
user interface has been created. It is built upon the output of
the software visualization generator, the visualization itself as
well as a JSON file containing additional meta-data, e.g., fully
qualified names, inheritance relationships, and call hierarchies.
This information is metaphor independent and is matched to
visual entities by a unique ID. It is mainly used by the user
interface to provide more information about the underlying
software system. GE TAVIZ uses X3D and X3DOM as the
main rendering platforms, because they provide a variety of
features fitting the visualization of software [19]. X3D is a
standard defined by the Web3D Consortium to facilitate the
creation and exchange of 3D scenes [20]. It defines an XML
exchange format which makes it a good choice as target
platform. Further, X3D supports different kinds of multimedia
features, such as video, audio, and animation. X3DOM was
developed to integrate X3D into HTML and thereby achieves
an easier use of web applications [21]. For these reasons, the
user interface of our toolset uses X3DOM instead of X3D.
It enhances the standard X3DOM interaction techniques, e.g.,
with new navigation types, tooltips with details, connectors for
relations, a package explorer, and a source code viewer.
The user interface is based on the model-view-controller
pattern and a publish-subscribe event model for a loose
coupling between the actions of the user and the manipulation
of the visualization. Every controller implements at least one
interaction technique like highlighting a selected element and
publishes or consumes corresponding events.
A setup configuration file defines the layout, the collection
of controllers, and parameters of one specific user interface.
Thus, a high number of user interface variants can be defined
for every use case in an easy and efficient way, considering
visualization specific optimizations for the interaction as well
as developing and integrating new interaction techniques or
enhancing existing ones.
VI. EVALUATION SERVE R
In order to provide optimized visualizations for different
aspects a deeper understanding of the pros and cons of the
software visualizations is indispensable. To gain this knowl-
edge evaluations with completely new visualizations and with
different variants of the existing visualizations are necessary.
Therefore, GE TAVIZ contains an evaluation server which uses
the presented user interface [17]. However, the coupling be-
tween both components is loose, the evaluation server can
easily integrate any other browser-based user interface.
The evaluation server is a Rails application built to facilitate
the planning and conduction of empirical evaluations with
software visualizations. An administrative interface allows the
evaluator to define and arrange the different parts of the
evaluation such as questionnaires as well as visualizations and
the respective tasks. Each evaluation includes a unique id hash,
making it possible to share the link to the evaluation with
the participants. It is also possible to provide each participant
a unique link to the experiment. The participants are guided
automatically through the evaluation process. Each page of
the process is viewable only once by each participant. The
times needed to answer the questions as well as the answers
themselves are persisted. When visualizations are displayed
further parameters are measured like the number of clicks and
the time the mouse is held down by the participant.
The evaluation server can be used in two different modes
of operation. It is possible to deploy the server to a web
server, so that the participants can do the evaluation remotely.
This makes it possible to reach more participants. A drawback
of this approach is the limited control over the participants.
The other option is to run the server on a local computer.
Conducting the evaluation this way increases the control over
the environment of the participants. The advantage of the
evaluation server in this setting is the automated guidance
of the participants through the experiment, which reduces the
influence of the evaluator on the participants.
VII. REL ATED WORK
GETAVIZ provides structural, behavioral, and evolutionary
views of object-oriented and procedural software systems in
2D and 3D. There are several software visualization tools with
a similar broad focus including sv3D [22] and cv3D [23],
Vizz3D [24], Codemap [25], SArF Map [26], Verso [27], and
BlendedCity [28]. However, the visualization of all aforemen-
tioned tools cannot be explored in a web browser and they
provide different user interfaces and interaction techniques,
which makes it impossible to evaluate these visualizations
under identical conditions.
EvalBench is a flexible Java library for information vi-
sualization evaluation [29] and provides similar features as
the evaluation server of GETAVIZ . The user can define the
evaluation process, including tasks and questionnaires and it
can be be combined with third-party visualization prototypes.
However, EvalBench lacks any remote capabilities and is
limited to Java-based user interfaces. Further evaluation solu-
tions such as Hierarchical Visualisation Testing Environment
(HVTE) [30], and Touchstone [31] have a different focus.
These tools support complex and ramified evaluation processes
and provide an integrated analysis of the evaluation, which
GETAVIZ does not. But, they are designed for simple 2D
visualization with minimal user interfaces, which is not suited
for software visualization.
Consequently, we have started developing GETAVIZ includ-
ing the user interface and the evaluation server for covering the
complete process from generating visualizations to rendering
them in a web browser up to conducting empirical evaluations
for any visualization metaphor.
VIII. CONCLUSION AND FUTURE WOR K
We have presented the functionality and the implementation
concept of GETAVI Z, a toolset to support the complete design
and evaluation process of software visualizations. We have
successfully applied the generative and model-driven approach
in order to support multiple visualization metaphors, metaphor
variants, programming languages, version control systems, and
output formats. To overcome the mentioned scalability issue
we plan to replace Xtext by a graph database, which will allow
us to visualize systems with several million LOC.
Currently, GETAVIZ is designed for researchers. To be suited
for end-users as well, especially the user interface of the
generator needs to be improved. Specifying the visualization
needs to be easier, e.g., through an task-oriented user interface,
where the user selects the task he or she has to solve and the
optimal visualization is chosen automatically.
We already conducted several quantitative and qualitative
studies that are based entirely or partly on the presented
toolset [6], [17], [32]. GE TAVI Z is developed continuously. In
the future, we intend to cover more programming languages,
e.g., Python and JavaScript, and visualization metaphors. On
this basis, we plan to conduct several experiment series to
increase and improve the empirically gained data in software
visualization to make software visualization more attractive
for practitioners.
REFERENCES
[1] S. P. Reiss, “The paradox of software visualization,” in 3rd IEEE
International Workshop on Visualizing Software for Understanding and
Analysis, 2005, pp. 1–5.
[2] L. Merino, M. Ghafari, and O. Nierstrasz, “Towards Actionable Visual-
isation in Software Development,” in 4th IEEE Work. Conf. Softw. Vis.,
2016, pp. 61–70.
[3] R. M¨
uller, P. Kovacs, J. Schilbach, U. W. Eisenecker, D. Zeckzer, and
G. Scheuermann, “A structured approach for conducting a series of
controlled experiments in software visualization,” in International Con-
ference on Information Visualization Theory and Applications (IVAPP),
Jan 2014, pp. 204–209.
[4] T. Munzner, “A nested model for visualization design and validation,”
IEEE Transactions on Visualization and Computer Graphics, vol. 15,
no. 6, pp. 921–928, Nov. 2009.
[5] M. Meyer, M. Sedlmair, and T. Munzner, “The four-level nested model
revisited: Blocks and guidelines,” in Proceedings of the 2012 BELIV
Workshop: Beyond Time and Errors - Novel Evaluation Methods for
Visualization, ser. BELIV ’12. New York, NY, USA: ACM, 2012, pp.
11:1–11:6.
[6] D. Baum, “Introducing Aesthetics to Software Visualization,” in WSCG,
vol. 23, 2015, p. 9.
[7] S. Ducasse, N. Anquetil, M. U. Bhatti, A. C. Hora, J. Laval, and
T. Girba, “MSE and FAMIX 3.0: an Interexchange Format and Source
Code Model Family,” HAL, Tech. Rep., nov 2011.
[8] O. Greevy, “Dynamix - a meta-model to support feature-centric
analysis,” in 1st Int. Work. FAMIX Moose Reengineering, 2007.
[9] S. Ducasse, T. Gˆ
ırba, and J.-M. Favre, “Modeling software evolution
by treating history as a first class entity,” in Work. Softw. Evol. Through
Transform., 2004, pp. 75–86.
[10] S. Diehl, Software Visualization: Visualizing the Structure, Behaviour,
and Evolution of Software. Springer, 2007.
[11] T. Munzner, Process and Pitfalls in Writing Information Visualization
Research Papers. Springer Berlin Heidelberg, 2008, pp. 134–153.
[12] A. R. Teyseyre and M. R. Campo, “An overview of 3D software
visualization.” IEEE transactions on visualization and computer
graphics, vol. 15, no. 1, pp. 87–105, 2008.
[13] C. Hansen, S. Miksch, K. Mueller, B. Preim, and C. Ware, “2D VS 3D
IEEE VIS 2014 Panel Proposal,” 2014.
[14] R. Wettel and M. Lanza, “Program Comprehension through Software
Habitability,” in ICPC 2007. IEEE CS Press, 2007, pp. 231–240.
[15] T. Munzner, Visualization analysis & design. CRC Press, 2014.
[16] R. M¨
uller and D. Zeckzer, “The recursive disk metaphor – a glyph-based
approach for software visualization,” in Proceedings of the 6th Interna-
tional Conference on Information Visualization Theory and Applications
(IVAPP ’15). Set´
ubal: SciTePress, 2015, pp. 171–176.
[17] J. Schilbach, “Analyse, Erzeugung und Evaluation animierter Soft-
warevisualisierungen,” Dissertation, Universit¨
at Leipzig, Leipzig, 2017,
submitted.
[18] R. M ¨
uller, P. Kovacs, J. Schilbach, and U. Eisenecker, “Generative
Software Visualization: Automatic Generation of User-Specific Visual-
izations,” in Proc. Int. Work. Digit. Eng., Magdeburg, Germany, 2011,
pp. 45–49.
[19] C. Anslow, S. Marshall, J. Noble, and R. Biddle, “Evaluating X3D for
use in software visualization,” Proceedings of the 3rd ACM symposium
on Software visuallization, pp. 161–162, 2006.
[20] Web3D Consortium, “Information Technology – Computer Graphics
and Image Processing – Extensible 3D (X3D),” 2008.
[21] J. Behr, P. Eschler, Y. Jung, and M. Z ¨
ollner, “X3DOM,” in Proceedings
of the 14th International Conference on 3D Web Technology (Web3D
’09). New York: ACM Press, jun 2009, p. 127.
[22] A. Marcus, L. Feng, and J. I. Maletic, “Comprehension of software
analysis data using 3D visualization,” in 11th Int. Work. Progr. Compr.
IEEE Computer Society, 2003, p. 105.
[23] X. Xie, D. Poshyvanyk, and A. Marcus, “Visualization of CVS
Repository Information,” in 3th Work. Conf. Reverse Eng. Ieee, 2006,
pp. 231–242.
[24] T. Panas, R. Lincke, and W. L ¨
owe, “Online-configuration of software
visualizations with Vizz3D,” in Proc. 2005 ACM Symp. Softw. Vis.
New York, New York, USA: ACM Press, may 2005, pp. 173–182.
[25] A. Kuhn, D. Erni, and O. Nierstrasz, “Embedding spatial software
visualization in the IDE: an exploratory study,” in Proc. 5th Int. Symp.
Softw. Vis. New York, USA: ACM, oct 2010, pp. 113–122.
[26] K. Kobayashi, M. Kamimura, K. Yano, K. Kato, and A. Matsuo,
“SArF map: Visualizing software architecture from feature and layer
viewpoints,” in 21st Int. Conf. Progr. Compr., 2013, pp. 43–52.
[27] O. Benomar, H. Sahraoui, and P. Poulin, “Visualizing Software Dynam-
icities with Heat Maps,” in 1st IEEE Work. Conf. Softw. Vis., 2013.
[28] T. Dal Sasso, R. Minelli, A. Mocci, and M. Lanza, “Blended, not stirred:
Multi-concern visualization of large software systems,” in 3rd IEEE
Work. Conf. Softw. Vis., 2015, pp. 106–115.
[29] W. Aigner, S. Hoffmann, and A. Rind, “EvalBench: A Software Library
for Visualization Evaluation,” in Eurographics Conference on Visualiza-
tion (EuroVis), vol. 32, no. 3, 2013.
[30] K. Andrews, W. Putz, and A. Nussbaumer, “The Hierarchical Visuali-
sation System (HVS),” in Proc. Int. Conf. Information Visu- alization,
2007.
[31] W. E. Mackay, C. Appert, M. Beaudouin-lafon, O. Chapuis, Y. Du,
J.-d. Fekete, and Y. Guiard, “Touchstone: Exploratory Design of Exper-
iments,” in SIGCHI Conf. Human Factors in Computing Systems, 2007.
[32] R. M¨
uller, P. Kovacs, J. Schilbach, and D. Zeckzer, “How to Master
Challenges in Experimental Evaluation of 2D versus 3D Software
Visualizations,” in 2014 IEEE VIS International Workshop on 3DVis:
Does 3D really make sense for Data Visualization?, IEEE. Paris,
France: IEEE, 2014.