PreprintPDF Available

Printmaking, Puzzles, and Studio Closets: Using Artistic Metaphors to Reimagine the User Interface for Designing Immersive Visualizations

Authors:
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

We, as a society, need artists to help us interpret and explain science, but what does an artist's studio look like when today's science is built upon the language of large, increasingly complex data? This paper presents a data visualization design interface that lifts the barriers for artists to engage with actively studied, 3D multivariate datasets. To accomplish this, the interface must weave together the need for creative artistic processes and the challenging constraints of real-time, data-driven 3D computer graphics. The result is an interface for a technical process, but technical in the way artistic printmaking is technical, not in the sense of computer scripting and programming. Using metaphor, computer graphics algorithms and shader program parameters are reimagined as tools in an artist's printmaking studio. These artistic metaphors and language are merged with a puzzle-piece approach to visual programming and matching iconography. Finally, artists access the interface using a web browser, making it possible to design immersive multivariate data visualizations that can be displayed in VR and AR environments using familiar drawing tablets and touch screens. We report on insights from the interdisciplinary design of the interface and early feedback from artists.
Content may be subject to copyright.
Printmaking, Puzzles, and Studio Closets: Using Artistic Metaphors
to Reimagine the User Interface for Designing Immersive
Visualizations
Bridger Herman, Francesca Samsel, Annie Bares, Seth Johnson Student Member, IEEE,
Greg Abram, and Daniel F. Keefe, Senior Member, IEEE
Fig. 1. Here we show the progression from the artist’s studio to a multivariate visualization. Left to right: collages from Samsel’s studio;
the visualization design interface; a simulation of five water masses underneath the Ronne-Filchner Ice Sheet in western Antarctica.
The data is part of the DOE, BER, E3SM, developed and run by the Climate Ocean Sea-Ice Modeling group at Los Alamos National
Laboratory.
Abstract
— We, as a society, need artists to help us interpret and explain science, but what does an artist’s studio look like when
today’s science is built upon the language of large, increasingly complex data? This paper presents a data visualization design interface
that lifts the barriers for artists to engage with actively studied, 3D multivariate datasets. To accomplish this, the interface must weave
together the need for creative artistic processes and the challenging constraints of real-time, data-driven 3D computer graphics. The
result is an interface for a technical process, but technical in the way artistic printmaking is technical, not in the sense of computer
scripting and programming. Using metaphor, computer graphics algorithms and shader program parameters are reimagined as tools in
an artist’s printmaking studio. These artistic metaphors and language are merged with a puzzle-piece approach to visual programming
and matching iconography. Finally, artists access the interface using a web browser, making it possible to design immersive multivariate
data visualizations that can be displayed in VR and AR environments using familiar drawing tablets and touch screens. We report on
insights from the interdisciplinary design of the interface and early feedback from artists.
Index Terms—Art and visualization, user interfaces, data visualization, visualization design
1 INTRODUCTION
The arts and humanities are crucial in formulating, interpreting, and
expressing challenging problems and ideas, including those that are
the subject of scientific inquiry like climate science, public health,
ethical engagement with technology and more. However, our current
“data-intensive paradigm” [11] makes it increasingly difficult for artists,
humanists, and others to engage deeply with such problems and ideas
since engaging with the underlying multidimensional data increasingly
requires a core background in data science or computing.
Our team has formed a design collective, the Sculpting Visualizations
Bridger Herman, Seth Johnson, and Daniel F. Keefe are with the Dept. of
Computer Science and Engineering, University of Minnesota Twin Cities,
Minneapolis, MN, United States. Emails: {herma582, joh08230,
dfk}@umn.edu.
Annie Bares is with the University of Texas at Austin, Austin, TX, United
States. Email: abares@utexas.edu.
Francesca Samsel and Greg Abram are with the Texas Advanced Computing
Center, University of Texas at Austin, Austin, TX, United States. Emails:
{fsamsel, gda}@tacc.utexas.edu.
Manuscript received xx xxx. 201x; accepted xx xxx. 201x. Date of Publication
xx xxx. 201x; date of current version xx xxx. 201x. For information on
obtaining reprints of this article, please send e-mail to: reprints@ieee.org.
Digital Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx
Collective. The ethos of the Collective derives from a desire to create
more enchanting visualizations, as well as improving the data-intensive
tools these visualizations are built on. Our team does this by bringing
the knowledge and experience of artists and designers into all facets of
the visualization process, from the theory behind a visualization tool to
its application, design, and the final products that it enables.
In this paper, we discuss the specific challenge of designing a user
interface for artists to create visualizations of actively studied, modern
3D datasets. For those who wish to engage with the big, data-driven
questions of our day, the vision is to welcome that engagement in
multiple modes, both as part of the artists’ own art practice and/or as
part of interdisciplinary collaborative efforts to better understand and
communicate science. We want the artists working in this space to be
able to work as artists. The design user interface elegantly mirrors the
ethos of our Collective by drawing from the tradition of printmaking.
In doing so, it creates a more intuitive, enchanting way for artists to
easily and quickly iterate through many visual possibilities derived
from their work. The artist-centered design process that the interface
enables allows for artists to create visualizations that enchant users with
their surprising, yet intuitive design choices that spark imagination and
curiosity.
Our design approach is grounded in an embrace of traditional, phys-
ical art processes. We ask, what if we could make the process of
providing the technical specification for a data visualization feel like it
fits within the artist’s studio, something that is synergistic with creative
visual processes based on experimentation and “working on the whole”
arXiv:2010.08859v1 [cs.HC] 17 Oct 2020
rather than the style of “linear thinking” and “efficiency” that is often
more closely associated with computing and big data?
Our technical approach makes use of the Artifact-Based Rendering
(ABR) technique presented in last year’s IEEE VIS technical track [15].
That technique makes it possible to render 3D visualizations using
visual elements created by digitally scanning real-world artifacts (e.g.,
2D paintings, drawings, ink washes, and prints; 3D clay, shaved wax,
arranged objects). The software places these visual building blocks in
the 3D visual space of the data and morphs, recolors, and otherwise
adapts them in response to data. For example, working with this
technique, artists can populate a virtual sea with hand-sculpted clay
glyphs, coloring each one according to the ocean temperature, to help
clarify for scientists five types of water masses critical to understanding
ice sheet melt rates (Figure 1). Artists have already used ABR to
produce visualizations that exhibit a unique, decidedly hand-crafted
style [31]. However, until now this process has required a programmer
to act as a guide, helping to set up appropriate links from data objects to
computer graphics render objects and to interpret the visual meaning of
various parameters exposed by the computer graphics shader programs.
Our technical goal is to create the interface needed to generate such a
complete and correct rendering specification that the ABR engine can
use to drive its computer graphics shader programs.
The interface to weave together the creative artistic process and
the technical data-driven 3D rendering engine was designed and im-
plemented over the course of approximately 8 months by an interdis-
ciplinary collaborative team (the co-authors) who collectively bring
expertise from the disciplines of visual art, environmental humani-
ties, and computer science. The team is separated by distance, and
it helped us to kick off the design project with an in-person “hunker
week,” which was then followed up with months of implementation and
iterative refinement supported by two regular team video conferences
per week. Our work began by understanding related work and then
moved to building a common language, brainstorming, implementing,
and iteratively refining the interface.
2 RE LATE D WORK
Our work is inspired by the long tradition of artists engaging with and
interpreting scientific data, and it contributes to literature in the inter-
disciplinary research areas of visual design for science and creativity
support tools.
2.1 Artists Engaging with Scientific Data
From Leonardo da Vinci to the present VISAP program, there is a
long history of artists engaging with scientific concepts and data. The
examples pictured in this paper deal specifically with climate science, a
common theme in contemporary art-sci-tech work. Artists who provide
a human context to environmental issues are widespread. InfoWhelm,
a recent publication by Houser, surveys contemporary art and literature
addressing climate and the environment [13]. Pioneered by artists
including Polli, Anadol, Miebach, West, Vensa, Viegas and Wattenburg,
just to name a few, the field has spurred university programs across the
United States, including the University of New Mexico, the University
of Oregon, Arizona State University, and UCLA [21, 25, 29,42, 45].
Other artists working directly from actively studied scientific data have
created site-specific interactive and sculptural installations [38, 39],
translated topographic data to ice sculptures of glaciers that make the
viewer gasp [35], and much more. Rather than striving to produce an
“unbiased” or “sterile” account of scientific data, artists often help us
understand and interpret the human connection to the data, creating
data visualizations that can successfully interweave both data and
emotion [41, 43].
2.2 Visual Design for Science
Artists also have a rich history of contributing to the field of data visual-
ization specifically, teaming with scientists and other data stakeholders
to present data more clearly. Cox outlined what artists can contribute
to science, specifically highlighting the powerful difference that artist-
designed colormaps can make as well as artists’ usefulness in depicting
high-dimensional spaces [10]. Many visualization techniques have
drawn inspiration from art (e.g., use of color [20], stippling and line
rendering [4, 18], narrative forms [5]). Several guidelines that improve
visualization clarity, engagement, and impact come directly from design
theory and principles [8,44]. Numerous programs [26], dashboards and
entire companies have been created to enable this effort. Most closely
related to our work, powerful visualization tools, systems, and user
interfaces have been designed specifically to support the role of artists
in data visualization, notably [6, 12, 16, 26, 33, 34].
Since it is built upon Artifact-Based Rendering (ABR) [15], our
interface supports the visualization of 3D multivariate scientific datasets
using artist-created media such as glyphs, colormaps, lines, and textures.
This is a defining aspect of our approach, since this technique is already
so closely tied to traditional artistic practice and leverages real-world
artistic skill. The interface also includes features found useful in other
digital visualization interfaces for artists. For example, we follow an
approach similar to ColorMoves [32] to support building, editing, and
tweaking custom dataset-specific colormaps in real time.
In general, the interface supports a mode of working that is fast,
iterative, and visual. Interfaces to powerful 3D visualization packages,
such as ParaView [3] and VisIT [9], are primarily designed for scientists
and technologists. The workflows they provide elegantly succeed at
defining a highly adjustable logical progression starting from data,
moving through a processing pipeline, and finally rendering an image.
However, the approach does not necessarily align with the creative
workflows of visual artists and designers. Instead, we seek a workflow
that includes a similar level of control over appearance but is more in the
style of fluid, sketch-based user interfaces. Artists have previously used
sketch-based user interfaces to prototype 3D visualizations [16, 17],
create custom illustrations of 2D fluid flows [33], sketch free-form
glyphs [14, 34], and create multi-layered animated 2D visualizations of
a variety of datasets [34]. In our interface, sketching and working with
many other forms of physical media happen primarily outside of the
digital interface but, in the style of Buxton’s work with “sketching user
experiences” [7], working with the interface can feel like sketching
in the same way that it enables a similar rapid, fluid style of visual
exploration.
2.3 Creativity Support Tools
Tools that enable rapid visual experimentation like sketching are ex-
cellent examples of a broader category of user interfaces known as
creativity support tools [36]. Visual creativity is often aided by digital
sketch pads, tablets, and other interfaces that enable artists and design-
ers to work directly with their hands, leveraging real-world, physical
skills. Our approach is almost an extreme version of this, leveraging
artists’ physical real-world work by digitizing it. Some artists have
commented that this enables greater expression and personalization,
since, for all the success of digital tools, there is a reason artists con-
tinue to work with physical media. Creativity has also been linked to
user interfaces that involve visual exemplars [40] and fun, which is
often encouraged through subtle animations and metaphors [37]. Our
work contributes to the growing interdisciplinary literature on creativity
support tools with a new specific focus on artists in visualization.
3 TH E VIS UAL IZ ATIO N DESIGN INT ER FACE
We begin the discussion of the user interface design by reflecting on why
this is a hard problem to solve. There are many reasons for this, but one
of the most important is the disconnect between what artists perceive
as the visual object they wish to operate on and how the underlying
computer graphics software organizes its data objects and rendering
objects. These various objects often have complex relationships; so,
designing an interface that fits the artists’ cognitive process is not as
simple as making a diagram of the software organization and adding
buttons and sliders to control the parameters.
Stepping back during a design session to look at a data visualization,
an artist might think, “let’s see what happens if we change those glyphs
to show water temperature instead of salinity” (Figure 2).
Notice how, to the visual artist, this thought is organized conceptually
around making a change to a visual object. “Those glyphs” are perhaps
Fig. 2. “Let’s see what happens if we change those glyphs to show water
temperature instead of salinity.”
the most visually dominant aspect of the image, and the goal is to make
what seems like a small visual edit to the colormap.
From the standpoint of the software, this operation actually involves
three classes of data that need to be managed in concert: 1. Data
Variables–temperature and salinity are both data variables; we might
also call them data fields in this case because for any (x,y,z) point within
the bounds of the data, we can look up the water temperature at that
point. 2. A Dataset–both these variables belong to a set of data that are
somehow related, in this case spatially; we might say we are visualizing
“the Gulf of Mexico dataset.” 3. Data Objects–“those glyphs” are yet
another class of data. They exist within the same dataset, but they
are not exactly data variables nor are they data fields. We could say
that they are derived from data field(s) in the sense that the glyphs
are generated by sampling into another variable of the data field such
as phytoplankton concentration or the nitrate concentration. In the
software, these are known by the rather nondescript term, “data object.
Generally, we cannot picture data variables directly, but we can use
them to modify a data object. For example, we might modify the glyph
size or vary the color in response to values of a data variable from the
same dataset. This leads to rules, such as: 1) data variables and data
objects both belong to datasets, and 2) you can’t draw a data variable
directly, but you can draw a data object and then modify some of its
properties based on a data variable, as long as they belong to the same
dataset. During our “hunker week,” it took our team multiple days
of deep discussion to believe we were finally on the same page with
the nuanced relationships between data variables, data objects, and
datasets, which aspects of these are precalculated and available to the
user and which are not, and where data are stored – inside a dataset, in
a data object, or both. We therefore realized a key design goal of the
interface should be to naturally encode such rules and avoid the need
for multi-day user training sessions.
The following request highlights another challenge, “Now, change
these flow lines so that they are drawn using this long thin clay form I
sculpted yesterday rather than the current textured ribbons” (Figure 3).
Again, from the artist’s standpoint, this request is completely natural.
It operates on the visual element of the “flow lines.” On the surface, it
does not ask to change that object in any fundamental way–they are still
the same lines. The request is just to change the visual representation.
Unfortunately, to the computer graphics programmer, this must be inter-
preted as a big change because the efficient approach to drawing these
lines is completely different in the two cases. The differences, even
requiring different triangle mesh geometries and different shader pro-
grams, are significant enough that most graphics programmers would
take the approach, “OK, if you want to make that change, then I will
just delete the old ribbon render objects I created for the lines earlier
and start over, making a new set of lines that use my instanced mesh
shader program to create something that looks visually like a line but
is really many copies of your clay mesh.” Creating a user interface
that embraces the language and creative visual design processes of
Fig. 3. “Now, change the flow lines so that they are drawn using this
long thin clay form I sculpted yesterday rather than the current textured
ribbons.
1
DATA SOURCES VIS COMPOSITION DESIGN LIBRARY
2
GULF OF MEXICO
FIRST H2O IN UNIVERSE
WAVELET NAMES
SKETCHY” STREAMLINES
“PRESSURE BUBBLES”
SEARCH
VOLUME RENDERING
RIBBON
LINE
TUBE
LINE
SURFACE WITH BLENDED TEXTURE
SURFACE WITH STAMPED TEXTURE
EDIT VARIABLES
VELOCITY MAG (S)
VELOCITY (V)
PRESSURE (S)
GLYPH SET
COLOR
ORIENTATION
ASPECT RATIO
min
max
GLYPHS IN A VOLUME
PIPE VOLUME
[ 0 ]
PIPE SURF
PIPE INSIDE STREAMLINES
KEY DATA
VARIA BLES
VELOCITY MAG (S)
VELOCITY (V)
PRESSURE (S)
GLYPHS
ON A LINE
STREAMLINES
LINE TEXTURE
LINE COLOR
PRESSURE 1.45 56.43
PIPE INSIDE GLYPHS IN
A VOLUME
Fig. 4. Digital mockup of the three-panel interface. The center Compo-
sition Panel is where artists link together data, stored on the left panel,
and visual elements, stored on the right.
artists while also preserving the full power of the computer graphics
and efficiencies needed to render large datasets in VR is a serious
challenge.
3.1 The Language of Printmaking
One of the joyful struggles of interdisciplinary collaboration is breaking
down the traditions, processes, and assumptions of our various disci-
plines in order to find the intersection points that we know are there but
can be so difficult to articulate clearly. We often use metaphor to assist
with building a common language in such conversations. So, during
the “hunker week” that kicked off our interface development effort, we
searched for metaphors that might help to translate the most complex
requirements of the 3D graphics data rendering engine into a language
that fits with artistic traditions.
Printmaking emerged as one of the most useful metaphors. In in-
taglio printmaking, artists first create a design by carving or etching
into a matrix, such as a metal plate. Ink is applied to the plate, filling
in the recessed design. Then, the print is pulled (the design is trans-
ferred to paper) by running the paper and plate through a press. A
single transfer in this style is called an impression, but prints routinely
combine several layers of impressions from multiple different plates.
An edition of identical prints can be produced by repeating the process
with the same inks, plates, and ordering, or, an infinite variety of new
prints can be created by combining plates in new ways, inking them
differently, or even adjusting pressure on the press. There are many ad-
ditional variations to the process including that wood, metal, stone, and
linoleum can all be used as matrices. However, one amusing constant
seems to be that if you ask a printmaker, “do you have any old plates in
your studio?” You will often get a smile and the answer, “yes, how did
you know my closet is absolutely overflowing with metal plates, wood
blocks, and ink!”
There are some useful connections between printmaking and data
visualization. Like a library of reusable computer graphics algorithms
that can draw points, lines, or surfaces in different colors and locations
based on the data sent to them, that closet full of plates provides a
reusable collection of design elements that can be reinked and rear-
ranged on the page in countless ways. Waiting in the closet until they
are needed, these plates (rendering algorithms) are only brought to life
when they are loaded with ink (data). It is true that printmaking notably
departs from 3D computer graphics in that the result is typically a 2D
image; however, printmaking is also the one traditional, physical art
form where the concept of building up a complete composition from
a series of separate layers is absolutely obvious. In fact, the technical
steps required to set up and pull each layer can be quite complex and
time consuming. So, many printmakers naturally think in terms of
layers and procedures that are quite reminiscent of the way multiple
data objects (streamlines, surfaces, points) are sent in serial through
the computer graphics pipeline and ultimately superimposed to form a
complete 3D visualization.
3.2 Early Ideation and Sketching
The concept of reusable plates that can be inked with data forms the
foundation for the interface. When combined with data, each plate
forms a “data impression,” a visual layer that becomes part of the
overall composition.
Another high-level concept is the notion of data and visual art/design
meeting in the middle to create a visualization. Figure 4 shows an early
design mockup exploring this concept. Notice the 3-panel design with
the “Data Sources” on the left, the “Design Library” on the right, and
the “Composition” panel in the middle. Adding data impressions to the
composition requires bringing together elements from both the left and
the right. Most traditional 3D data visualization pipelines start from the
data and proceed in a linear fashion through a pipeline until an image
is rendered at the end. In an effort to better support design that places a
priority of visual decisions, this interface makes it possible to start with
the design, pulling colors, forms, and textures into the composition
before linking them with data.
The links between data and visuals could be made by drawing lines
between input and output ports to form a pipeline in the style of Data
Explorer [2], but we designed an alternative inspired by the block-based
visual programming tools often used to teach programming [22, 27].
This puzzle piece metaphor has the great advantage of visually encoding
the difficult-to-explain rules mentioned in the earlier discussion of “why
designing this interface is a hard problem.”
The puzzle connectors are designed to be abstract enough to work as
interface elements, but specific enough to evoke the sense of the item
they are representing. For example, the point data connector looks like
bubbles coming out of the puzzle piece, and the colormap connector
looks like a painter’s palette. Also worth mentioning are the action
icons located throughout the interface which use the Material Design
icon pack [1], which is a popular choice among smartphone and web
apps alike.
3.3 Data Impressions
The complete, current visual language of plates, data, and visual ele-
ments is shown in Figure 5. Let us describe how artists work with these
building blocks to design a visualization composed of multiple data
impressions.
There are several ways to create a data impression, but a typical
approach starts with selecting a plate from the design palette in the
right panel, also known as the “studio closet.” This collection of plates
is expandable. Whenever a new data-driven rendering algorithm is
added to ABR, we add a new plate type. The pattern on each plate is
an example of the visual style it can create, but this is just one example.
The plate will produce something different depending on how it is
inked.
Since the rendering is 3D, the plate needs to know where its pattern
should be applied in space. In the underlying technical system, this
information comes from the “data object” discussed earlier, which is, in
our experience, one of the most challenging technical aspects to explain.
Our interface addresses this by, again, using metaphor. Printmakers
are familiar with the need to align or register a pattern, and they often
use registration or “key” plates to accomplish this. Thus, the interface
presents data objects as “key” data.
Key data are necessary. There is no way to draw data variables
without providing some spatial registration, so every plate includes at
least one puzzle-piece slot for key data. Notice, also, that the slot types
vary based on the different icons for registering the plate’s pattern to
3D locations in space (points), 3D curves (lines), and forms (surfaces).
Figure 5 shows just the plates available in our current implementation.
Going forward, we expect our team will always include computer
graphics researchers and artists working together to develop new plates
and corresponding graphics algorithms.
A specific example of a “Leafy Chlorophyll” data impression de-
signed by an artist is shown in Figure 5. As we see from the shape of
the puzzle piece, this plate only works when registered to point-style
key data. In the Gulf of Mexico Biogeochemistry dataset pictured
in several figures of this paper, there are multiple options associated
with this type of key data, including “Chlorophyll Concentration” and
“Nitrate Concentration.” In this case, the artist has registered the plate to
“Chlorophyll Concentration.” We often use “Concentration” to signify
a density-based sampling, which means the pattern will be sparse in
areas of low concentration and dense in areas of high concentration.
Beyond the spatial registration provided by key data, all of the plates
we have created to date also include additional settings to further cus-
tomize the data impression. The interface presents these in a collapsible
list attached to the bottom of each plate. When a plate is placed in the
center Composition Panel, the list automatically opens up to show all
available options, as shown in the “Leafy Chlorophyll” example.
All of the settings in this list are optional, and we require all plates
to have reasonable defaults. So, as soon as the plate is registered with
key data, a 3D visual will appear in ABR, which is usually set up to run
side-by-side on a second monitor and/or in an attached VR/AR headset.
This enables the artist to explore the visual results in real-time 3D and
react with visual changes. Several settings have been adjusted in the
“Leafy Chlorophyll” example by attaching data variable puzzle pieces
from the left data panel and visual design puzzle pieces from the right
design panel.
Returning to some of the technical challenges that must be overcome
in the interface design, note the separation in this interface between
key data and data variables. With the settings shown in Figure 5, the
“Leafy Chlorophyll” data impression produces a visualization like the
one in Figure 2 Temperature. Making the design change described in
that Before–After example is trivial. The artist must only replace the
“Salinity” puzzle piece in the “Color Variable” slot with the “Tempera-
ture” puzzle piece from the scalar variables section of the data palette
on the left panel of the interface.
Beyond this quick switch to a different data variable it is also possible
for the artist to decide that the way that pattern is distributed in space is
not working. Perhaps, the combination of leafy glyphs works perfectly
with the background color scheme and it makes sense to include these
somewhere in the composition, but the chlorophyll concentration data
happens to be very unevenly distributed and the leafy glyphs are all
clustering together in a way that is not very readable. It is possible in
this case to keep all of the existing data and visual settings the same
but swap in different key data, “Nitrate Concentration,” for example.
This will have the effect of retaining all of the plate’s visual style but
re-registering the pattern to a different 3D spatial distribution.
Let us consider the other Before–After picture mentioned earlier.
Recall that the design edit illustrated in Figure 3 is a drastic change
from the standpoint of the computer graphics algorithm that should
be used, even though artists think of it as a visual change to the same
set of lines. This example requires more manipulation of the interface
to achieve, but all of the underlying complexity is hidden behind the
metaphors and iconography. The Ribbons image was created using a
Fig. 5. (A and D): The iconography used in the interface, showing the types of Key Data, Data Variables, and Visual Elements available to the artist in
the interface. (B): Example of a glyph plate “Leafy Chlorophyll” which is registered with the “chlorophyll-points” density-based point sampling and has
the “Temperature” scalar data variable encoded with a colormap from the design palette, as well as a “drum” glyph. The dark gray slots indicate
entries that have not yet been assigned by the artist and remain at the default values shown in brackets. Other types of plates (ribbons and surfaces)
are also shown. (C): 20 examples of Visual Elements available to the artist in the interface, including Colormaps, Glyphs, Lines, and Textures.
textured ribbon plate where artists can provide several styles of texture,
including one used here to give the ribbons their patterned edge. The
ribbons are also colored according to data. To use a sculpted clay form
to depict the lines instead, the artist must first notice that the ribbon
plate does not include a slot for glyph key data–it is impossible to ink
this plate with glyphs because the underlying software approach to the
two styles of line is so different. The solution is to “go back to the
closet” and find a different plate that is more suited to glyphs. Once
they place this new plate in the composition panel, they can move all
the important repeated elements (key data, data variables, color) from
the original plate to the new one.
The composition panel holds all of the data impressions created for
the visualization. Artists often reposition these within the panel to
organize the space, and the panel itself can be panned if the artist has
more layers than they have screen space. Drawing inspiration from
digital image manipulation software, each data impression also includes
buttons to hide, collapse, expand, or delete the impression. When an
artist saves their work, the placement of each layer in the composition
panel persists between sessions.
3.4 Importing and Editing Visual Assets
Artists seldom work in a vacuum, and the interface embraces this
concept by enabling artists to incorporate any visual elements stored
in the public Sculpting Vis Library [30] in a way that feels magical.
The Sculpting Vis library is simply loaded in one web browser window
while the design interface runs in another. Then, any of the glyphs,
colormaps, and textures in the library can included by simply clicking
on their thumbnail images in the library window and dragging these
into the design interface browser window. This automatically adds them
to the current working palette (right panel) and triggers the connected
instance of ABR to download the original raw 3D model files, image
data, etc. so that these elements may be used for 3D computer graphics
rendering.
The library itself is rapidly expanding and currently contains a se-
lection of glyphs, colormaps, lines, and textures curated by artists in
the Sculpting Vis Collective. Individuals may also upload new visual
elements to the library via the ABR applets [15].
Rather than working to create generic, generalizable color maps,
glyphs, and textures, one of the benefits of this design interface is
that it makes it increasingly practical to create data-specific visualiza-
tions, that is, visualizations that include color maps, glyphs, and other
elements that are fine-tuned for the particular data at hand. To this
end, the interface includes a data-specific colormap editor inspired by
the powerful ColorMoves tool [32]. Double-clicking the color map’s
thumbnail icon in the interface launches the colormap editor, which
shows the colormap on top of the data histogram for the variable that
the colormap is attached to. This allows the artist to tune the colormap
relative to the actual data values. The interface includes features to add,
subtract, and adjust the colormap’s control points.
3.5 Implementation
To support a multitude of devices including desktop and laptop com-
puters, tablets, and large-format touch screen displays, the interface
is built for web browsers using a combination of Django, JavaScript,
jQuery UI, and other libraries like Data Driven Documents [47]. The
puzzle piece connections are implemented with snapping so that a con-
nection is made whenever a correct puzzle piece is dropped near, even
if not precisely on, a valid slot. If the slot is already occupied with
another piece, it is replaced, and if the piece does not match the slot,
the connection is refused. In this way, the implementation makes it
impossible to drop a “Glyph” visual element into a “Colormap” slot in
a data impression, and it is impossible to drop a scalar variable into a
vector variable slot.
To support diverse applications, the system is implemented using a
modular structure, where the web-based design interface, the ABR 3D
rendering engine, and the data engine are three distinct sub-systems that
connect to each other using network sockets. This means, for example,
that the data can be hosted on a supercomputer, the graphics can be
rendered on a machine with a powerful graphics card, and a designer
can craft a visualization on a tablet while interactively monitoring their
design modifications on a laptop with a remote viewer. This opens up
possibilities for more artists to become involved in the visualization
design process by drastically reducing the hardware requirements for
building visualizations of large scientific datasets.
We have extended ParaView [3] to act as a data server within this
framework. To create the figures shown in this paper, we used Par-
aView to prepare datasets from two supercomputer climate simulations
actively studied by our collaborators: Biogeochemistry in the Gulf of
Mexico [24,28, 46] and Sea Ice Climate data [23]. The design interface
does not support data wrangling itself; filtering data, deriving new data
variables, and similar operations can be done in ParaView before or
during a design session. This is one limitation in the sense that we do
not expect artists to have knowledge to do this level of data wrangling.
However, after data have been loaded into ParaView once and saved,
A B C
Fig. 6. Deborra Stewart-Pettengill works with master printmakers at Wingate Studio (A) to realize her chine-coll
´
e designs. She commonly works with
patterns, which have been digitized to form streamlines in the Gulf of Mexico visualization (B) and a texture on the land (C). Image (A) Copyright
2020 Wingate Studio; used with permission.
the design interface makes reloading these data for design work simple
through its Load and Save functionality.
The three modules are connected as follows: The geometric rep-
resentation of the data is sent from the data server (ParaView) to the
rendering engine (ABR), which pre-processes and optimizes the data
for rendering. The ABR engine connects to a Python server which de-
livers the interface to the artist. As the artist makes changes to the visual
styles on the interface, messages are sent via WebSocket to the engine,
which updates the encodings that are rendered accordingly. Graphics
from the engine can optionally be rendered to a depth texture [19], and
sent to any connected remote viewers. VR headsets can also be con-
nected directly to the engine, which renders the visualization graphics
at 120+ frames per second. The scale of the visualizations in VR is
initialized to a table-scale default, but can be adjusted via bimanual
interactions from the user. Graphics can also be easily exported as PNG
images, which is useful for artists using an ABR-created visualization
as a part of a transmedia piece.
4 INSIGHTS FROM ART IS TS
This section discusses the experience and insights from artists on our
team who have had a chance to work with the interface in detail as well
as feedback artists who have just started the process of designing visu-
alizations with the interface. We begin by describing design sessions
we facilitated with two practicing artists as an introduction to the tool.
We cast these two introduction / design sessions, simply, as a chance
to work creatively with a dataset scientists are currently using to un-
derstand the biogeochemistry of the Gulf of Mexico [46]. These data
include surface layers for the ocean and land, streamlines for the ocean
currents representing the direction of eddy curvature (clockwise or
anticlockwise), and concentrations for nitrates and chlorophyll. The
sessions were conducted remotely so as to respect the social distancing
necessitated by the ongoing pandemic. The software ran on a local
computer with the artists connecting remotely over a video conferenc-
ing link. In one case, that link supported having the artist take over
control of the local computer and use the interface with her own mouse.
In the other, we acted as the artist’s hands on the interface, sharing the
screen over video and following her directions.
4.1 Vis Design Session 1
The first artist, Deborra Stewart-Pettengill, works with many forms of
traditional artistic media and had no experience working with 3D scien-
tific data prior to this session. Her current art processes include print-
making with chine-coll
´
e, an intricate technique that involves adding
color and form to a print by using the press to apply shapes of thin
cutout paper to the print. She was interested in experimenting with her
abstracted natural and organic forms on a data visualization after she
was contacted by a member of the Sculpting Vis Collective.
After some introduction to the project and discussion to find common
ground, we realized we were all curious to see how the line quality she
and the master printmakers at Wingate Studio achieve with chine-coll
´
e
(Figure 6A) might translate into a digital data visualization. So, she
digitized some of this work using her smartphone camera, and we used
the ABR’s Infinite Line and Texture Mapper applets [15] to prepare the
results for attaching to data.
“I’d like to start with something neutral in the terrain,” Stewart-
Pettengill stated at the onset of our session; together, we assigned a
generic brown colormap and left it for the time being. Later, toward
the end of the session, she returned to this color; “the one thing I want
to change is the color of the land.” The colormap was modified to
tease out the most varied part of the elevation data – “That’s looking a
lot better, the peachiness of it looks good with the blue-violet water.”
Stewart-Pettengill was enthusiastic about including her own chine-coll
´
e
work – when she first saw the 8 chine-coll
´
e textures in her palette,
she immediately recognized them: “Oh yeah, there are my strokes!
Cool!” Later, she applied these to the streamlines depicting ocean
currents and experimented with adjusting line width and coloration
(Figure 6B,C). In fact, she repeatedly came back to the streamline
pattern and colormap throughout the process, particularly once she
began assigning encodings to the chlorophyll and nitrate glyph layers.
We will return to the theme of the importance of bringing one’s own
work from the studio into the new project of visualizing data. This
emerged as a consistent theme.
Another important lesson from the introductory session with Stewart-
Pettingill was the need for visual experimentation. She is “just so used
to experimenting with things” as she works, and reflected that the
interface enabled a similar process during her session. To provide a
sense of the scope of experimentation that is possible, we noticed that
in addition to the 8 line textures mentioned earlier, she also worked
with over 10 glyphs and more than 20 colormaps for the ocean current
streamlines, the nitrates, and the ocean floor during this first-ever design
session.
4.2 Vis Design Session 2
The second artist, Stephanie Zeller, had prior experience working with
3D scientific visualizations including custom colormap creation and
has worked with the Sculpting Vis Collective previously. Zeller is
trained in traditional media with a focus on painting. Her work of late
examines the viewer’s relationship with the digital environment through
traditional media, especially in terms of data consumption and analysis.
Zeller is also a freelance writer and uses a breadth of digital software
to augment her work in data journalism through both digital illustration
and information visualization.
Of her session, Zeller remarked “As I worked, I was coming first
from an artistic angle, closely followed by a scientific angle. I wanted
it to be both aesthetically appealing and to also maintain the ability to
distinguish between variables, which requires a fair amount of stepping
Fig. 7. The top image shows chine-coll
´
e etchings by Samsel, from a
series entitled Osmosis. Details from the etching are converted into
textures usable in ABR visualizations, shown in the bottom row alongside
textures from Stewart-Pettengill.
back and moving in again. She spent a short time experimenting with
colormaps for the ocean floor once a texture was applied, then moved
on to working on the terrain colormap. She clearly judged the terrain
as critical to the composition, spending more than 20 minutes to fine-
tune a brown/tan hued colormap to work well specifically for these
data. She later decided to take the design in a new direction, shifting
to a green/white colormap to achieve suitable contrast with the rest
of the visualization elements (particularly the streamlines) and to not
overpower the main subject, which is the visualization in the ocean.
The design process was clearly iterative, with each design decision
coming as a reaction to what was seen. At one point, Zeller decided that
there was “too much green” in the visualization, and swapped out the
old olive chlorophyll glyphs for a blue instead. Later, she returned to
the colormaps for the land and the streamlines, making fine adjustments
to achieve a balance between them.
Similar to Stewart-Pettengill, Zeller was also keen to incorporate her
own artwork in the visualization; in her case through color. During this
introductory session, she worked with more than 50 different colormaps,
editing 18 of them relative to the data histogram, and creating 2 more
from scratch. She felt that many of the colormaps given in the curated
library didn’t resonate with her style and created new ones adapted
from her own work in data journalism, which can be seen on the ocean
current streamlines (Figure 8).
In her own paintings, Zeller often builds very large surfaces (8 feet
or more in length), and works with details up close. Every so often, she
steps far enough back to see the entire piece and understand the color
relationships in the part she just worked on, and make an informed
next move in the part. Building visualizations with the interface was
appealing to her because she felt she could address design concerns
from up close and far away using her familiar processes. This was
reflected in her usage of the interface – when choosing colors and
glyphs, she started with detail-oriented view, quickly moved to a more
global perspective, then made changes in the visualization design based
on how the color interactions between variables both close up and far
away. Zeller felt that for similar reasons, it was intuitive to design for
both a close-up 3D perspective and a bird’s-eye 2D perspective using
the interface.
4.3 Interpretation and Extended Use
Francesca Samsel, an artist in the Sculpting Vis Collective is a co-
designer of the interface and has worked with it in various stages of
development to create works like those pictured in Figure 1 and 9,
which build so clearly upon her art practice, for example, the form, line
quality, metaphor, and color of the Osmosis series (Figure 7).
In this section, we report both on Samsel’s own observations working
with the interface and her reflections on the feedback received from
other artists. Together, these coalesce around three major themes.
Fig. 8. Stephanie Zeller’s process for designing a visualization of the
Gulf of Mexico. (1) shows an early step in the visualization, (2) includes
progress on the streamlines and terrain colormap, and (3) shows the
final result including several custom colormaps designed by Zeller.
4.3.1 Scope and fidelity of visual vocabulary
The visual variation exhibited by artists was significant because the
workflow presented by the interface allows each individual to contribute
their artistic vision to the visualization design. In many cases, this is
accomplished by enabling artists to utilize elements that they have
previously created, such as chine coll
´
e and custom colors discussed
earlier. Samsel shows how the same can be accomplished with hand-
sculpted clay forms to create custom glyph vocabularies (Figure 10).
Relative to typical scientific visualization software, the scope of visual
variation that is possible is much larger, similar to working in a studio.
So, subtle variations in visualization design can be explored to discover
combinations that work together.
Additionally, since the interface is focused on enabling an interactive
design process, artists can make minute changes to the visual inputs,
and often these have a profound effect on the resulting visualization.
Figure 12 demonstrates how an artist with the ability to fine tune the
color can apply color contrast principals to impact the visualization for
clarity.
We see it as a mark of a good interface that this fidelity to fine-tune
the visualization correlates with artistic skill. The ability to render
an object without thinking is the root of visual invention. This is the
means via which artists push their work/vocabulary forward. The idea
is to leave the critic outside the room and trust in the process. The key
to the arts is to gain the technical skill so that your hands flow freely,
then when you enter the studio to turn off the linear or rational decision
making process and follow the work itself as it guides you to the next
step.
4.3.2 Rapid iteration and stimulation of artistic imagination
The scope and fidelity of the visual vocabulary combined with the speed
at which one can explore visual alternatives seems to lead to a tool
that stimulates the artistic imagination. “This is fun!” – both of the
artists seeing the interface for the first time expressed enjoyment in the
process. Artists can get into the creative zone, that special mental place
where artistic magic happens, even while they are working with 3D
multivariate scientific datasets.
It is necessary that any tool living up to an artist’s specifications
must easily support iteration in order to facilitate work in the creative
zone. While Figures 10, 11, and 12 demonstrate the impact of shape
Fig. 9. This visualization of water masses underneath the Ronne-Filchner Ice Sheet shows: five ocean masses, their locations and temperature;
two directions of eddy flow; the ocean floor depth; and the topography of the Antarctic. Here we are illustrating the power of including artists in the
process. In order to render seven overlapping variables in a 3D simulation, the visualization uses distinguishable glyphs and hues. The visualization
shown here provides scientific value, as scientists have not previously seen the movement of these water masses which are critical to predicting the
melt rate of the ice on the underneath side of the Antarctic ice sheets. Data - E3SM, BER, DOE.
Fig. 10. Detail in the glyph selection process. The yellow glyphs in the left
column (A and C) compare a disk verses a spherical glyph. On the right,
the green glyphs compare a triangular shape (B) verse an elongated
form (D). While these are subtle shifts, they provide critical contrast when
working on large complex data.
and color on our ability to distinguish between variables, Figure 9
demonstrates that engagement is not at cross purposes with scientific
needs. The iterations shown here were accomplished in under a minute
by our experienced artist. It is this flexibility and rapid iteration that
Fig. 11. The includes the ability for artists to upload and or create new
colormaps via the Color Loom applet using the ABR technique. While
this provides limitless options, the artists felt strongly that they needed to
be able to control the distribution of the hues across the data in order to
create the contrast and visual distinction with in the imagery. Thus an
internal colormapping tool was added to the interface. On the right is the
color interface enabling on to control hue distributions.
enables artistic discovery. The key is to remove the barriers to achiev-
ing this creative state so that artists can use their artistic skills built
up over a lifetime of experience to add their voices and visions to the
multidisciplinary efforts to wrangle increasingly large and complex
scientific data. Science and art have a common thread in that many sci-
entific breakthroughs, like artistic breakthroughs, happen when intense
thought, contemplation, and experimentation meet the subconscious
and the accidental, underscoring the need for artistic expression in our
society and for science.
4.3.3 Limitations
We have described some feedback from artists on the occasion of
their first introduction to the interface, and it is important to note that
this is not a system to be mastered in a single session. The scope of
the vocabulary feels limitless, which is a good thing; artists are used
to starting with limitless possibilities and narrowing to the essential.
However, the possibilities here are even more expansive than sitting
down in front of a canvas and paints; it is more like walking into a new
studio with paint, clay, printing press, video software, and more. Artists
tell us that they know they will need more time to experiment before
they can take full advantage of the tool. Similar to artists learning a
new medium, we fully expect that as artists spend more time with the
Fig. 12. Intuition might dictate that the water in an Antarctic visualization
should be blue (left). However, in practice with complex multivariate data
such as the six water masses mixing shown in this figure, using blue as
the ocean can quickly become overwhelming. One solution is to try a
more neutral color (middle and right). The subtle differences between the
middle and right images is something easily accomplished in the ABR
interface, enabling fine-tuned control of visualizations.
interface, the resulting variety in visualizations will expand as well.
We also wish to be clear that for all of the benefits for accessibility
presented in this interface, it does not address the major challenge of
data wrangling. For the data pictured in this paper, we have relied
upon ParaView to accomplish that task. By structuring the design
interface and rendering engine as a modular system that can connect to
existing tools like ParaView via network sockets, we are able to read
data from this tool that scientists already use for analysis and have
a communication strategy that can be mimicked with other scientist-
facing data analysis tools. However, this does not address the problem
that, for many actively studied datasets, the initial task bringing data
into such tools can be a daunting challenge.
5 CONCLUSIONS AND FUTURE WORK
Throughout the construction of this artist-focused interface for creating
visualizations of 3D multivariate data, we have broken apart the con-
ceptual components of existing visualization software and reassembled
them into a new whole, leveraging artistic metaphor and language to
present the data visualization design process in a new way. Like prior
work in building visualization interfaces that support artists in their
creative methodologies, the interface emphasizes rapid exploration of
design alternatives, and by building the interface on top of the Artifact-
Based Rendering engine, the result also enables artists to bring color,
texture, line, and form from their existing art practices to the world of
data-intensive science. Our future work includes continuing to explore
new visual encoding styles, which we plan to add to the interface as new
plates. We are also excited by the potential of tangible user interfaces
that might create an even tighter connection and seamless workflow
between the physical task of creating in the studio and incorporating
those creations into data-driven 3D renderings.
ACK NOW LE DG ME NT S
This research was supported in part by the National Science Founda-
tion (IIS-1704604 & IIS-1704904). MPAS-Ocean simulations were
conducted by Mark Petersen, Phillip Wolfram, Mathew Maltrud and
Xylar Asay-Davis as part of the Energy Exascale Earth System Model
(E3SM) project, funded by the U.S. Department of Energy (DOE),
Office of Science, Office of Biological and Environmental Research
with analyses conducted by PJW, MEM, and RXB under ARPA-E
Funding Opportunity No. DE-FOA-0001726. E3SM simulations are
conducted at Argonne Leadership Computing Facility (contract DE-
AC02-06CH11357); National Energy Research Scientific Computing
Center (DE-AC05-00OR22725); Oak Ridge Leadership Computing Fa-
cility (DE-AC05-00OR22725); Argonne Nat. Lab. high-performance
computing cluster, provided by BER Earth System Modeling; and Los
Alamos Nat. Lab. Institutional Computing, US DOE NNSA (DE-AC52-
06NA25396). The authors would also like to thank the artists who tried
out the interface, and Wingate Studio for providing the printmaking
images.
REFERENCES
[1]
Material design icons. https://material.io/resources/icons/, Accessed 1st
February, 2020.
[2]
G. Abram and L. Treinish. An extended data-flow architecture for
data analysis and visualization. ACM SIGGRAPH Computer Graphics,
29(2):17–21, 1995.
[3]
J. Ahrens, B. Geveci, and C. Law. Paraview: An end-user tool for large
data visualization. The visualization handbook, 717, 2005.
[4]
Z. AlMeraj, B. Wyvill, T. Isenberg, A. A. Gooch, and R. Guy. Automati-
cally mimicking unique hand-drawn pencil lines. Computers & Graphics,
33(4):496–508, 2009.
[5]
B. Bach, N. H. Riche, S. Carpendale, and H. Pfister. The emerging genre
of data comics. IEEE computer graphics and applications, 37(3):6–13,
2017.
[6]
S. Bruckner and M. E. Groller. Volumeshop: An interactive system for
direct volume illustration. IEEE, 2005.
[7]
B. Buxton. Sketching user experiences: getting the design right and the
right design. Morgan kaufmann, 2010.
[8]
A. Cairo. The Functional Art:An introduction to information graphics and
visualization. New Riders, Berkeley, CA, 2013.
[9]
H. Childs, E. Brugger, B. Whitlock, J. Meredith, S. Ahern, D. Pugmire,
K. Biagas, M. Miller, C. Harrison, G. H. Weber, H. Krishnan, T. Fogal,
A. Sanderson, C. Garth, E. W. Bethel, D. Camp, O. R
¨
ubel, M. Durant,
J. M. Favre, and P. Navr
´
atil. VisIt: An End-User Tool For Visualizing and
Analyzing Very Large Data. In High Performance Visualization–Enabling
Extreme-Scale Scientific Insight, pp. 357–372. Oct 2012.
[10]
D. J. Cox. Using the supercomputer to visualize higher dimensions: An
artist’s contribution to scientific visualization. Leonardo, 21(3):233–242,
1988.
[11] J. Gray. Jim Gray on eScience: A transformed scientific method (2007).
[12]
H. Guo, N. Mao, and X. Yuan. Wysiwyg (what you see is what you get)
volume visualization. IEEE Transactions on Visualization and Computer
Graphics, 17(12):2106–2114, 2011.
[13]
H. Houser. Infowhelm: Environmental Art and Literature in an Age of
Data. Columbia University Press, New York, NY, 2020.
[14]
T. Isenberg, M. H. Everts, J. Grubert, and S. Carpendale. Interactive
exploratory visualization of 2d vector fields. In Computer Graphics Forum,
vol. 27, pp. 983–990. Wiley Online Library, 2008.
[15]
S. Johnson, F. Samsel, G. Abram, D. Olson, A. J. Solis, B. Herman,
P. J. Wolfram, C. Lenglet, and D. F. Keefe. Artifact-Based Rendering:
Harnessing Natural and Traditional Visual Media for More Expressive
and Engaging 3D Visualizations. IEEE transactions on visualization and
computer graphics, 26(1):492–502, 2019.
[16]
D. F. Keefe, D. Acevedo, J. Miles, F. Drury, S. M. Swartz, and D. H.
Laidlaw. Scientific sketching for collaborative vr visualization design.
IEEE Transactions on Visualization and Computer Graphics, 14(4):835–
847, 2008.
[17]
D. F. Keefe, D. B. Karelitz, E. L. Vote, and D. H. Laidlaw. Artistic
collaboration in designing vr visualizations. IEEE Computer Graphics
and Applications, 25(2):18–23, 2005.
[18]
A. Lu, C. J. Morris, J. Taylor, D. S. Ebert, C. Hansen, P. Rheingans, and
M. Hartner. Illustrative interactive stipple rendering. IEEE Transactions
on Visualization and Computer Graphics, 9(2):127–138, 2003.
[19] E. J. Luke and C. D. Hansen. Semotus visum: a flexible remote visualiza-
tion framework. IEEE, 2002.
[20]
E. B. Lum and K.-L. Ma. Hardware-accelerated parallel non-photorealistic
volume rendering. In Proceedings of the 2nd international symposium on
Non-photorealistic animation and rendering, pp. 67–ff, 2002.
[21] N. Mielbach. Straight talk with Nathalie Miebach, 2013.
[22]
E. Pasternak, R. Fenichel, and A. N. Marshall. Tips for creating a block
language with Blockly. In 2017 IEEE Blocks and Beyond Workshop (B&B),
pp. 21–24. IEEE, 2017.
[23]
M. R. Petersen, X. S. Asay-Davis, A. S. Berres, Q. Chen, N. Feige,
M. J. Hoffman, D. W. Jacobsen, P. W. Jones, M. E. Maltrud, S. F. Price,
et al. An evaluation of the ocean and sea ice climate of e3sm using mpas
and interannual core-ii forcing. Journal of Advances in Modeling Earth
Systems, 11(5):1438–1458, 2019.
[24]
M. R. Petersen, D. W. Jacobsen, T. D. Ringler, M. W. Hecht, and M. E.
Maltrud. Evaluation of the arbitrary Lagrangian-Eulerian vertical coor-
dinate method in the MPAS-Ocean model. Ocean Modelling, 86(0):93 –
113, 2015. doi: 10. 1016/j.ocemod. 2014.12.004
[25]
A. Polli. Atmospherics/weather works: A spatialized meteorological data
sonification project. Leonardo, 38(1):31–36, 2005.
[26]
C. Reas and B. Fry. Processing: A Programming Handbook for Visual
Designers and Artists. MIT Press, Cambridge, MA, 2014.
[27]
M. Resnick, J. Maloney, A. Monroy-Hern
´
andez, N. Rusk, E. Eastmond,
K. Brennan, A. Millner, E. Rosenbaum, J. Silver, B. Silverman, et al.
Scratch: programming for all. Communications of the ACM, 52(11):60–
67, 2009.
[28]
T. Ringler, M. Petersen, R. Higdon, D. Jacobsen, P. Jones, and M. Maltrud.
A multi-resolution approach to global ocean modeling. Ocean Modelling,
69(0):211–232, 2013. doi: 10. 1016/j.ocemod. 2013.04.010
[29]
F. Samsel. Art-Sci-Tech: Examining the Spectrum. IEEE VIS, VISAP,
2013.
[30]
F. Samsel, G. Abram, S. Johnson, B. Herman, and D. F. Keefe. The
Sculpting Vis Library. http://sculpting-vis.tacc.utexas.edu/library.
[31]
F. Samsel, A. Bares, S. Johnson, and D. F. Keefe. Scientific Visualization:
Enriching Vocabulary via the Human Hand. IEEE VIS Arts Program,
2019.
[32]
F. Samsel, S. Klaassen, and D. H. Rogers. Colormoves: Real-time inter-
active colormap construction for scientific visualization. IEEE computer
graphics and applications, 38(1):20–29, 2018.
[33]
D. Schroeder, D. M. Coffey, and D. F. Keefe. Drawing with the flow: a
sketch-based interface for illustrative visualization of 2d vector fields. In
SBM, pp. 49–56. Citeseer, 2010.
[34]
D. Schroeder and D. F. Keefe. Visualization-by-sketching: An artist’s inter-
face for creating multivariate time-varying data visualizations. IEEE trans-
actions on visualization and computer graphics, 22(1):877–885, 2015.
[35]
A. Segal. Grewingk glacier. https://www.adriensegal.com/grewingk-
glacier, 2015. Accessed March 2019.
[36]
B. Shneiderman. Creating creativity: user interfaces for supporting in-
novation. ACM Transactions on Computer-Human Interaction (TOCHI),
7(1):114–138, 2000.
[37]
B. Shneiderman. Designing for fun: how can we design user interfaces to
be more fun? interactions, 11(5):48–50, 2004.
[38]
G. Singh. Wearing multiple hats. IEEE Computer Graphics and Applica-
tions, 38(4):6–8, 2018.
[39]
M. Swackhamer, A. J. Johnson, D. Keefe, S. Johnson, R. Altheimer, and
A. Wittkamper. Weather report: Structuring data experience in the built
environment. Proceedings of Architectural Research Centers Consortium,
pp. 102–111, 2017.
[40]
M. Terry and E. D. Mynatt. Side views: persistent, on-demand previews
for open-ended tasks. In Proceedings of the 15th annual ACM symposium
on User interface software and technology, pp. 71–80, 2002.
[41] A. Thudt. Visualizations for personal reflection and expression. 2018.
[42]
F. Viegas and M. Wattenberg. Artistic Data Visualization: Beyond Visual
Analytics, pp. 182–191. Springer Berlin Heidelberg, Berlin, Heidelberg,
2007. doi: 10. 1007/978-3-540-73257-0 21
[43]
Y. Wang, A. Segal, R. Klatzky, D. F. Keefe, P. Isenberg, J. Hurtienne,
E. Hornecker, T. Dwyer, and S. Barrass. An emotional response to the
value of visualization. IEEE computer graphics and applications, 39(5):8–
17, 2019.
[44]
C. Ware. Information Visualization: Perception for Design. Morgan
Kaufmann Publishers, San Francisco, CA, 2012.
[45]
R. West, R. Malina, J. Lewis, S. Gresham-Lancaster, A. Borsani, B. Merlo,
and L. Wang. Dataremix: Designing the datamade. Leonardo, 2015.
[46]
P. J. Wolfram, T. D. Ringler, M. E. Maltrud, D. W. Jacobsen, and M. R.
Petersen. Diagnosing isopycnal diffusivity in an eddying, idealized midlat-
itude ocean basin via lagrangian, in situ, global, high-performance particle
tracking (light). Journal of Physical Oceanography, 45(8):2114–2133,
2015.
[47]
N. Q. Zhu. Data visualization with D3. js cookbook. Packt Publishing Ltd,
2013.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Abstract The Energy Exascale Earth System Model (E3SM) is a new coupled Earth system model sponsored by the U.S Department of Energy. Here we present E3SM global simulations using active ocean and sea ice that are driven by the Coordinated Ocean‐ice Reference Experiments II (CORE‐II) interannual atmospheric forcing data set. The E3SM ocean and sea ice components are MPAS‐Ocean and MPAS‐Seaice, which use the Model for Prediction Across Scales (MPAS) framework and run on unstructured horizontal meshes. For this study, grid cells vary from 30 to 60 km for the low‐resolution mesh and 6 to 18 km at high resolution. The vertical grid is a structured z‐star coordinate and uses 60 and 80 layers for low and high resolution, respectively. The lower‐resolution simulation was run for five CORE cycles (310 years) with little drift in sea surface temperature (SST) or heat content. The meridional heat transport (MHT) is within observational range, while the meridional overturning circulation at 26.5°N is low compared to observations. The largest temperature biases occur in the Labrador Sea and western boundary currents (WBCs), and the mixed layer is deeper than observations at northern high latitudes in the winter months. In the Antarctic, maximum mixed layer depths (MLD) compare well with observations, but the spatial MLD pattern is shifted relative to observations. Sea ice extent, volume, and concentration agree well with observations. At high resolution, the sea surface height compares well with satellite observations in mean and variability.
Article
Full-text available
This article presents ColorMoves, an interactive tool that promotes exploration of scientific data through artist-driven color methods in a unique and transformative way. We discuss the power of contrast in scientific visualization, the design of the ColorMoves tool, and the tools application in several science domains. ColorMoves is a freely available resource that can be found at SciVisColor.org.
Article
Full-text available
As we increasingly rely on data to understand our world, and as problems require global solutions, we need to effectively communicate that data to help people make informed decisions. The special Art on Graphics article explores the potential of data comics and their unique ability to communicate both data and context via compelling visual storytelling.
Article
When assessing the value of visualizations, researchers traditionally focus on efficiency, comprehension, or insight. However, analyzing successful data physicalizations leads to a deep appreciation for hedonic qualities. Informed by the role of emotion in psychology, art, design, marketing, and HCI, we argue for an expanded definition of value, applicable to all forms of data visualization.
Article
We introduce Artifact-Based Rendering (ABR), a framework of tools, algorithms, and processes that makes it possible to produce real, data-driven 3D scientific visualizations with a visual language derived entirely from colors, lines, textures, and forms created using traditional physical media or found in nature. A theory and process for ABR is presented to address three current needs: (i) designing better visualizations by making it possible for non-programmers to rapidly design and critique many alternative data-to-visual mappings; (ii) expanding the visual vocabulary used in scientific visualizations to depict increasingly complex multivariate data; (iii) bringing a more engaging, natural, and human-relatable handcrafted aesthetic to data visualization. New tools and algorithms to support ABR include front-end applets for constructing artifact-based colormaps, optimizing 3D scanned meshes for use in data visualization, and synthesizing textures from artifacts. These are complemented by an interactive rendering engine with custom algorithms and interfaces that demonstrate multiple new visual styles for depicting point, line, surface, and volume data. A within-the-research-team design study provides early evidence of the shift in visualization design processes that ABR is believed to enable when compared to traditional scientific visualization systems. Qualitative user feedback on applications to climate science and brain imaging support the utility of ABR for scientific discovery and public communication.
Article
At the University of Minnesota, Daniel Keefe and Marc Swackhamer are members of MINN_LAB, a trans-disciplinary collective that creates urban site-specific installations combining scientific visualization, architecture, design and landscape architecture.