Thinking with Things: Landscapes, Connections, Performances as modes of Building Shared Understanding

Abstract

[EDIT: ResearchGate added this automatically, and it's an 'Early Access' version IEEE posted, that has various errors in it. Also ResearchGate has the wrong journal. I'll update this with the right version, and an OA preprint, once it's published properly.]

This article explores the relatively underexplored potential for physicalisations to materialise qualitative data related to human experiences and knowledge domains. Our reading of "data" in this context extends from imperceptible systems and infrastructures to mental models and the phenomenological dimensions of experiences themselves. Physical objects can be regarded as a form of knowledge with which to inquire about human life, bring about improved conditions, and imagine alternative realities. Objects are made of materials, which are manipulated materials into various configurations. The materials used in the process of externalisation have a profound influence on the resulting forms, and through them on how knowledge is constructed and internalised. We pay detailed attention to the characteristics of materials and how they are combined, in the context of interdisciplinary exchange. We are motivated by the need for a shared understanding of what work materials can do in the making of physicalisations. We suggest this work is useful in the analysis of physicalisations, specifically where they seek to articulate the phenomena of lived experience.

Full text

Thinking With Things:
Landscapes,
Connections, and
Performances as Modes
of Building Shared
Understanding
Dan Lockton
Carnegie Mellon University
Laura Forlano
Illinois Institute of Technology
John Fass
London College of Communication
Lisa Brawley
Vassar College
Abstract—This article explores the relatively underexplored potential for physicalizations
to materialize qualitative data related to human experiences and knowledge domains. Our
reading of “data” in this context extends from imperceptible systems and infrastructures
to mental models and the phenomenological dimensions of experiences themselves.
Physical objects can be regarded as a form of knowledge with which to inquire about
human life, bring about improved conditions, and imagine alternative realities. Objects are
made of materials, which are manipulated materials into various configurations. The
materials used in the process of externalization have a profound influence on the resulting
forms, and through them on how knowledge is constructed and internalized. We pay
Digital Object Identifier 10.1109/MCG.2020.3027591
Date of publication 29 September 2020; date of current
version 23 October 2020.
Theme Article: Data Physicalization
Theme Article: Data Physicalization
38 0272-1716 ß2020 IEEE Published by the IEEE Computer Society IEEE Computer Graphics and Applications
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

detailed attention to the characteristics of materials and how they are combined, in the
context of interdisciplinary exchange. We are motivated by the need for a shared
understanding of what work materials can do in the making of physicalisations. We
suggest this work is useful in the analysis of physicalizations, specifically where they
seek to articulate the phenomena of lived experience.
&THIS ARTICLE EXPLORES “thinking with things”
at the intersection of generative codesign and
data physicalization. This article and our work
take a view of data physicalization that goes
beyond the expression of a specific quantitative
dataset in physical form. Instead, as designers
what we mean by data is closer to what is gener-
ated in design practice, more specifically, in gen-
erative codesign situations. We are interested in
the relationship between what people can imag-
ine, what materials are available to them, and
what types and kinds of shared understanding
emerge. We have applied this thinking to many
different topics, including emotional response,
social networks, and most recently how we
might help people express their feelings and atti-
tudes to the systems they inhabit. By experience
we mean to explore the phenomenological
nature of everyday life; for example, how people
think about their relationships to technology,
how people understand their own personal and
professional trajectories, and how these shape
their understanding of disciplinary boundaries.
This work has taken on importance as the inter-
section of complex sociotechnical systems
comes to dominate human experience, and the
way those systems are regulated and arranged
becomes ever more opaque. We seek to access
these types of knowledge by giving people the
opportunity to express them in physical form.
Tversky
1
suggested that “when thought over-
whelms the mind, the mind puts it into the world,”
noting how “people use anything at hand, their
hands, their bodies, arrangements of sticks and
stones and coffee cups, sketches in the sand, scrib-
bles on paper napkins, and more” as “external rep-
resentations of thought...designed to serve
thought, for self or for others.” While the use of
drawing in this context has been relatively widely
explored, physical models have been less so.
In design fields such as architecture, plan-
ning, and product design, physical models can
narrow the gap (and the glitches) between
abstract concepts and concrete implementation
by allowing complexities to surface as part of
the design process. Physical models are often
used to surface implicit assumptions and values
as well as to communicate and test ideas; they
are especially useful tools for working with the
qualitative, experiential elements of a design,
i.e., the quality of the lighting, the effect of an
acoustic environment, the feel of the surface of
an object. Physical models allow for the develop-
ment of new guiding metaphors, which can be a
critical way for designers to generate and com-
municate their internal understandings. They
also enable the public to convey their experi-
ence of a design-concept in process in order to
more meaningfully inform a design. Kirsh
2
found
that “the materiality of external representations
provides affordances that internal representa-
tions lack.” We are interested in what these affor-
dances may be and how participants in codesign
activities have taken advantage of them. More
specially, we are interested in what materials
one might use to physicalize the qualitative data
that is generated and enabled in any design or
codesign process. How do materials matter?
To explore these points, we use examples
of our work drawn from a half-day codesign
workshop held in the context of the Relating
Systems Thinking and Design conference in
Chicago in 2019. The methodological focus of
our workshop was on “Tangible Thinking.” The
thematic focus was on how participants concep-
tualize and experience “disciplinarity and inter-
disciplinarity”—abstract ideas that regulate and
structure the lives of participants in a profes-
sional conference. We explored three different
ways of making ideas tangible—topological, con-
nective, and performative—as described below.
BACKGROUND
In the field of science and technology studies
(STS), many social scientists have studied the
November/December 2020 39
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

ways in which scientists visualize and model
their data, creating images and objects that
allow for the development of new ideas and
knowledge. This has led to a wide variety of con-
cepts such as epistemic things,
3,24
epistemic
objects,
4
conscription devices, and boundary
objects. One of the features of these things, devi-
ces, and objects is that they foster multiple peo-
ple to engage with an idea at the same time. In
addition, they can be useful, as in the case with
boundary objects, for fostering interdisciplinary
conversations. A related concept, that of
“trading zones” is also useful for understanding
the ways in which interdisciplinary knowledge
can be shared, and, even, for the formation of
new fields of knowledge.
6
The idea here is that
for new areas of knowledge to emerge, they
must first develop pidgin languages and dialects
that cross the existing fields.
In what are known as “inventive methods”
7
as
part of the postcritical turn, scholars in a range
of fields are themselves exploring the use of
visual and physical forms of knowledge-making
through practice-based research, design as
inquiry, and research creation.
8,9,10,11
For exam-
ple, scholars in the humanities have engaged
with experimentation and objects through what
Sayers calls conceptual matter.
12
This has led to
a wide range of critical making,
13
critical data
practices, critical technological praxis, and criti-
cal media practices.
Some of these concepts have already been
adopted in the field of design. For example, build-
ing on the concept of boundary objects and cul-
tural probes, Halpern, Erickson, Forlano, and Gay
developed the concept of boundary probes based
on a codesign workshop activity.
14
Designers have
been interested both in the discursive potential of
prototypes, as well as their speculative potential.
A related concept, provotypes (provocative pro-
totypes), draws productively on the tension
between stakeholders for the purpose of partici-
patory innovation. Finally, speculative design,
design fiction, and experiential futures have also
made use of objects for promoting critical thinking
about our present conditions as well as alternative
possible futures. There are strong parallels with
emergent themes and methods in data physicali-
zation, such as constructive visualization
5
and
personal physicalization constructions,
15
in which
participants use a variety of materials and compo-
nents to construct their own physicalizations of
datasets, including data about patterns in their
own lives, for purposes of self-reflection and
understanding, and we build on insights from, and
experience of workshops in this rich series of data
physicalization explorations.
16
As social scientists, urbanists, design resea-
rchers, and designers, we have used the affor-
dances of physical objects in our previous
projects about the future of work, autonomous
vehicles, and digital social networks. In bringing
our interests to the analyses of socio-technical
systems, we have observed that many diagrams
focus on depicting the links in networks but
very little has been done to illustrate the quality
of relations in the models. How might different
theories about the social be prototyped and
embedded in physical models? For example, in
sociology, theories about the strength of weak
ties, could be integrated into physical models.
In the words of Haraway, “It matters what
matters we use to think other matters with; it
matters what stories we tell to tell other stories
with; it matters what knots knot knots, what
thoughts think thoughts, what ties tie ties. It
matters what stories make worlds, what worlds
make stories.”
17
Therefore, “matters” matter, but
does matter also matter? And, if so, in what ways
does it matter? Furthermore, what kinds of mat-
ter support what kind of thinking and how does
their materiality matter?
THE WORKSHOP
Our article seeks to explore these questions
based on a workshop that we ran at RSD 8 in Chi-
cago in Fall 2019. The workshop was attended by
designers, students, systems theorists, and aca-
demics from varying disciplinary locations within
higher education and from differing areas of prac-
tice in the design industry. Most of the partici-
pants did not know each another. The workshop
took four hours: it featured a brief orientation and
context-setting session, followed by three one-
hour sessions each of which explored a different
mode of physicalizing ideas: landscape, connec-
tion, dynamics. Participants worked in small
groups of five to eight people to codesign models
in response to questions posed about how they
Data Physicalization
40 IEEE Computer Graphics and Applications
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

understood and experienced disciplinarity and
interdisciplinarity. At the close of each session,
groups shared their models drawing on the narra-
tive that cocreating the model had generated. The
workshop involved groups of people working col-
laboratively on a common creative task. Drawing
on the tradition of codesign, we emphasized the
generative nature of the exercise and the value of
shared experience. Our use of richly varied materi-
als is in sharp contrast tothe stifling conformity of
codesign methods and their walls of Post-it Notes.
The workshop emphasized a cocreated expres-
sive activity designed to enable the surfacing of
latent or otherwise unexpressed awareness that
enhances a discussion at group level. We describe
each of the three sessions and their differing mate-
rial strategies below.
Deploying Elements of Landscape
In the first session, participants worked with a
predesigned kit of parts that featured elements of
landscape, the Mental Landscapes kit
18
(see Fig-
ures 1, 2). This kit comprises a selection of laser-
cut card elements acting as “visual prostheses,”
19
embodying a particular set of metaphors based
around stylized landscapes and features within
landscapes, such as hills, roads, bridges, fields,
fences, and weather. These include: hills, moun-
tains, and raised ground of many sizes and col-
ors—both 3-D cones and flat elevations held
vertically using slotted blocks; lakes, ponds, and
rivers of many sizes and colors, plus “whirlpools”
or eddies; fields/areas of land, of many sizes and
colors, including a “ground” sheet; roads, bridges,
and fences, which could also be interpreted as rail-
way tracks; trees and cacti of different shapes and
sizes; silhouettes of people of different sizes;
weather elements such as sun/moon, clouds
(cirrus-esque and cumulus-esque), clouds with
rain, clouds with snow, clouds with lightning
bolts, held vertically using crocodile clips on rods
(whirlpools could also be used as “cyclones”);
sticky notes for use as labels or annotations; and
generic shapes, modifiable in different ways. The
main material affordances involve the flexibility
of—and ability to write on—the card elements
(see Figure2); in addition, the variety of easily (re)
positionable lightweight elements is a significant
property, inviting particular forms of interaction
and configuration. Participants can also modify
the card elements by cutting them and joining
them together, and—although it can probably go
without saying—creative reinterpretations of
what particular elements represent, or could be
metaphors for, are strongly encouraged. Our
design process aimed to maximize participants’
ability to express their thinking, while not over-
whelming them with sheer quantity of premade
elements. We wanted to provide opportunities for
expression without prescribing a particular narra-
tive format, although “journeys” of various kinds
have often resulted.
The Mental Landscapes kit has been used in
workshops where participants assemble and
arrange a variety of elements to make abstracted
model landscapes which on some level represent
or translate their mental models of concepts,
experiences, or relations between ideas. Through
a series of workshops in both academic and non-
academic settings, participants have built models
representing their own career paths, life journeys,
Figure 2. Elements of the Mental Landscapes kit
laid out for participants to choose from. The wooden
blocks are used to enable flat elements to be held
vertically. (Image: Manuela Aguirre Ulloa; used with
permission.)
Figure 1. Elements of the Mental Landscapes kit
laid out for participants. (Image: Dan Lockton.)
November/December 2020 41
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

group projects, how they feel within their work-
place culture, and visions of humanity’s past and
future. Some activities have been individual, while
others are group based to try to examine the col-
lective imaginaries of a particular theme or idea.
Aside from gaining insights around the topics
being explored—acting as a form of probe—the
primary aim of the workshops at this stage has
been to help scope possibilities for the kit’s devel-
opment and to explore how this kind of metaphor-
based physicalization method could be used in
user research, in higher education classrooms,
and, perhaps, even in art therapy.
Landscapes are a common type of metaphor
in speech, particularly for talking about rela-
tions between parts of a whole, or mapping the
structure of one concept onto another. Discus-
sions about disciplines are very often
expressed using landscape metaphors—e.g.,
“What is your field?” The English term, “field” is
derived from “feld” meaning open country.
There is an expanding awareness of the value of
interdisciplinary teams; at the same time, there
is as well growing scholarly literature describ-
ing the ongoing transformation of disciplinary
structures of knowledge production. These
changes are signaled by a growing list of pre-
fixes: interdisciplinary, multidisciplinary, trans-
disciplinary, predisciplinary, cross-disciplinary,
antidisciplinarity, and postdisciplinary. In his
discussion of “postdisciplinarity,” Mario Bia-
gioli has argued that the conceptual model of
the organization of knowledge in the sciences is
shifting—from “From Discipline and Canon to
Collaborations and Problems”—in ways that
scholars in other fields (there’s that word)
might find useful.
20
It is perhaps especially pro-
ductive to use the Mental Landscape kit within
a context, where the landscape metaphor may
no longer adequately express people’s experi-
ence of what they know and how they work.
In the context of the Chicago workshop series
explored in this article, Mental Landscapes was
used for the first session. We posed the following
questions as prompts for participants: “How do
we each imagine our knowledge in relation to
other people’s? What would our “areas of
expertise” look like if they were actual “areas”?
We asked participants, in groups, to choose ele-
ments and build models, on a black foam core
surface, that responded to the questions. The
suggestion was that participants might initially
create their own “parts” of the model, and then
discuss how to build a landscape together from
them, prompting discussion around the intersec-
tions and differences between people’s areas of
expertise and knowledge.
Some illustrative moments and vignettes
from the group modeling include: 1) a group
who found that the issue of the climate crisis
was shaping their collective landscape, with its
inevitable approach being modeled as some-
thing like a storm front, a bright red rectangle
sweeping in, toward everything in its path; 2) a
participant who showed his disciplinary training
and expertise (in computer science) as being
“walled in,” corralled behind a fence—he said he
felt he could see other areas of interest outside,
but felt unable to reach them; and 3) a group
who used “rivers” labeled “Why? What?” as a
way to unite the different areas of knowledge
they identified they had collectively, seeing the
rivers as being something like a “common
thread” (to use another metaphor) of question-
ing running through their professional expertise
(see Figures 3 and 4).
In each case here, the material properties,
and specific affordances, of the elements (flexi-
ble, positionable, perforated, placeable in a
meandering way) contributed to the kinds of
sensemaking and storytelling that were possible.
Deploying elements of landscape involves the
arrangement of materials prefigured into shapes
and forms to reflect emerging sense-making. How-
ever, participants’ use of, for example, rivers to
link parts of the landscape together suggests a
Figure 3. Both the climate crisis storm front and
‘“fenced in” area are visible here. (Image: Dan Lockton.)
Data Physicalization
42 IEEE Computer Graphics and Applications
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

need unmet by a static set of elements: It is per-
haps these connections between ideas and con-
cepts, and the nature of those links, that prompt a
different narrative.
Emphasizing Connective Tissue
The next stage of the workshop process was
based on methods arising from Aguirre Ulloa
and Paulsen.
21
Participants were asked to aug-
ment their landscape models using materials
that emphasize connections and relations (ele-
ments of this second workshop overlay are visi-
ble in Figure 4). Participants were asked: How
are the different areas of knowledge related?
What relationships influence our fields of prac-
tice? What do these relationships look like?
What are their qualities, properties, values, and
“materials”? Workshop materials included cop-
per wire, elastic thread, yarn, and elastic band-
ing (see Figure 5). Their variegated material
qualities—thick, rough, smooth, stretchy, shiny,
elastic, fragile—were intended to help partici-
pants focus especially on the character of “in-
between”—and to coconstruct physical forms
that conveyed the nature of social and epistemo-
logical relationships that characterize their
experiences of interdisciplinarity.
The workshop’s connective materials did
indeed provoke just this sort of conversation
about cross-disciplinary and interdisciplinary
work. For example, one participant working in a
group of five chose a short strand of silver
sequence from the pile of materials at the front
of the room. She described her choice to her
group members, explaining that working in a
way that connects two different disciplines can
seem very appealing from the outside, like a
shiny thing: “I chose this [strand of silver
sequence] because, in many cases, the connec-
tion between two different disciplines seems...
well, everyone says, ‘oh! that’s cool, that’s great,
you’re working in sociology and design!’” Then,
she knotted the strand to make her more
emphatic point: trying to forge a career by work-
ing across disciplines can be very difficult: “But
then—and you know, and this is a very personal
thing—for my career, it’s not cool. So it’s like
shiny and nice, but then...” And, here another
member of the group completed her sentence,
adding to the building narrative “—and then
there’s the reality.” The first participant began
to drape her shiny strand from one paper cone
(taken to represent a discipline) to another
paper cone (see Figure 6). At this point, another
participant adds, “But maybe the shiny thing is
not the connector piece.” And, another still,
“Yes, maybe put it around here, yeah, I’m not
sure if it is a connector” (see Figure 7). The
group was collectively building a narrative about
the paradoxical character of interdisciplinary
work.
Exploring Dynamic Performance
The third session of the workshop introduced
the notion of time. In this session, participants
created new models rather than augmenting the
model they had created in the first two sessions
by deploying elements of landscape and empha-
sizing connective relationships. The materials
used had varying temporal qualities: they are
Figure 4. Rivers of “Why? What?” This photo (taken
during the next stage of the workshop) also includes
connective material elements. (Image: Dan Lockton.)
Figure 5. Supply of connective materials
emphasizing differing qualities of relation: Elastic,
shiny, rigid, lose, firm, fuzzy. (Image: John Fass.)
November/December 2020 43
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

expressed or materialized in time: rolling, dissolv-
ing, dripping, magnifying, and flashing on and off.
The materials included glass marbles, wooden
chutes, LED lights, a magnifying lens, ink, and
sugar cubes. These materials were intended to
encourage/allow participants to express temporal
experience and the dynamic nature of ideas;
these materials facilitated the performance of
dynamic change: knowledge systems change,
knowing takes time. A second performative
aspect came in here when participants altered
and rearranged the materials in a live explanation
of what they did. Instructions to participants did
not narrowly specify what to do but rather mate-
rials are presented, a theme is proposed and a
duration is given. People were free to organize,
classify, and align these materials in ways that
best suit the moment. This ambiguity helps us to
mediate the binaries experienced in spoken
language and provokes modes of material engage-
ment that allow people to discover unexpected
connections.
One group brought these materials into a
meaningful gestalt that expresses the dynamic
and shifting nature of interdisciplinary relation-
ships between technology, academia, science,
policy, and design (see Figure 8). This group
sought to capture “the tension and influence
between arts and everything else” by showing in
two and three dimensions how elements of the
chosen dispositif are related. Attention, which
they explained was represented by the marble in
a wooden chute, is shown moving from technol-
ogy toward art and design, while attention in
academia increasingly turns toward critiques of
technology. Using the lights allowed the group
to show how attention turns gradually toward
effects of system interactions—it is not a sudden
realization but a sense of growing awareness.
The specific focus for another group’s physic-
alization was the growing influence of business
on government policy. This was shown by the
use of lenses, which moved in and out (in the
live explanation of the model) that enable
“looking in” and “zooming in.” This happened at
different speeds and at different levels of magni-
fication for the various aspects of the system.
Red ink, standing for human blood, was dripped
onto the sugar cubes, demonstrating the poten-
tial cost in human lives of policies overly influ-
enced by business objectives. Again the
progressive, gradual absorption by the sugar
cubes of the ink stands for a dawning realization
rather than immediate understanding. Relation-
ships are shown using curling lines drawn onto
Figure 6. Knotted strand of silver sequins used to
convey the appeal and also the difficulty of
interdisciplinary work/working across disciplines.
[Image: Lisa Brawley (video still).]
Figure 7. Participants discuss where to place
the shiney strand of silver. [Image: Lisa Brawley
(video still).]
Figure 8. Participants organized materials to show
their interrelationships. (Image: John Fass.)
Data Physicalization
44 IEEE Computer Graphics and Applications
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

the base paper layer to indicate that interdisci-
plinary relationships are not linear or static; “the
messy relationship that connects all these differ-
ent fields together is human behavior.”
The materials were chosen by the group so
as to allow for contrasting and complementary
representations (see Figure 9). So the steady,
unblinking nature of an LED light illuminates a
specific area of the model and is small enough to
be limited in what it can highlight, but can also
fit inside one of the wooden cubes. Sugar cubes
can be stacked or lined up, cultured ink diffuses
slowly through them onto the paper below, but
must be carefully dripped onto them requiring
deliberation and a steady hand. The glass mar-
bles run through the wooden chutes much faster
and are here used as connective devices to help
explain how attention shifts between knowledge
domains. Lenses sit on the base paper magnify-
ing a specific area. When the group explained
what they had done and why, they picked up
these lenses and moved them slowly from arms
length away to directly above a confluence of
the orange and green lines drawn in tape on the
base paper.
The collaborative and participatory nature of
the modeling exercise means that materials
were able to work in a variety of ways. Group
members used the physical materials to express
various aspects of the system, such as gradual
state change, the placing of awareness, and shift-
ing attention. The materials were selected by the
group for their relative sizes. Objects are
arranged in a way that communicates at a
coherent scale, no material dominates the
model. The contrasting speeds of change have
been carefully arranged; the fast run of a marble
through a chute, the slow absorption of liquid
by a sugar cube, a human controlled zoom in
with a lens. The characteristics of the individual
materials allow the group members to collec-
tively and consensually assign them a particular
metaphorical function that is played out in the
model.
What is being expressed in this physicaliza-
tion is a group understanding of how elements
of a gradually changing situation are related to
one another. Using physical materials means
making choices about what the right materials
are and what the most effective arrangement is.
Using materials that change shape, position,
intensity, and distance complicates things by
rendering the whole physicalization unstable.
The data being physicalized here are differing
internal attitudes and opinions about a set of
interdisciplinary relationships, a highly abstract
subject difficult to express in spoken or written
words. The physical and spatial qualities of the
representation allow for everything to be said at
once. The situation, a generative codesign work-
shop, affords a group telling of the physicaliza-
tion, transforming it into a time-based narrative
object.
In the second example from the same session,
two participants chose the following rather
unlikely selection of materials: a marble, photo
sensitive paper, orange water soluble ink, a seg-
ment of wooden track, a magnifying lens, and an
LED light (see Figure 10). They together began to
play with these materials—or to see how much
play the materials afforded. The diverse proper-
ties of these materials seemed to invite this sort
of unscripted exploration: the team dropped the
marble, then dipped it in ink and dropped it
again, and watched the path it traced across the
paper. They were discovering, and discovering
together, what the materials would do and what
they could do with them. They were not only
learning about what each would do with the
materials, but also learning about each other in
this way (see Figure 11). One participant
explained, “We had no idea what these would do,
or what we would do with them; we had to begin
in order to discover that.” One participant
Figure 9. This image shows participants arranging
the elements to show how the attention is shifting
over time from technology toward art and design.
(Image: John Fass.)
November/December 2020 45
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

suggested the process itself demonstrated the
temporal dimensions of working with complexi-
ties such as areas of expertise that shift overtime.
DISCUSSION: WHAT’S THE USE?
AMBIGUITY, PHYSICALIZATION, AND
SHARED NARRATIVE
Across the three sessions of the workshop,
multivalent materials both enticed and enabled
members of codesign groups to develop a
shared narrative about their understanding of
and experience with, in this case, cross-disciplin-
ary work; the differing qualities of varied materi-
als became prompts and props for storytelling—
their differing materials often calling out differ-
ent kinds of stories. In addition, the modeling
process–making, arranging, discussing, negotiat-
ing, telling, proposing, placing, connecting,
doing–allowed for more than representing, exter-
nalizing, or expressing. Instead, we argue that
there is a deepening of mutual understanding
through shared discovery. Participants act with
materials in a way provoked both by their physi-
cal properties and by their dynamic connection.
Different kinds of materials arranged in temporal
relation and experienced in shared encounter
elicit richer and more richly shared understand-
ings than straightforward representations. As a
result, we find ourselves designing not so much
physical tools for representation as material sit-
uations where the work of developing shared
understanding is made perceivable.
In all these examples, participants are not
simply externalizing interior concepts and feel-
ings they hold as individuals, but in interacting
with the tangible materials and with other mem-
bers of their group, they are developing ideas
and building a model that both supports and
reflects elements of the narrative they are creat-
ing together. What results is not a model (or
data physicalization) in a typical sense. Rather it
is a material trace, tangible evidence left behind
the conversation the group had. One question
we are left with is what is the nature of the model
created by these traces of shared use? They “are
not stable artifacts whose correct interpretation
is just a matter of visual literacy, but phenomena
that emerge from a recipients’ extensive engage-
ment with the world and with the knowledge of
others.” In this key sense, the structures that the
groups created are analogous to “autographic
visualizations” created by “material traces” of a
shared encounter.
22
Comparing the outcomes to the physicaliza-
tions created by participants in the series of con-
structive visualization and personal physicalization
workshops discussed earlier,
5, 15, 16
while at first
glance the models produced have much in com-
mon in their creative use of materials, a key differ-
ence is that the outcomes we describe here are
Figure 10. Drip, splat, roll: photo sensitive paper,
orange water soluble ink, a segment of wooden
track, a magnifying lens, and an LED light. (Image:
John Fass.)
Figure 11. Participants exploring together the
capacities of the varied materials they chose.
(Image: John Fass.)
Data Physicalization
46 IEEE Computer Graphics and Applications
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

primarily, perhaps entirely, about qualitative
relationships and characteristics rather than the
quantitative data. While some previous data phys-
icalization workshops, particularly those focused
on self-reflection, have included this aspect as part
of the constructions produced by participants—for
example, by Thudt et al.
15
a participant with hip
pain "felt that the physicality of materials made it
easier to capture his physical pain" and used colors
to create what Thudt et al. call "direct visualization
of experience"—these workshops have all started
from a premise of representing mainly quantitative
datasets, with perhaps a qualitative layer. The
explorations we discuss here had quite a different
form of initial prompt.
As Sara Ahmed reminds “usefulness” is not
merely instrumental but is about capacities that
are open to the future. The capacity is not so
much “‘in” the tool [or in this case, in the work-
shop materials we provided] but depends upon
how the tool is taken up or “put to use.”
23
Thus,
another question might be what kinds of tangible
materials and combination of materials leave
them powerfully “open to the future”—which is
to say inviting, enticing, or otherwise readily
available to be used by participants as they craft
shared understandings. Coming to a shared
understanding takes work but this work is often
invisible and intangible. Here, by asking partici-
pants to use these diverse physical materials to
create models of interdisciplinarity, we were
also asking them to make visible—to leave a
trace of—the effort required to understand each
other’s point of view.
An important aspect of how materials were
chosen for this workshop is that their usefulness
is ambiguous: unlike a stack of Post-its, there is no
set or preordained way to use these tangible mate-
rials. In addition, the workshop materials used
across the three sessions emphasize increasing
degrees of ambiguity—from landscape elements
that might more readily suggest a “field,” to con-
nective threads that lend themselves to thinking
of the “inter” of “interdisciplinarity, to varied time-
expressing materials (a marble, a sugar cube, light
sensitive paper) that complicate the spatial
metaphors often used to describe “areas of knowl-
edge.” The strengths of our approach to physicali-
zation are also its limitations: in centering on
metaphors and language, our workshop methods
highlight interpretative, perspectival, and contex-
tually specific modes of understanding. These
methods often prompt deeper questions, rather
than provide definitive answers.
CONCLUSION
We used three modes—deploying elements
of landscape, emphasizing connective tissue,
and exploring dynamic performance—to eluci-
date and physicalize complex qualitative data.
The work we have done can be seen as an expan-
sion of the affordances and possibilities of the
field of data physicalization. We have taken data
to stand for the qualitative nature of lived experi-
ence as discoverable through the actions of
making and talking about making, a physical
representation. We are concerned with the phys-
ical properties of the materials that allow this to
happen, but are increasingly alert to the emer-
gent social circumstances that arise when doing
so in a collaborative shared situation. We draw
on notions of postcritical materiality and the
types of knowledge generated in practice-based
design research to position our work as
intrigued by the interplay of artifact and person,
relations and telling, material and time.
In the examples that we have discussed the
topic was interdisciplinarity, how people under-
stand their ways of knowing and working. Our
approach deployed ambiguity to the extent that
not assigning narrowly specific meanings to
materials, connections, and movements but
rather leaving these “open to the future”—ena-
bles and requires participants to posit meanings
for themselves—to develop shared narratives of
what individual elements mean and what their
arrangement in time and space implies about
the questions they are answering together. A
shared expression of the relations between
materials, contextualized by the modes we have
described, often results in a shared telling of the
exercise with participants taking turns to explain
the particular meanings they have imparted to
the materials.
There is a politics of materials that is also
important here: where people might have seen or
used them before, how they may trigger associa-
tions from childhood play, office life, or craft mak-
ing, and how they may not be readily associated
November/December 2020 47
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

with design activity. A politics of materials also
extends to the processes that make them avail-
able to us to use as the “raw materials” of a co-
design workshop. These aspects of thinking tan-
gibly are beyond the specific scope of this article.
We acknowledge the limitations of this article
to the extent that it is based on a single work-
shop, conducted in the context of a design con-
ference, over the course of a single day. These
constraints determined both the availability of
participants, and the nature of the materials that
we were able to physically transport to the con-
ference venue. We position this article as report-
ing on the first of a series of similar workshops.
The type of knowledge generated through
this kind of cocreated, shared encounter with
ambiguous physical materials is discursive, con-
tingent, and sometimes unanticipated. It is char-
acterized by serendipitous discovery and an
uncovering of unseen connections. In this way,
we see this work as more than extractive. That
is, it goes beyond externalizing thoughts, opin-
ions, or attitudes that are internal. The proper-
ties and characteristics of objects and the
relationship between them act as mediating
agents not only for how peoples’ mental models
and lived experiences structure their under-
standing of the world around them, but for an
opening toward new shared realizations and
deepened understanding—not just for surfacing
but also for generating new forms of knowledge.
ACKNOWLEDGMENTS
This paper examines a workshop we con-
vened at RSD8: Relating Systems Thinking and
Design Symposium, 17–19 October 2019, Chi-
cago: “Tangible Thinking: Materializing how we
imagine and understand interdisciplinary sys-
tems, experiences, and relationships.” The con-
ference proceedings include a more extensive
set of references and examples than we could
include here. A more extensive set of references
is available at http://imaginari.es/thinking.
We especially thank Manuela Aguirre Ulloa
(Designit & Oslo School of Architecture and
Design), for codesigning and leading the second
session, which is based in her doctoral
research. She was unable to join as a coauthor
of this essay because she was completing her
dissertation.
From Carnegie Mellon, we also thank Matt
Prindible for crucial support in preparing for
and documenting the workshop, Katie Herzog
for insights around materials and form arising
from her Emotional Modeling project, Tammar
Zea-Wolfson for additions to the Mental Land-
scapes kit, and Delanie Ricketts for originating
the kit. We also thank Karianne Rygh and Palak
Dudani (Oslo School of Architecture and Design)
for insightful input and feedback on the work-
shop design and during the workshop discus-
sion. All authors contributed equally to this
work.
&REFERENCES
1. B. Tversky 2015, “The cognitive design of tools of
thought,” Rev. Philosophy Psychol., vol. 6, pp. 99–116.
https://doi.org/10.1007/s13164-014-0214-3
2. D. Kirsh, “Thinking with external representations,” AI
Soc., vol. 25, no. 4, pp. 441–454, 2010.
3. H.-J. Rheinberger, Toward a History of Epistemic
Things: Synthesizing Proteins in the Test Tube,
Stanford, CA, USA: Stanford Univ. Press, 1997.
4. K. Henderson, “Flexible sketches and inflexible data
bases: Visual communication, conscription devices,
and boundary objects in design engineering,” Sci.
Technol. Hum. Values, vol. 16, no. 4, pp. 448–473,
1991.
5. S. Huron, S. Carpendale S., A. Thudt A., A. Tang, and
M. Mauerer“Constructive visualization,” in Proc. Conf.
Des. Interactive Syst. Assoc. Comput. Mach., New
York, NY, USA, pp. 433–442, 2014.
6. P. Galison, “Trading with the enemy,” in Trading Zones
and Interactional Expertise: Creating New Kinds of
Collaboration, M. E. Gorman ed. Cambridge, MA,
USA: MIT Press, 2010, pp. 25–52.
7. C. Lury and N. Wakeford, Inventive Methods: The
Happening of the Social. New York, NY, USA:
Routledge, 2012.
8. L. Candy, “Practice based research: A guide. CCS
report,1, pp.1–19, 2006.
9. J. Zeisel, “Inquiry by design,” Environment/Behavior/
Neuroscience in Architecture, Interiors, Landscape,
and Planning, 2006.
10. D. P. McCormack, “Thinking-spaces for research-
creation,” Inflexions, vol. 1, no. 1, pp.1–16, 2008.
11. L. Sanders and P. J. Stappers, Convivial Design
Toolbox: Generative Research for the Front End of
Design. BIS, 2012.
Data Physicalization
48 IEEE Computer Graphics and Applications
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

12. J. Sayers, Making Things and Drawing Boundaries:
Experiments in the Digital Humanities. Minneapolis,
MN, USA: Univ. Minnesota Press, 2018.
13. M. Ratto, “Critical making: Conceptual and material
studies in technology and social life,” Inf. Soc., vol. 27,
no. 4, pp. 252–260, 2011.
14. M. Halpern, I. Erickson, L. Forlano, and G. Gay,
“Designing collaboration: Comparing cases exploring
cultural probes as boundary-negotiating objects,” in
Proc. Comput. Supported Collaborative Work, San
Antonio, TX, USA, 2013, pp. 1093–1102.
15. A. Thudt, U. Hinrichs, S. Huron, and S. Carpendale,,
“Self-reflection and personal physicalization
construction,” in Proc. CHI Conf. Hum. Factors
Comput. Syst. Assoc. Comput. Mach., New York, NY,
USA, 2018, pp. 1–13.
16. S. Huron, P. Gourlet, U. Hinrichs, T. Hogan, and Y.
Jansen, “Let’s get physical: Promoting data
physicalization in workshop formats,” in Proc. Conf.
Des. Interactive Syst. Assoc. Comput. Mach., New
York, NY, USA, 2017, pp. 1409–1422.
17. D. J. Haraway, “SF: Science fiction, speculative
fabulation, string figures, so far,” ADA: A J. Gender
New Media Technol., no. 3, 2011. [Online]. Available:
http://adanewmedia.org/2013/11/issue3-haraway/.
18. D. Ricketts and D. Lockton, “Mental landscapes:
Externalizing mental models through metaphors,”
Interactions, vol. 26, no. 2, pp. 86–90, 2019.
19. D. Jonassen and Y. H. Cho, “Externalizing mental
models with mindtools,” in Understanding Models for
Learning and Instruction, pp. 145–159, Boston, MA,
USA: Springer, 2008.
20. M. Biagioli, “Postdisciplinary liaisons: Science studies
and the humanities,” Crit. Inquiry, vol. 35, no. 4,
pp. 816–833, 2009, Accessed: Feb. 5, 2020. doi:
10.1086/599586.
21. M. Aguirre-Ulloa and A. Paulsen, “Co-designing with
relationships in mind,” FormAkademisk -
Forskningstidsskrift Des. Og Designdidaktikk, vol. 10.
no. 1, 2017. https://doi.org/10.7577/
formakademisk.1608
22. D. Offenhuber, “Data by proxy—material traces as
autographic visualizations,” IEEE Trans. Vis. Comput.
Graphics, vol. 26, no. 1, pp. 98–108, 2019.
23. S. Ahmed, Queer Phenomenology: Orientations,
Objects, Others. Durham, NC, USA: Duke Univ. Press,
2006.
24 B. Ewenstein and J. Whyte, “Knowledge practices in
design: The role of visual representations as ‘epistemic
objects,” Org. Stud., vol. 30, no. 1, pp. 7–30, 2009.
Dan Lockton is involved with a number of ACM
conferences including CHI, DIS, and C&C (Creativity
and Cognition), along with design conferences
including DRS and EPIC. After working in the design
industry, he was a Research Assistant with Brunel
University, a Research Fellow with the University of
Warwick, a Senior Research Associate with the Royal
College of Art’s Helen Hamlyn Centre for Design,
and a Research Tutor in innovation design engineer-
ing with the Royal College of Art, before joining Car-
negie Mellon University in 2016 as an Assistant
Professor. He established the Imaginaries Lab at
Carnegie Mellon, a new design research studio. His
research interests are centered on the use of design
methods to investigate people’s understanding of
the world, and help people imagine new ways of liv-
ing and understanding, to enable more environmen-
tally and socially sustainable futures. He recived the
B.Sc. degree in industrial design engineering from
Brunel University in 2004, the M.Phil. degree in tech-
nology policy from the University of Cambridge in
2005, and the Ph.D. degree in design from Brunel in
2013. Contact him at danlockton@cmu.edu.
Laura Forlano has been faculty at the Institute of
Design, Illinois Institute of Technology since 2011,
where she is currently an Associate Professor of
design and Director of the Critical Futures Lab. Prior
to joining the Institute of Designe, she completed
Postdoctoral Associate positions at Yale University
and Cornell University. She is a member of the Society
for the Social Studies of Science (4S), the International
Communication Association, and the Design
Research Society. She is an Editor of three books:
Bauhaus Futures (MIT Press, 2019), digitalSTS
(Princeton University Press, 2019) and From Social
Butterfly to Engaged Citizen (MIT Press, 2011). Her
research is focused on the aesthetics and politics at
the intersection between design and emerging tech-
nologies. She reccived the B.A. degree in Asian stud-
ies from Skidmore College in 1995, the Diploma
degree in international relations from The Johns Hop-
kins University in 2000, the MIA degree in international
affairs from Columbia University in 2001, and the
Ph.D. degree in communications from Columbia Uni-
versity in 2008. Contact her at lforlano@id.iit.edu.
November/December 2020 49
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.

John Fass is currently the Course Leader for MA
User Experience Design at London College of Com-
munication, and runs decomputation, an elective in
information experience design with the Royal College
of Art. He sits on the Program Committee for
Research Through Design and received doctoral
funding from the AHRC followed by a Postdoctoral
Fellowship Award. His current research interests
include methods of externalisation, post humanist
design methods, and interface ethics. He recived the
B.A. degree in photography, film, video, and anima-
tion from West Surrey College of Art and Design in
1992, the MRes degree in information environments
with distinction from London College of Communica-
tion in 2011, and the Ph.D. degree in communication
design from Royal College of Art in 2018. He is a
member of the Design Research Society and IxDA.
He is the corresponding author of this article. Contact
him at j.fass@arts.ac.uk.
Lisa Brawley is the Director of the Urban Studies
Program, Vassar College, following a three-year
appointment as the Anne McNiff Tatlock Chair of Mul-
tidisciplinary Studies. She received doctoral funding
from the Whiting Foundation. She taught American
Studies with Loyola University Chicago and Kent
State University before joining the Urban Studies and
American Studies Programs at Vassar College in
2000. She is coauthor of a bilingual book on urban
design and agricultural modernity, D’ apr 
es nature:
Frederick Law Olmsted et le Park movement
am 
ericain (2014). She edited Postmodern Culture
for nine years and now serves on its advisory board.
She serves on the Scientific Board of the journal
Justice Spatiale/Spatiale Justice. Her research
addresses cultural histories of capitalist urbanization
in the U.S. with a current focus on participatory
design methods, feminist spatial practices, and the
just transition movement. She is a member of the
Design Research Society and the American Studies
Association. She received the B.A. degree in studio
art and feminist theory from the Center for Interdisci-
plinary Studies at Davidson College in 1982, the M.A.
degree in performance studies from New York Uni-
versity in 1986, and the Ph.D. degree in American
studies from the Committee on Ideas and Methods,
University of Chicago in 1995. Contact her at
lbrawley@vassar.edu.
Data Physicalization
50 IEEE Computer Graphics and Applications
Authorized licensed use limited to: Carnegie Mellon Libraries. Downloaded on October 25,2020 at 16:43:06 UTC from IEEE Xplore. Restrictions apply.