Content uploaded by Edward Melcer
Author content
All content in this area was uploaded by Edward Melcer on Oct 12, 2017
Content may be subject to copyright.
Motion, Emotion, and Form: Exploring
Affective Dimensions of Shape
Abstract
In this paper, we present a study examining how
individuals embody emotion within form. Our findings
provide a general taxonomy of affective dimensions of
shape consistent with and extending previous
literature. We also show that ordinary people can
reasonably construct embodied shapes using affective
dimensions, and illustrate that emotion is conveyed
through both visual dimensions and tactile
manipulations of shape. Participants used three distinct
strategies for embodiment of emotion through shape:
the look of a shape (visual representation), creation of
a shape symbolizing the experience of an intended
emotion (metaphor), and by evoking the intended
emotion in the creator through affective movements
and manipulations during construction (motion). This
work ties together and extends understanding around
emotion and form in HCI subdomains such as tangible
embodied interaction, emotional assessment, and user
experience evaluation.
Author Keywords
Form; Affect; Emotion; Embodiment
ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g.,
HCI): Miscellaneous.
Permission to make digital or hard copies of part or all of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. Copyrights
for third-party components of this work must be honored. For all other
uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
CHI'16 Extended Abstracts, May 07-12, 2016, San Jose, CA, USA
ACM 978-1-4503-4082-3/16/05.
http://dx.doi.org/10.1145/2851581.2892361
Edward Melcer
New York University
Brooklyn, NY 11201, USA
eddie.melcer@nyu.edu
Katherine Isbister
University California, Santa Cruz
Santa Cruz, CA 95064, USA
katherine.isbister@ucsc.edu
Introduction
In HCI, research linking dimensions of shape to
phenomena such as emotion is fairly limited. Certain
visual dimensions of form have been shown to express
a wide range of emotions [9, 15, 21, 22, 25, 28], but
there is no unified understanding of which dimensions
relate to which emotions [20]. To address this gap, we
explore the relationship between shape and emotion,
providing a taxonomy of affective shape dimensions
and insight to how individuals embody emotion in form.
Related Work
In this section we present an overview of related work
on emotion and form, tangible interaction, models of
emotion, and emotional assessment tools. We note that
studies tend to focus on perception and modulation of
emotion rather than embodying it, indicating a novel
space for exploration through our study.
Emotion and Form
There has been a variety of work examining the
relationship between emotion and form. Poffenberger
studied stimulus-response mapping of adjectives to
small, medium, and large angular or sinusoidal waves
going in flat, ascending, or descending directions [25].
These results were replicated by Collier and mapped
onto valence-arousal dimensions showing downward
facing lines represent low arousal emotions, upward
facing lines represent high arousal, angular lines
represent negative valence, and smooth curves
represent positive valence [9]. Interestingly, the
expression of emotion through curvature and
orientation of form appears to apply more broadly to
complex two-dimensional figures such as the Disney
flour sack [28] which can convey a range of emotions
without clearly defined human form (see Figure 1).
Also of importance is the somewhat generalizable
affective dimensions of three-dimensional shapes found
by Isbister et al through development of their Sensual
Evaluation Instrument. They identified that rounded
shapes correspond to positive valence, spiky shapes to
negative valence, smooth shapes to low arousal, and
protruding surfaces to high arousal [14, 15].
Recent work has also been done in the mapping of
words to form through a CAD tool (EmotiveModeler) for
emotive design [22, 23]. By typing an emotion-based
word into the tool, a three-dimensional shape will be
generated based on many of the affective dimensions
described above. Conversely, fuzzy logic models have
been used to perceive emotion from characteristics of
shape and validated through an empirical study with
design students and professionals [1].
Tangible Interaction
Tangible interaction can be thought of as encompassing
a range of systems and interfaces relying on embodied
interaction, physical representation and tangible
manipulation of data, and embeddedness in real space
[4, 13, 16]. The related work that deals with emotion
tends to focus on the use of tangible and embodied
interactions to evoke emotion (e.g., through physical
gestures [12]) or modulate it [2, 10, 17].
A Circumplex Model of Emotion
For understanding emotion, our work utilizes Russell's
circumplex model of affect (see Figure 2) which
proposes that all affective states can be described with
two dimensions, valence—the positive and negative
spectrum of emotional states—and arousal—the
energetic level of an emotion [26]. In this model,
emotion can be understood as linear combinations of
Figure 1: The Disney flour
sack expressing emotions
[28].
Figure 2: Russell’s circumplex
model of affect [26].
valence and arousal at varying degrees. Additionally,
this allows a more continuous representation of
emotion than earlier theories of basic emotions which
contained a discrete and limited set of emotions [26].
Recent work in emotion supports this notion of ‘core
affect’ comprised of valence and arousal [3].
Visual and Tangible Emotional Assessment Tools
Most emotional assessment tools rely entirely on words
or Likert scales. However, there are some pictorial tools
that utilize the relationship between form and emotion
for assessment. A prominent example is the Self-
Assessment Manikin (SAM) [6] which uses the
pleasure, arousal, dominance (PAD) model of emotion.
SAM depicts a graphic character arrayed along a nine-
point scale (see Figure 3). For the pleasure dimension,
the character is portrayed with a smiling face on one
end, a neutral face in the middle, and a frowning face
on the other end. The arousal dimension modulates
size and intensity of an explosion and the manikin's
apparent alertness through facial features. In the
dominance dimension, manikin size is modulated.
There is also one example of an emotional assessment
measure that uses 3D form. The Sensual Evaluation
Instrument (mentioned above) [15] provides users with
a set of hand-sized, ambiguous physical objects meant
to afford a channel of emotionally evaluative
communication through dialog between user and
designer (see Figure 4) [15, 18]. The creators of the
instrument documented consistencies in response of
users to shape characteristics across cultural contexts
[14]. However, it has been noted that the taxonomy of
the shapes is limited and participants have expressed a
need for additional affective dimensions and more
forms to better capture emotional nuances [15, 24].
Clay Shape Construction Study
The SEI tool provided us with inspiration for an
approach to conducting exploratory research about the
relationship of emotion and form. The SEI researchers
had an artist create their objects. Would non-experts
make use of similar form factors to convey emotions?
Considering the significance of tangibility and
collaboration in many affective domains [13, 15, 29],
we felt it was of interest to examine how individuals
and groups construct objects intended to convey
specific emotions through form, when given the tools to
do so. However, creating digital models with Computer
Aided Design (CAD) tools is a difficult task for most
users [7]. We decided to experiment with a modality
that was tangible and accessible to everyday creators
while remaining flexible in manipulations of form.
Plasticine clay—commonly used in clay animation—was
a natural choice since the material can maintain fine
grain details of shape, but has a low barrier to entry for
moderately skilled manipulation and use. Additionally,
mixed emotions (i.e., multiple emotions experienced
simultaneously [5]) are important to emotion research
[19, 20, 24], so we were curious if the complexity of
multiple emotions could be expressed through a single
shape. We hypothesized participants would accurately
construct shapes representative of emotion, but
anticipated difficulty during collaborative construction
due to the implicit, individual subjectivity of emotion.
We anticipated that many affective dimensions of shape
from related work [9, 15, 25] would be utilized in the
embodiment of emotion in clay shapes.
Methodology
PARTICIPANTS, EXPERIMENTAL SETUP, AND PROCEDURE
A total of 6 male and 8 female subjects (ages 22-51,
M=36.5) participated in the clay shape construction
Figure 3: Three points from the
Self-Assessment Manikin [6]. Top
left, from the pleasure/valence
dimension; top right, from the
arousal dimension; bottom, from
the dominance dimension.
Figure 4: The Sensual Evaluation
Instrument [15].
study. Of the 14 participants, 11 reported prior
experience with clay based activities, and no
participants were professional designers. For the
experiment, we placed two desks in separate rooms
with a 5 pound block of plasticine clay. A webcam was
also placed several feet above each desk to capture
top-down video and audio from the construction and
interview portions of the test.
Each test was conducted with two participants. Before
beginning, we randomly chose 1 of 4 emotional sets for
the procedure (see Table 1). Each set was designed to
encompass a broad spectrum of high and low
arousal/valence emotions that would fall in different
quadrants of Russell's circumplex model. During phases
with clay construction, participants were instructed to
think-aloud by describing their thoughts, actions, and
feelings as they worked. When working individually,
participants were placed in separate rooms and when
working collaboratively, participants were placed at a
shared desk in the same room. The study consisted of
6 experimental phases (see Table 2) over about one
hour. After testing, audio recordings were transcribed
and analyzed. Basic text analysis was used to identify
shape descriptions with respect to an emotion and
count the frequency of shape description occurrence.
Descriptions were later clustered by theme and emotion
to create a taxonomy of affective dimensions.
Results and Discussion
The clay shape construction study generated a total of
42 clay shapes among the 14 participants (see Figure 5
for a broad sampling). Of that, 35 shapes were thought
to accurately represent their intended emotion by at
least one creator. During creation of shapes, several
themes emerged around the embodiment of emotion.
AFFECTIVE SHAPE DESCRIPTIONS: AROUSAL AS OVERALL
STRUCTURE AND VALENCE AS FINE-GRAINED DETAIL
Audio recordings were taken during the shape creation
process and later analyzed to identify themes in the
descriptions of shape and its relationship to certain
emotions. The themes around common affective
dimensions of shape described by participants were
then synthesized into a general taxonomy (see Figure
6). Shape descriptions that often appeared in usage for
emotions of similar arousal levels (i.e., frustration and
excitement, boredom and contentment) were put in
separate high/low arousal categories.
Our taxonomy identifies many known affective
dimensions of shape, such as spiky shapes conveying
frustration and round shapes conveying contentment. It
also identifies many new dimensions, such as flatness
conveying boredom or symmetry conveying
contentedness. Importantly, we found that the
expression of emotion through form is not as simple as
just combining different affective dimensions into one
shape. Instead, both visually and in the usage of
descriptions, the arousal of a shape tended to be
expressed through its overall structure (e.g., the entire
shape is big, has visual variation, etc.) while valence
tended to be expressed through fine-grained detail of
elements within the shape (e.g., the shape contains
some arches and spikes that extend upwards).
VISUAL REPRESENTATION, METAPHOR, AND MOTION
When constructing shapes, we found that participants
embodied emotion in three distinctly different ways: (1)
through the look of a shape (visual representation), (2)
through creation of a shape symbolizing the experience
of an intended emotion (metaphor), and (3) by evoking
the intended emotion in the creator through affective
Single
Emotion
Mixed
Emotion
1
Contentment
Disinterest
& Surprise
2
Boredom
Excitement &
Frustration
3
Excitement
Boredom &
Frustration
4
Frustration
Contentment
& Boredom
Table 1: Emotional sets used for
testing.
Study Phases
1
Practice replication of two
Sensual Evaluation
Instrument objects
individually
2
Individual construction of
a single emotion
3
Collaborative construction
of a single emotion
4
Individual construction of
a mixed emotion
5
Collaborative construction
of a mixed emotion
6
Posttest semi-structured
interview
Table 2: Experimental phases of
the study.
movements and manipulations during construction of a
shape (motion). Of the 42 shapes constructed, 15
shapes were created by 12 participants using visual
representation for embodiment; 21 shapes were
created by 13 participants using metaphor; and 6
shapes were created by 6 participants using motion.
During embodiment through visual representation,
focus is placed on visual features of the shape itself
that represent and evoke an emotion in the viewer
(e.g., a flat shape for boredom or a spiky shape for
frustration). For one such occurrence, P11 notes their
use of visual representation to embody boredom in
their object (Figure 7 left): "...I don't think it should
have any curves or even things protruding out of it. But
just sort of flat and consistently the same size so that
nothing really catches your eyes" (P11, Boredom).
During embodiment through metaphor, focus is
narrative driven and relies on the shape symbolically
representing a person or object experiencing the
intended emotion (e.g., curving a shape inwards to
represent a person curled up from boredom or trapping
an object in an enclosed space to show frustration). For
instance, P10 describes how they embodied
contentment in their object through metaphor (Figure 7
middle): "Now I'm gonna make a ball that lives in a
bowl. He's totally happy... It's very comfortable. I bet
it's really soft. So he can get out if he wants to. He's
not confined" (P10, Contentment).
For embodiment through motion, focus is on utilizing
the construction process to evoke an intended emotion
in the creators themselves. Motions and manipulations
that modulate the builder's internal emotion (e.g.,
repeated rolling to feel boredom, or tearing and
smashing the clay to feel anger) are key to this form of
embodiment. When constructing a shape representative
of boredom, P4 embodied boredom through the act of
repeatedly rolling and folding clay until they felt bored
(Figure 7 right): "I rolled it out into a flat line because I
couldn't think of anything more monotonous than
rolling clay. And then when it got long and narrow, I'm
bored. Maybe I'll just fold it. And I just kept folding and
folding and folding" (P4, Boredom).
Figure 7: Shapes constructed with three different forms of
embodiment. Left is boredom embodied through visual
representation. Middle is contentment embodied through
metaphor. Right is boredom embodied through motion.
Figure 5: A taxonomy of affective dimensions of shape identified from the clay shape study.
Figure 6: A broad sampling of
affective shapes created during
the clay shape study.
Individual vs. Collaborative Construction
Preference for individual or collaborative construction
was split among participants: 6 individual, 6
collaborative, and 2 liked both equally. However, all
participants noted contrasting benefits to each
approach. The individual benefits described were a
more emotionally pure experience, greater creativity,
higher accuracy representing emotions, and ease of
constructing the intended shape. Conversely, the
collaborative benefits described were increased
emotional perspective and context, modularity of the
construction process, and greater enjoyment overall.
Far more dialog on emotion was also generated during
collaborative conditions as participants had to navigate
social dynamics, explain their thoughts, and justify
every potential manipulation to the shared shape.
One notable problem participants had was difficulty in
successfully creating mixed emotion shapes and shapes
through metaphor. Although the overall success rate
for construction of shapes (i.e., at least one creator felt
a shape accurately represents the intended emotion)
was reasonable at 83%, the success rates for
collaborative construction of mixed emotion and
metaphor shapes was much lower at 58% and 64%
respectively. This poor result reflects the common
disagreement between participants on how shapes
should be manipulated based on narrative or a complex
mixed emotion representation; often leading to shapes
that neither participant was happy with.
Discussion
Based on participant discussions of emotion and
success rates from the results, it appears that ordinary
individuals can embody emotion within form to some
extent. While many of our hypotheses about these
results were intuitive, we found one unexpected aspect
of embodiment: the importance of motion and tactile
manipulation for the expression of emotion in form.
When creating affective shapes, embodiment of
emotion occurred through visual representation,
metaphor, and motion. The use of motion-oriented
forms of embodiment with visual forms helps tie
together understanding of the relationship between
motion, emotion, and form in domains such as tangible
embodied interaction [4, 11–13, 27], user experience
evaluation [29], and emotional assessment [8, 15, 18,
20]. There are also parallels between current research
on emotion in form and our results. Many affective
dimensions of shape identified in our study (see Figure
6) match, closely parallel, or extend affective
dimensions found in other studies. Specifically, what is
legible to individuals are dimensions manipulating core
affect [3]; such as sharp angled edges represent
negative valence while rounded edges represent
positive valence [9, 15, 25], and smooth surfaces
represent low arousal while visual variation/various
extrusions represent high arousal [15].
Future Work
Moving forward, we hope this work will provide
theoretical underpinnings for future projects and
affective design endeavors in more hedonically oriented
HCI domains such as TEI, mood modulation, user
experience evaluation, and emotional assessment. We
believe our taxonomy describing affective dimensions
of shape will aid in the creation of CAD tools that afford
users quick and simple affective manipulations during
the modeling process. We also feel this work naturally
translates to the design and evaluation of tangible tools
meant to evoke emotions or allow users to express
their internal state.
References
1. Achiche, S. and Ahmed-Kristensen, S. 2011. Genetic
fuzzy modeling of user perception of three-dimensional
shapes. Artificial Intelligence for Engineering Design,
Analysis and Manufacturing. 25, 01 (2011), 93–107.
2. Alonso, M.B. et al. 2008. Squeeze, rock, and roll; can
tangible interaction with affective products support
stress reduction? Proceedings of the 2nd international
conference on Tangible and embedded interaction -
TEI ’08. (2008), 105.
3. Barrett, L.F. 2014. The Conceptual Act Theory: A
Précis. Emotion Review. (2014), 1–20.
4. Baskinger, M. and Gross, M. 2010. Tangible interaction
= form + computing. Interactions. 17, 1 (2010), 6.
5. Blythe, M.A. et al. 2003. Funology: from usability to
enjoyment.
6. Bradley, M. and Lang, P.J. 1994. Measuring Emotion :
the Self-Assessment Semantic Differential Manikin and
the. Journal of behavior therapy and experimental
psychiatry. 25, I (1994), 49–59.
7. Carrington, P.A. et al. 2015. “Like This , But Better”:
Supporting Novices ’ Design and Fabrication of 3D
Models Using Existing Objects. iConference 2015
(2015).
8. Castellano, G. et al. 2007. Recognising Human
Emotions from Body Movement and Gesture
Dynamics. Affective Computing and Intelligent
Interaction. (2007), 71–82.
9. Collier, G.L. 1996. Affective synesthesia: Extracting
emotion space from simple perceptual stimuli.
Motivation and Emotion. 20, 1 (1996), 1–32.
10. Desmet, P.M.A. 2015. Design for Mood : Twenty
Activity-Based Opportunities to Design for Mood
Regulation. International Journal of Design. 9, 2
(2015), 1–19.
11. Djajadiningrat, T. et al. 2004. Tangible products:
Redressing the balance between appearance and
action. Personal and Ubiquitous Computing. 8, 5
(2004), 294–309.
12. Fagerberg, P. et al. 2004. EMoto: Emotionally
engaging interaction. Personal and Ubiquitous
Computing. 8, 5 (2004), 377–381.
13. Hornecker, E. and Buur, J. 2006. Getting a grip on
tangible interaction. Proceedings of the SIGCHI
conference on Human Factors in computing systems -
CHI ’06 (2006), 437.
14. Isbister, K. et al. 2007. The sensual evaluation
instrument: Developing a trans-cultural self-report
measure of affect. International Journal of Human
Computer Studies. 65, 4 (2007), 315–328.
15. Isbister, K. et al. 2006. The Sensual Evaluation
Instrument: Developing an Affective Evaluation Tool.
CHI 2006 Proceedings (2006), 1163–1172.
16. Ishii, H. and Ullmer, B. 1997. Tangible bits: towards
seamless interfaces between people, bits and atoms.
Proceedings of the SIGCHI conference on Human
factors in computing systems (1997), 241.
17. Karlesky, M. and Isbister, K. 2014. Designing for the
Physical Margins of Digital Workspaces: Fidget Widgets
in Support of Productivity and Creativity. Proceedings
of the 8th International Conference on Tangible,
Embedded and Embodied Interaction - TEI ’14 (2014),
1–8.
18. Laaksolahti, J. et al. 2009. Using the Sensual
Evaluation Instrument. Digital Creativity. 20, 3 (2009),
165–175.
19. Larsen, J.T. and Mcgraw, A.P. 2014. The Case for
Mixed Emotions. Social and Personality Psychology
Compass. (2014), 263–274.
20. Melcer, E. and Isbister, K. 2014. CSEI : The
Constructive Sensual Evaluation Instrument.
Workshop on Tactile User Experience Evaluation
Methods at CHI 2014 (2014).
21. Melcer, E. and Isbister, K. 2014. Emotional Space:
Understanding Affective Spatial Dimensions of
Constructed Embodied Shapes. 2nd ACM symposium
on Spatial user interaction (2014), 143.
22. Mothersill, P. 2014. The Form of Emotive Design.
Massachusetts Institute of Technology, Cambridge,
MA.
23. Mothersill, P. and Bove Jr, V.M. 2015. EmotiveModeler.
Interactions.
24. Obrist, M. et al. 2014. Temporal, Affective, and
Embodied Characteristics of Taste Experiences: A
Framework for Design. Proceedings of the 32Nd
Annual ACM Conference on Human Factors in
Computing Systems (2014), 2853–2862.
25. Popfenberger, A.T. and Barrows, B.E. 1924. The
Feeling Value of Lines. Journal of Applied Psychology.
(1924).
26. Russell, J.A. 1980. A Circumplex Model of Affect.
Journal of personality and social psychology. 39, 6
(1980), 1161–1178.
27. Sonneveld, M.H. 2007. Aesthetics of tactual
experience. Delft University of Technology (TU Delft).
28. Thomas, F. and Johnston, O. 1995. The Illusion of
Life: Disney Animation.
29. Tscheligi, M. et al. 2014. “Touch me”: workshop on
tactile user experience evaluation methods. Proc. CHI
EA. (2014), 41–44.
