Article

Embracing First-Person Perspectives in Soma-Based Design

Abstract

A set of prominent designers embarked on a research journey to explore aesthetics in movement-based design. Here we unpack one of the design sensitivities unique to our practice: a strong first person perspective—where the movements, somatics and aesthetic sensibilities of the designer, design researcher and user are at the forefront. We present an annotated portfolio of design exemplars and a brief introduction to some of the design methods and theory we use, together substantiating and explaining the first-person perspective. At the same time, we show how this felt dimension, despite its subjective nature, is what provides rigor and structure to our design research. Our aim is to assist researchers in soma-based design and designers wanting to consider the multiple facets when designing for the aesthetics of movement. The applications span a large field of designs, including slow introspective, contemplative interactions, arts, dance, health applications, games, work applications and many others.
informatics
Article
Embracing First-Person Perspectives in
Soma-Based Design
Kristina Höök 1, *ID , Baptiste Caramiaux 2,3,4, Cumhur Erkut 5ID , Jodi Forlizzi 6,
Nassrin Hajinejad 7, Michael Haller 8, Caroline C. M. Hummels 9, Katherine Isbister 10,
Martin Jonsson 11 ID , George Khut 12, Lian Loke 13 ID , Danielle Lottridge 14 , Patrizia Marti 9, 15 ID ,
Edward Melcer 16, Florian Floyd Müller 17 , Marianne Graves Petersen 18, Thecla Schiphorst 19 ,
Elena Márquez Segura 10, Anna Ståhl 20 , Dag Svanæs 21, 22, Jakob Tholander 23
and Helena Tobiasson 24
1Media Technology and Interaction Design (MID), Royal Institute of Technology (KTH),
100 44 Stockholm, Sweden
2UMR STMS Ircam-CNRS-UPMC, 1 pl. Igor Stravinsky, 75004 Paris, France; baptiste.caramiaux@ircam.fr
3Schulich School of Music, McGill University, Montreal, QC H3A 1E3, Canada
4
CNRS—LRI, Univ. Paris-Sud, Inria, University Paris-Saclay, Bât 650 Noetzlin st., 91190 Gif-sur-Yvette, France
5Department of Architecture, Design & Media Technology, Aalborg University, 2450 Copenhagen SV,
Denmark; cer@create.aau.dk
6Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA;
forlizzi@cs.cmu.edu
7Institute of Informatics and Automation, Hochschule Bremen, City University of Applied Sciences,
28199 Bremen, Germany; nassrin.hajinejad@hs-bremen.de
8School of Informatics, Communications and Media, Upper Austria University of Applied Sciences,
4232 Hagenberg, Austria; haller@fh-hagenberg.at
9Department of Industrial Design, University of Technology, 5612 AZ Eindhoven, The Netherlands;
c.c.m.hummels@tue.nl (C.C.M.H.); marti@unisi.it (P.M.)
10 Department of Computational Media, University of California, Santa Cruz, CA 95064, USA;
katherine.isbister@ucsc.edu (K.I.); elena.marquez@ucsc.edu (E.M.S.)
11 Department of Natural Sciences, Technology and Environmental Studies, Södertörn University,
141 89 Huddinge, Sweden; martin.jonsson@sh.se
12 UNSW Art & Design, University of New South Wales, Sydney 2052, Australia; g.khut@unsw.edu.au
13 Sydney School of Architecture, Design and Planning, University of Sydney, Sydney 2006, Australia;
lian.loke@sydney.edu.au
14 Yahoo Inc., Sunnyvale, CA 94089, USA; dlottridge@yahoo-inc.com
15 Department of Social, Political and Cognitive Science, 53100 Siena, Italy
16 New York University Tandon School of Engineering, New York University, Brooklyn, NY 11201, USA;
eddie.melcer@nyu.edu
17 Exertion Games Lab, RMIT University, Melbourne, VIC 3000, Australia; floyd@exertiongameslab.org
18 Department of Computer Science, Aarhus University, 8000 Aarhus, Denmark; mgraves@cs.au.dk
19 School of Interactive Arts and Technology, Simon Fraser University, Burnaby, BC V5A 1S6, Canada;
thecla@sfu.ca
20 RISE SICS, 164 40 Kista, Sweden; anna.stahl@ri.se
21 Department of Computer Science (IDI), Norwegian University of Science and Technology (NTNU),
NO-7491 Trondheim, Norway; dag.svanes@idi.ntnu.no or dasv@itu.dk
22 Digital Design Department, IT-University of Copenhagen, DK-2300 Copenhagen, Denmark
23 Department of Computer and Systems Sciences (DSV), Stockholm University, 114 18 Stockholm, Sweden;
jakobth@dsv.su.se
24 Department of Public Health and Clinical Medicine, Umeå University, 901 87 Umeå, Sweden;
helena.tobiasson@umu.se
*Correspondence: khook@kth.se; Tel.: +46-7-0561-7035
Received: 9 November 2017; Accepted: 19 January 2018; Published: 1 February 2018
Informatics 2018,5, 8; doi:10.3390/informatics5010008 www.mdpi.com/journal/informatics
Informatics 2018,5, 8 2 of 26
Abstract:
A set of prominent designers embarked on a research journey to explore aesthetics in
movement-based design. Here we unpack one of the design sensitivities unique to our practice:
a strong first person perspective—where the movements, somatics and aesthetic sensibilities of the
designer, design researcher and user are at the forefront. We present an annotated portfolio of design
exemplars and a brief introduction to some of the design methods and theory we use, together
substantiating and explaining the first-person perspective. At the same time, we show how this felt
dimension, despite its subjective nature, is what provides rigor and structure to our design research.
Our aim is to assist researchers in soma-based design and designers wanting to consider the multiple
facets when designing for the aesthetics of movement. The applications span a large field of designs,
including slow introspective, contemplative interactions, arts, dance, health applications, games,
work applications and many others.
Keywords:
movement-based interaction; somaesthetic design; aesthetics; somatics; first-person
perspective
1. Introduction
We are currently experiencing a rise in movement-based design, both commercially and in
academic explorations. This trend follows the development of new interaction technologies—sensing,
actuating, wireless connectivity—together forming a rich space of design possibilities. As we will
argue here, these movement-based designs can draw upon the history of movement practices that
predate the academic field of Human-Computer Interaction (HCI) [1].
Some predict that the domain of body-, movement- and biosensor-based [
2
] interactions could be
as big as or even bigger than desktop and mobile. They will be reaching beyond application areas in
HCI where physical coordination and learning-by-doing are naturally important (such as movement
rehabilitation, dance movement therapy, movement education including dance and sports training,
self-cultivation, yoga, emotion regulation for management of stress and anxiety related health problems,
and similar) into areas that do not yet see aesthetics of movement as a fundamental cornerstone in
their design processes (such as communication within multi-stakeholder design processes, interaction
with everyday Internet of Things applications, quantified self and health apps, elderly homecare).
As interactive technologies proliferate in everyday life, they start to shape our cultural expressions
and, furthermore, they start to shape us. Technologies transform our experience through amplifying or
reducing reality, and by translating our behavior through inviting and inhibiting actions [
3
]. We see
it already in technologies such as the smartphone, which has altered everything from how we meet,
socialize, or play, to even changing our bodily movements (e.g., [
4
]). We inscribe our understanding
of our bodily selves into them and they in turn shape us, what we do and how we do them, both in
terms of practices, but also in a literal, corporeal sense. These technologies will (depending on how
we design them) encourage certain movements, certain aesthetic experiences, certain practices and
understandings of our bodies—while not encouraging others. They will influence our availability for
certain qualities of interaction and not others. They will alter how we move, our postures, our muscles
and nervous system reactions and orientations. This will continue to affect how we do our work tasks
as well as our leisure activities, social communication, play, and artistic expressions, and therefore
ultimately, how we live. This is why we need to cultivate our understanding of what it means to be a
sensing, feeling and moving body, living in the world, shaping and being shaped by our lifeworld.
It is worth noting how many commercial design attempts fall short in delivering on the promise
of movement-based interaction. Movement interfaces, such as Wiimote or Kinect, created a lot of
excitement, but it has been notoriously difficult to design games and other applications for these
platforms [
5
7
]. Likewise, we have seen an increased interest in virtual and augmented reality [
8
,
9
],
but again, only few applications last past the initial excitement. We might argue that this is fine—many
Informatics 2018,5, 8 3 of 26
games and mobile apps are short-lived—but there seems to be a missed opportunity in sustaining
longer-term experiences, and even more importantly, there is an opportunity to expand beyond what
these first technologies have provided so far.
The recommendations in this paper flow from a workshop held in conjunction with the ACM
SIGCHI-conference in 2016, involving some of the most prominent researchers in the field, coming
together to work out the most promising and rigorous strategies for engaging in soma-based design.
By soma-based design we mean a large space of movement-based designs and design practices
specifically engaging with aesthetics, which will be further explicated over the course of this paper.
We noted that we had many joint experiences and insights, as well as concerns. We compared our
approaches to design and found many common traits: certain methods, certain practices, certain
sensitizing exercises, certain orienting conceptualizations—together forming a systematic way of
approaching design, providing us with rigor in our research. In this paper, we aim to make two
contributions: we will lay out the research area, and second we will share what we mean by rigor
in our work through explaining and unpacking one of the most prominent traits of our practice:
afirst-person perspective on movement-based design and why it is crucial in this area.
As this work has its home in Human-Computer Interaction (HCI) we use the term ‘user’ to denote
the person who will, in the end, use the system created. This term has been much debated in the
community, as it communicates a passive stance, that is someone who is given a design and not an
active co-constructor of the interaction. In reality, whenever a system is realized in the world—being
used—it continues to be ‘designed’ by the interpretations and choices of the end-user, filling it with
meaning and interactions, often filling it with content [
10
,
11
]. In addition, most systems these days are
in a perpetual beta-state, where they are being redesigned while being used in response to end-user
behaviors. The line between being a designer and being a user is blurred. This perpetual beta-state
also means that a system is never entirely finished—it is a work in progress, continuously shaping and
being shaped by user activities. In the context of the work presented here, the term actor, participant
or co-constructor would have been more appropriate. The end-user will actively co-construct their
experience, the meaning of the interactions, together shaping practices and engagements over time.
But as actor is not a well-established term in HCI, we have decided to continue to use user.
Given this richer understanding of what we mean by user, in short, a first-person perspective
places the user’s lived experience at the core of the design process—the lived experience of moving and
being moved become the main unifying activity during the design process. In particular, by using the
concept soma (drawing upon Hannah [
12
] and Shusterman [
13
]), we emphasize the living,purposive,
sentient,perceptive body or bodily subjectivity of the first-person perspective. We do not shun from
engaging with our own subjective feelings and passions. If one wants to design for the lived experience
of moving and being moved, designers have to be engaged in their own somatics by doing and
experiencing while designing [
14
]. As designers, we must be able to distinguish between all the fine
nuances of different movements, emotional experiences, tactile engagements, or mirrorings of our
bodily processes in interactive design. We also need to feel, touch, interact with, and experience the
affordances of the “materials” (digital or other materials) we use to compose the dynamic gestalt of the
interaction [15].
We unpack and substantiate the first-person perspective through three accounts. First, we
provide an annotated portfolio [
16
] through first-person accounts of design exemplars. The design
exemplars—or ultimate particulars in the words of Stolterman [
11
]—serve as definitions made through
design [
17
,
18
] in a Research through Design (RtD) practice [
6
,
18
27
]. The design exemplars serve
two purposes here: first, they show how our first-person perspective is able to generate viable,
functioning, qualitatively different design work feeding off our movements and bodily engagements;
second, through providing first-person accounts of use we provide a glimpse of the kind of somatic
engagement they enable.
In our second account of the first-person perspective, we share some of the soma design methods
employed by the authors. At the core of each method lies a double first-person engagement: on the one
Informatics 2018,5, 8 4 of 26
hand, empathically shaping experiences for future users through our own, first person, soma-based
engagements, and at the same time changing our soma-based aesthetic appreciation skills as designers.
This double engagement requires authenticity and personal engagement.
In our third account, we provide some of the theoretical underpinnings and contributions from
our work, building on concepts from pragmatism and somaesthetic theories.
Taken together, these three accounts substantiate what we mean by rigor in soma-based design.
With a first-person perspective, there is no short-cut, no simplified wireframes, no language-based
distancing between designers and what is being designed. The only path to design is through
thoroughly living your own designs, experiencing them on a profound level, empathically engaging
with the somatic experiences of your future users.
2. Background
Let us start by providing a brief background on the first-person perspective; providing theoretical
underpinnings, as well a review of how it has already been used in HCI and interaction design research
and practice. Second, it is helpful to ground the discussion that took place with terminology that
the workshop participants drew upon. Here, we provide background on concepts that surround the
soma-based design space: aesthetics, somatics, and somaesthetics.
2.1. First-Person Perspective: Theoretical Underpinnings
A number of research projects on soma-based interaction have used Merleau-Ponty’s phenomenology
of the body [
28
] as a theoretical backdrop (e.g., [
29
,
30
]). Merleau-Ponty makes a distinction between
the first-person perspective on the body, the lived body, the body through which we live our lives
and experience the world, and the third-person perspective of seeing the body as an object in the
world. In German there are two terms for the body, Leib and Körper, corresponding to the first- and
the third-person perspective, respectively, while in English we use body for both. In addition to the
first person and the third person perspectives on the body, there is also the interpersonal second
person perspective of empathically experiencing the body of the other through our own bodies. In [
31
],
Merleau-Ponty goes deeper into how our bodies are in this way interconnected through body language
and gaze.
Merleau-Ponty’s phenomenology of the body differs from Dewey’s aesthetics [
32
] in that he was
not primarily concerned with experiences that require a certain framing, such as aesthetic experiences,
but with the body in “unframed” everyday living. In everyday living, the meaning of a certain
movement, like turning your head, differs with context. The same head movement could be part of
watching out for cars while crossing a street, or part of a headshake to signal disagreement. From a
phenomenological perspective it thus becomes meaningless to ask for the meaning/experience of a
specific body movement decontextualized from the totality of the life of that person. As with all design,
it is important to be aware of the multiplicity of possible bodily user experiences that can arise from a
product in use, depending on the users, their tasks and use situations. While first- and second-person
perspectives are crucial for successful soma-based design, such practices do not eliminate the need to
explore in depth the different contexts-of-use for the product and be open to the fact that it might be
used in ways we as designers can never imagine.
2.2. The First-Person Perspective in HCI Studies and Design
Different perspectives on the body, third-person (I-It), second-person (I-You) or first-person (I-Me),
inspire different approaches to design. Most user-centered design projects are done from a third-person
perspective; observing, interviewing, and testing on users, but not stepping into their shoes (even if,
e.g., Beuthel and Wilde attempt to literally feel someone else’s disability or bodily condition to inform
the design process [
33
]). In contrast, the first-person approach that is exemplified by the cases in this
paper uses the designer’s lived body as a resource in the design process. Doing this requires a certain
Informatics 2018,5, 8 5 of 26
practiced sensitivity to the kinesthetic “feel” dimension of interaction design [
29
,
34
], and allows for a
close loop between design and user experience.
The first-person perspective does, however, create a blindness about the ways in which our
lived bodies are different, and how these differences color our user experiences. One can imagine having
a different body, e.g., being short or tall, skinny or obese, but one cannot experience being another
body. The closest we get is through a second-person perspective on the body, where the designer uses
his or her body as an instrument for feeling the bodily aspects of the user experience of the other in
co-design practices.
The first-person engagement with design is not new to HCI. In ethnography, the first-person,
subjective perspective, has been used to introduce the observer’s/researcher’s subjective voice in
so-called autoethnographic accounts of experiences. In HCI, autoethnography has been a way to get at
bodily experiences. One example is in Höök’s study of her own horseback riding experiences and how
those could be translated into design [35].
In design work, the first-person perspective has been introduced in works on autobiographical
design [
35
37
]. As Neustaedter and Sengers frame it, such a design process “rest(s) on the user’s genuine
need for the system”, which allows for “a much tighter coupling between user input and implementation” [
38
].
Beyond using their own needs as input, the experts who participated in the writing of this paper
have all used design methods that not only start from their own needs, but strongly engage with their
own somatics, tying their work to the felt dimensions of movement.
2.3. Aesthetics—Building on the Pragmatist Movement
Aesthetics is a way to examine connections between sensation, feeling, emotion and subjective
understanding and values. A shared starting point for many of the authors of this paper lies in the
writings on experience and aesthetics of pragmatist philosopher John Dewey [
32
] (often used to define
UX in HCI-literature, e.g., [
39
42
]). Dewey saw an aesthetic experience as related to, but also set apart
from, the general flow of experiences in our everyday life, with a clear beginning and end. Using the
analogy of a mountain peak that emerges from the land around it, aesthetic experiences represent an
intensification of life experiences. Significantly for Dewey, an aesthetic experience was something that
we can refer to afterwards in definite terms: “An experience has a unity that gives it its name, that meal, that
storm, that rupture of a friendship. The existence of this unity is constituted by a single quality that pervades the
entire experience in spite of the variation of its constituent parts” [
32
]. As any experience is always unique,
even the mundane, everyday ones, Dewey set aesthetic experience apart with his emphasis on “that”
meal, storm, or rupture of a friendship. This emphasis on aesthetic experience as something outside
the ordinary is problematic with regard to some of the somaesthetic design aims outlined below, as we
may also want to talk about the aesthetic experiences of the everyday, of the repeated engagement,
or experiences that are perhaps not so memorable that we can talk about them as “that” experience.
Note also how the beauty of an aesthetic experience is not equal to only the positive emotions or safe
experiences. Scary or disturbing experiences can be considered aesthetic as well.
A second insight we bring from Dewey’s account of aesthetic experience is how we can achieve
them—both as artists/designers and also as viewers/users. A little-known historical fact regarding
Dewey’s theoretical development is that he studied for 16 years with the somatic pioneer F.M.
Alexander, and this practice had an enormous influence on his pragmatic philosophical writings
about aesthetic experience [
43
]. Dewey saw aesthetic experience not as something that happens to
us, but as something we engage in, actively, subjectively. When we craft or engage with our aesthetic
experience, we should, according to Dewey, attempt to “empty” the material (out of which we build
the experience) of all its potential. We should make use of every element that is used to create it, letting
the different parts be explored and exploited to their fullest.
Informatics 2018,5, 8 6 of 26
2.4. Somatics and Somaesthetics: Bringing in a Stronger Emphasis on the Soma
Later philosopher and pragmatist Richard Shusterman further developed Dewey’s pragmatist
stance towards aesthetic experiences by introducing a theoretical framework he named somaesthetics [
44
].
By putting together the word soma, with aesthetics, to refer to our sensory appreciations, Shusterman
draws our attention to the importance of our bodily movements, sensations, and somatic training,
as part of our ways of being and thinking—but more importantly here, the aesthetics of those
experiences. We draw upon Hannah’s definition of somatics and soma here: “Somatics is the field which
studies the ‘soma’: namely the body as perceived from within by first-person perception. When a human being is
observed from the outside—i.e., from a third-person viewpoint—the phenomena of a human ‘body’ is perceived.
But, when the same human being is observed from the first-person viewpoint of his own proprioceptive senses,
a categorically different phenomenon is perceived: the human soma” [12].
In developing aesthetics of somatic experience, Shusterman’s aim was to take Dewey’s pragmatic
stance towards aesthetic experience even closer to the realm of life and practice through bringing
the living, sentient body into the equation. While Dewey and others had mentioned the body and
the lower cognitive skills” that enable us to experience beauty, their interpretation of aesthetics was not
fundamentally grounded in the movements of our bodies and the sensibilities developed through
somatic training—in contrast to the work by Shusterman: My concept of soma as a living, purposive,
sentient, perceptive body or bodily subjectivity provides an altogether different direction. We can, as I have tried
to demonstrate, improve our perceptual faculties through better use of the soma” [13].
In the second sentence, he points to the possibility of improving through “better use of the soma”.
While training your aesthetic sensibilities has been a long-standing aim for anyone interested in the
arts, Shusterman emphasizes that the route to improved aesthetic sensibilities goes through active
engagement through the soma.
Shusterman’s aim is grounded in the original purpose of being a philosopher: to improve oneself.
He does not take aesthetic experience as a given, an experience to be studied as an object out there,
but instead wants to engage us all in actively, creatively changing and improving our experiences.
By educating ourselves somatically, we are engaging in “the highest art of all—that of living better lives” [
13
].
Shusterman’s somaesthetics has been informed through a range of bodywork traditions and
methodologies, from yoga and tai chi to more contemporary somatic movement education methods
such as those developed by F.M. Alexander (Alexander Technique) [
45
] and Moshe Feldenkrais
(Feldenkrais Method) [
46
]—both of which focus on the use of touch and movement as information
for learning more effective coordination of body movement. These bodywork practices emphasize
skills that we have previously learned, often at a sensory-motor level, that have become habitual and
often below conscious awareness (as in walking or sitting)—which is in general a good thing, so we
can operate in daily life without having to consciously think about and plan every movement. But we
also adopt bad habits that can cause pain and loss of agency, making us unnecessarily restricted in our
movements, thus leading us to miss out on the richness and aesthetics of our own somatics.
Engaging in different bodywork practices to rediscover or reacquire skills can only happen
through first-person experience and reflection. Novel ways of engaging with our soma cannot be
‘put into us’—we cannot be moved into shape. The concept of self-agency in somatics practice is key.
Self-agency is the result of the reflective practice of self-observation coupled with intention. It is
only when we move that we can experience “being moved” emotionally, physically, and analytically,
through our own somas [
47
]. A limited repertoire of movement becomes a limited repertory of
experiences [46].
But what sets apart aesthetic movements of the soma from just any movement or any experience?
In his writings, Shusterman repeatedly comes back to the need to disrupt habits, to focus and become
more aware so as to be able to discern the different parts of a movement or emotion, feeling every
small change in your body. For example, training your breathing to achieve better focus, or correcting
your eating habits from what is “habitual” to shape new, more reflective, thoughtful actions. Returning
to Dewey, Shusterman discusses the difference between “routine, un-intelligent habit” and “intelligent
Informatics 2018,5, 8 7 of 26
or artistic habit” that is fused with thought and feeling. To Shusterman, the art of somatic reflection
and conscious control is thus itself a “refined intelligent habit emerging from and coordinating a background
of countless other habits that constitute the developing bundle of complex, unstable, opposing attitudes, habits,
impulses we call the self ” [44].
These discoveries, or learnings, may lead to better appreciation, richer more meaningful bodily
ways of being in the world, and more beauty. Shusterman talks about an “awakening” from mindless,
joyless behaviors (referring to Thibaud’s idea of an awakening). This training is important for artists
and designers to refine their designerly skills and consequently their designs, as well as for users to be
able to experience somaesthetics in interaction and appreciate richer bodily ways of being in the world.
That is, when we speak of the ‘user experience’ of some design, it will be an active somatic stance
towards engagement—a living, purposive, sentient, perceptive body or bodily subjectivity engaging
in meaning-making processes. The user experience is not determined in a first brief encounter with
some particular design, but is a co-constructed aesthetic appreciation that arises over time. It requires
an active engagement (both by designer and user), resulting in an improvement of perception, bodily
movement, emotion, intentionality and meaning-making. It aims to change those involved, increase
their skills.
3. Methods
The insights presented in this paper derive from two phases of investigation. First, we conducted a
workshop at the ACM SIGCHI annual conference [
23
] to explore the first-person perspective, attracting
some of the leading experts in the field of soma-based design. Second, during the writing of this paper,
we continued the analysis of theoretical concerns, engaged in ongoing dialog about our practice and
research methods, discussed design exemplars not presented at the workshop, and conducted an
extensive literature review. Overall, our research method can be characterized as a Research through
Design (RtD) analysis. Let us describe these activities in some more detail before we turn to the data
the two phases generated.
3.1. Move to Be Moved Workshop
The workshop was divided into two main parts. The first part involved sensitizing and
foregrounding somatic and experiential sensibility through a series of first person experiences.
For example, to hone the participants’ abilities to be aware of and be empathetic towards oneself
and others, we performed a Collaborative Walking exercise devised by one of the authors [
25
] (drawn
from the Body Weather performance training methodology [
48
]). All workshop participants stood
in a line, shoulder to shoulder, very close to one another, with a red thread joining the whole line,
from the first participant to the last, through their mouths, see Figure 1. We were asked to close our
eyes and then walk together, very slowly, without losing contact with our neighbours. After the
exercise, we were asked to articulate our experience, sharing any insights or discoveries on walking.
This transported us to the space of felt experiences—reminding us not only intellectually of why we
were there, but somatically. It also provided a shared somatic experience of performing a simple
movement exercise, but with heightened awareness. The exercise itself was selected for its close
similarity to the everyday act of walking, but reframed in an unusual configuration for the purpose of
defamiliarizing the habitual—a somatic principle we elaborate upon in Section 4.2.1.
To fully engage with the issues of designing for somatic experiences, several of the participants
had brought soma-based designs so that they could be tested, felt and discussed. We all engaged
in hands-on first-person experiences with each design. After testing a system by sensing, feeling,
listening to and participating in the interaction, and probing our own somatic responses, we were
ready to discuss key aspects of the design, the experience, or the design process with its designer.
One of the key aspects that we explored was to notice how the exploratory exercises and interactions
affected our physical state, and to bring that state into our dialogues and discussions. This deepened
Informatics 2018,5, 8 8 of 26
our sense of what it means to engage end-users in aesthetic, movement-based interactions—and made
us more honest.
Informatics 2018, 5, x FOR PEER REVIEW 8 of 26
Figure 1. Collaborative Walking at the workshop.
To fully engage with the issues of designing for somatic experiences, several of the participants
had brought soma-based designs so that they could be tested, felt and discussed. We all engaged in
hands-on first-person experiences with each design. After testing a system by sensing, feeling,
listening to and participating in the interaction, and probing our own somatic responses, we were
ready to discuss key aspects of the design, the experience, or the design process with its designer.
One of the key aspects that we explored was to notice how the exploratory exercises and interactions
affected our physical state, and to bring that state into our dialogues and discussions. This deepened
our sense of what it means to engage end-users in aesthetic, movement-based interactions—and
made us more honest.
3.2. Research through Design Analysis
The second half of the workshop (and the writing of the paper post-workshop) was spent in
discussion, aimed at articulating aspects of our research, such as key aesthetic and material
considerations, design methods and theoretical perspectives, as well as views on the politics of the
body and its role in interaction design.
One reason HCI struggles with the first-person perspective lies in how we ‘prove’ the validity
of our knowledge contributions in the academic HCI and interaction design fields. Traditionally,
HCI, given its heritage from cognitive psychology, has put emphasis on the final user study as the
ultimate—objective—proof that your design does what it claims. Replicable user studies are seen as a
sign of quality [49]. But in addition to this view, the RtD community has successfully argued that the
process of designing can serve a similar role to that of the “final user study”. In the design process,
we follow different design paths, some ending in failure, some in success, occasionally obvious
through involving end-users in testing early ideas, but often the failure or success can already be
determined when sketching. The decisions rely on design judgments [50], on our ability to imagine
future interactions, on our feel for aesthetics, and on our empathy with a future imagined user [26].
Figure 1. Collaborative Walking at the workshop.
3.2. Research through Design Analysis
The second half of the workshop (and the writing of the paper post-workshop) was spent
in discussion, aimed at articulating aspects of our research, such as key aesthetic and material
considerations, design methods and theoretical perspectives, as well as views on the politics of
the body and its role in interaction design.
One reason HCI struggles with the first-person perspective lies in how we ‘prove’ the validity
of our knowledge contributions in the academic HCI and interaction design fields. Traditionally,
HCI, given its heritage from cognitive psychology, has put emphasis on the final user study as the
ultimate—objective—proof that your design does what it claims. Replicable user studies are seen
as a sign of quality [
49
]. But in addition to this view, the RtD community has successfully argued
that the process of designing can serve a similar role to that of the “final user study”. In the design
process, we follow different design paths, some ending in failure, some in success, occasionally obvious
through involving end-users in testing early ideas, but often the failure or success can already be
determined when sketching. The decisions rely on design judgments [
50
], on our ability to imagine
future interactions, on our feel for aesthetics, and on our empathy with a future imagined user [
26
].
Sketching and form-giving become the analytical thinking process. The design process becomes the
evidence of why the final result is valid [27,51,52].
As the material we design in is best framed as a socio-digital material [
53
,
54
]—consisting of the
various digital and other materials we use to create our design, and at the same time speaking of a use
practice and behavior unfolding over time—a design cannot be understood solely by inspecting it as a
static object. It comes to life as we start making it part of our practice.
Informatics 2018,5, 8 9 of 26
In RtD, specific design exemplars serve as ‘definitions’, ‘facts’ or as a physical hypothesis [
55
],
organized in design research programs (in design studios) [
10
] or created for paradigms [
56
], used
to extract knowledge in the forms of annotated portfolios [
16
,
57
], strong concepts [
58
], methods, and
experiential qualities [
37
]. None of this work is or can ever be replicable, as design is a creative
practice [
59
]. But it can be internally consistent and fulfil criteria such as being contestable, defensible
and substantive [60].
Together, we decided to analyze the design exemplars brought to the workshop (as we had all
jointly experienced them), the design methods that shape our practices, and provide an account of
the theoretical concepts that has started to shape and be shaped by our design practice—definitions,
strong concepts, experiential qualities, and so on.
What RtD literature does not always acknowledge is the strong role of the designer/user—the
subjective attitude, aesthetic sensitivity, politics, taste, values and bodily experiences—governing
design choices and use. In the analysis of our design exemplars, design practices and theory
contributions, started during the workshop but later continued in the work behind this paper, the role
of the designer/user and their felt experience of the interaction were repeatedly emphasized. This can
perhaps be contrasted with the current artificial intelligence trend turning towards data analytics as the
solution to our design problems. Data analytics can be used as a way to understand user behavior (as in
A/B-testing), or even as a way to create designs automatically [
61
]. Though we are not opposed to
making use of data analytics, we argue strongly in this paper for a continuation and even a deepening
of designer-driven experiential practices and shifting of the perspective to instead seeing data as a
material in the design—with and through designers and users.
4. Data: Design Exemplars, Design Practice and Theoretical Underpinnings
Let us now turn to the data generated by the two activities (the workshop and the RtD analysis).
We divide our data into: an autobiographical account of five design exemplars, jointly experienced
at the workshop; first-person design methods and material encounters soma-based interaction
practitioners typically engage with; and some of the theoretical underpinnings of our work (that is,
not the generic concepts from philosophy introduced in the background above, but the concepts
specifically arising when creating or engaging with soma-based designs).
4.1. Annotated Portfolio of Design Exemplars Experienced at the Workshop
Let us start with an autobiographical account of the five designs that were brought to the
workshop, illustrating how a first-person perspective is necessary in designing, experiencing and
communicating what they are.
4.1.1. BrightHearts
The first example is BrightHearts [
62
], an interactive artwork and biofeedback application for
mobile phones and tablets, developed as a tool for reducing pain and anxiety experienced by children
undergoing painful medical procedures. Decreases in heart rate are used to modulate the diameter
and color of a collection of overlapping concentric circles, see Figure 2. The more relaxed you are,
the less bright and warm colors appear at the interface. The application invites users to explore
ways to decrease their average heart rate through a combination of muscle relaxation, slow breathing
and relaxing memories—decreases are rewarded with gentle chimes, and new layers of colorful
circular imagery. The project continues Khut’s on-going exploration of the somaesthetic potential of
biofeedback as a way to facilitate deeply embodied experiences, self-efficacy, and reflections on the
ways we experience and apply our attention to voluntarily influence our stress and relaxation responses.
First-person experience is central to design of these displays—the movement and responsiveness of
the various layers and sounds is literally coded onto and around the artist/designer’s own physiology,
and adjusted to achieve a balance between responsiveness to changes in heart rate, and movement
(animation) qualities appropriate for the application’s relaxation-training aims. These mappings are
Informatics 2018,5, 8 10 of 26
then tested with other users. Works such as this are designed to respond in an aesthetically compelling
and rewarding way to voluntary adjustments to nervous system reflexes—the responsiveness of
the sounds and visuals to the dynamics user’s nervous state—the sense of how these displays ‘fit’
in relation to these movements—can only be fully appreciated by the person interacting with the
display, in much the same way as with clothing—where ‘fit’ is best judged by the person wearing the
clothing item.
Informatics 2018, 5, x FOR PEER REVIEW 10 of 26
ways we experience and apply our attention to voluntarily influence our stress and relaxation
responses. First-person experience is central to design of these displays—the movement and
responsiveness of the various layers and sounds is literally coded onto and around the artist/designer’s
own physiology, and adjusted to achieve a balance between responsiveness to changes in heart rate,
and movement (animation) qualities appropriate for the application’s relaxation-training aims. These
mappings are then tested with other users. Works such as this are designed to respond in an
aesthetically compelling and rewarding way to voluntary adjustments to nervous system reflexes—
the responsiveness of the sounds and visuals to the dynamics user’s nervous state—the sense of how
these displays ‘fit’ in relation to these movements—can only be fully appreciated by the person
interacting with the display, in much the same way as with clothing—where ‘fit’ is best judged by
the person wearing the clothing item.
Figure 2. BrightHearts visualizations.
Here is a first-person account by Höök (to appear in [63]):
I tried the BrightHearts app at a workshop that I was running. I was quite stressed about the whole
situation as I was trying to keep track of the progress of the workshop, orchestrating activities,
making sure everyone got to try out all the demos participants had brought, trying to facilitate
discussions, pushing us towards the interesting topics we wanted to debate. Once I put on the system,
I was also stressed about the fact that everyone could see how my relaxation was progressing through
watching the interface (portrayed on an iPad placed on the floor between us). I am usually able to
calm myself down through deep breathing and turning my focus inwards, but here it took me a while
to calm down the animations and make the color go from warmer to colder colors. As my mind was
scurrying in all directions to do with the workshop and the situation at hand, and the chitchat and
discussion by people sitting around me distracted me. But the animations of my heart were also
mesmerizing. I remember wishing that I could go and sit in a corner on my own to explore my
connection to the system.
4.1.2. The Soma Mat
The Soma Mat uses directed heat stimuli to subtly guide your attention to different body parts
[64–66], see Figure 3. The mat can be used as a support for exercises aimed at increased body
awareness such as guided meditation, for body scanning, in Feldenkrais sessions, or simply for
relaxation. Integrated into the mat are a number of heat pads that can provide spatially directed heat
pulses, and a pair of speakers. An application on a mobile phone is used in conjunction with the mat,
with a number of pre-recorded audio sessions aimed at increasing body awareness. The instructions
engage participants in very slow movements, while simultaneously directing attention to the
different parts of our bodies and how they interact.
Figure 2. BrightHearts visualizations.
Here is a first-person account by Höök (to appear in [63]):
I tried the BrightHearts app at a workshop that I was running. I was quite stressed about the whole
situation as I was trying to keep track of the progress of the workshop, orchestrating activities,
making sure everyone got to try out all the demos participants had brought, trying to facilitate
discussions, pushing us towards the interesting topics we wanted to debate. Once I put on the
system, I was also stressed about the fact that everyone could see how my relaxation was progressing
through watching the interface (portrayed on an iPad placed on the floor between us). I am usually
able to calm myself down through deep breathing and turning my focus inwards, but here it took me
a while to calm down the animations and make the color go from warmer to colder colors. As my
mind was scurrying in all directions to do with the workshop and the situation at hand, and the
chitchat and discussion by people sitting around me distracted me. But the animations of my heart
were also mesmerizing. I remember wishing that I could go and sit in a corner on my own to explore
my connection to the system.
4.1.2. The Soma Mat
The Soma Mat uses directed heat stimuli to subtly guide your attention to different body
parts
[6466]
, see Figure 3. The mat can be used as a support for exercises aimed at increased
body awareness such as guided meditation, for body scanning, in Feldenkrais sessions, or simply for
relaxation. Integrated into the mat are a number of heat pads that can provide spatially directed heat
pulses, and a pair of speakers. An application on a mobile phone is used in conjunction with the mat,
with a number of pre-recorded audio sessions aimed at increasing body awareness. The instructions
engage participants in very slow movements, while simultaneously directing attention to the different
parts of our bodies and how they interact.
The carefully tuned thermal stimuli serve to guide and sustain attention to different body parts.
While engaging in these somatic exercises, you sometimes feel quite vulnerable. Pains, aches and
bodily memories surface during the exercise. The materials of the mat are chosen to provide enclosure
and softness to help you feel taken care of and feel empathy with yourself. Workshop participants
tried out the mat and instructions.
Informatics 2018,5, 8 11 of 26
Informatics 2018, 5, x FOR PEER REVIEW 11 of 26
Figure 3. Soma Mat.
The carefully tuned thermal stimuli serve to guide and sustain attention to different body parts.
While engaging in these somatic exercises, you sometimes feel quite vulnerable. Pains, aches and
bodily memories surface during the exercise. The materials of the mat are chosen to provide enclosure
and softness to help you feel taken care of and feel empathy with yourself. Workshop participants
tried out the mat and instructions.
Here is a first-person account of the Soma Mat, used together with a second system, named Breathing
Light (it subtly follows your breathing by dimming a light [64]). This account is by Jonsson using the
system in a different setting—in the open office space of the research center he was working in:
The Soma Mat and Breathing Light demo was situated in a corner of an open office space. The oval
mat on the floor was partly secluded by something looking like a lamp shade with a curtain of cotton
strings hanging down towards the mat. As I laid down on the mat, I was first disturbed by the people
passing by, feeling slightly embarrassed by the act of laying down on the floor at work in the middle
of the day. As I positioned myself on the mat, the sting curtain from the Breathing Light somewhat
helped to block out the surrounding activities.
My mind was still occupied with the various activities I had been engaged with during the day,
and I felt a bit reluctant and stressed, questioning whether I really had time for a session at this time.
I tried to shake off these feelings and took a deep breath, trying to relax. Immediately I noticed how
the soft light above my head changed with my breathing, dimming out slowly as I exhaled and
intensifying again when I inhaled.
I picked up the phone and put on the headphones, opened the app and started the session. As I
could hear the familiar voice starting to talk in my headphones, I leaned back, closed my eyes, and
took some time to reflect on how I actually felt. I could feel the stress lingering in my body, and the
throbbing pain in my heel, that I never really seem to entirely get rid of, the tension over my neck
and shoulder, and spikes of pain in parts of my back. The recorded voice continued by asking
questions to help guide my attention to different parts and experiences of my body—the different
limbs, the skin, the breathing, the head, my back. As I directed my attention to different parts of my
body, the mat subtly heated up underneath those body parts. The warmth came on slowly and
dissipated slowly, creating a soothing sensation.
After a while, the voice was replaced by a calm rhythmic soundscape, similar to the sound of
waves on a beach, while the slow pulses of heat continued to move between different parts of the body,
moving from heels to shoulders and from left to right. Through my closed eyelids, I could still notice
Figure 3. Soma Mat.
Here is a first-person account of the Soma Mat, used together with a second system, named
Breathing Light (it subtly follows your breathing by dimming a light [
64
]). This account is by Jonsson
using the system in a different setting—in the open office space of the research center he was working in:
The Soma Mat and Breathing Light demo was situated in a corner of an open office space. The oval
mat on the floor was partly secluded by something looking like a lamp shade with a curtain of cotton
strings hanging down towards the mat. As I laid down on the mat, I was first disturbed by the
people passing by, feeling slightly embarrassed by the act of laying down on the floor at work in the
middle of the day. As I positioned myself on the mat, the sting curtain from the Breathing Light
somewhat helped to block out the surrounding activities.
My mind was still occupied with the various activities I had been engaged with during the day,
and I felt a bit reluctant and stressed, questioning whether I really had time for a session at this
time. I tried to shake off these feelings and took a deep breath, trying to relax. Immediately I noticed
how the soft light above my head changed with my breathing, dimming out slowly as I exhaled and
intensifying again when I inhaled.
I picked up the phone and put on the headphones, opened the app and started the session. As I
could hear the familiar voice starting to talk in my headphones, I leaned back, closed my eyes, and
took some time to reflect on how I actually felt. I could feel the stress lingering in my body, and
the throbbing pain in my heel, that I never really seem to entirely get rid of, the tension over my
neck and shoulder, and spikes of pain in parts of my back. The recorded voice continued by asking
questions to help guide my attention to different parts and experiences of my body—the different
limbs, the skin, the breathing, the head, my back. As I directed my attention to different parts of
my body, the mat subtly heated up underneath those body parts. The warmth came on slowly and
dissipated slowly, creating a soothing sensation.
After a while, the voice was replaced by a calm rhythmic soundscape, similar to the sound of waves
on a beach, while the slow pulses of heat continued to move between different parts of the body,
moving from heels to shoulders and from left to right. Through my closed eyelids, I could still notice
how the intensity of the light above my head changed in close correspondence with my breathing.
The warmth from the mat had spread throughout my body, and I noticed the different thermal
sensations of heat coming from the mat, and the coolness from the surrounding air.
Informatics 2018,5, 8 12 of 26
At this stage, I felt entirely secluded from the surrounding space, enclosed in a safe and calm space,
and my attention was subtly guided by heat and light towards different body parts and functions.
I felt immersed, in the space as well as in the experience emanating from the mixed sensations of the
external stimuli felt through my body, and the sensation of feeling my body through the external
stimuli. My initial distinct sensations of pain had gradually been replaced by richer and more
nuanced sensations, which let the sensations of pain to fade into the background, overshadowed by
the richer bodily sensations of my breathing, different sensations of heat and light, and the sensations
of different parts of my body resting on the floor.
When the voice finally asked me to open my eyes and slowly get up, I woke up to a very different state
of mind compared to before the session. Initially I felt a bit dizzy and foggy-headed, but entirely calm
and present in my body, with the sensations of tension and pain much less prominent. Gradually,
I felt increasingly revived and energetic, and more present in the now, a sensation that lingered for
hours after the session.
4.1.3. The Tail and Ears
Our third example from the workshop is the Tail and Ears project [
67
], see Figures 4and 5. These
are two design projects involving mechanical body extensions for use in theatre settings. The tail
responds to the swaying of your hips, using an inertial measurement unity, with accelerometer and
gyro, on the hip. The ears are controlled by bend sensors on a glove.
The way the creators describe the background to these two projects is “The lived body is our
experienced body, the body through which we live our lives, which is different from seeing the body as an object
in the world. Designing for the body with the first-person perspective of Merleau-Ponty makes us aware of how
technology is incorporated into the experienced body, and how it thus changes us” [
67
]. The idea is that the tail
and the ears extend the actor’s body, creating a richer array of expressions. In [
68
], Svanæs reported
his own first-person experience of wearing the tail that he had created: One of the most interesting
experiences for me was when I took off the tail after wearing it for an hour, and had a surprising physical feeling
of being tailless”.
Informatics 2018, 5, x FOR PEER REVIEW 12 of 26
how the intensity of the light above my head changed in close correspondence with my breathing.
The warmth from the mat had spread throughout my body, and I noticed the different thermal
sensations of heat coming from the mat, and the coolness from the surrounding air.
At this stage, I felt entirely secluded from the surrounding space, enclosed in a safe and calm
space, and my attention was subtly guided by heat and light towards different body parts and
functions. I felt immersed, in the space as well as in the experience emanating from the mixed
sensations of the external stimuli felt through my body, and the sensation of feeling my body through
the external stimuli. My initial distinct sensations of pain had gradually been replaced by richer and
more nuanced sensations, which let the sensations of pain to fade into the background, overshadowed
by the richer bodily sensations of my breathing, different sensations of heat and light, and the
sensations of different parts of my body resting on the floor.
When the voice finally asked me to open my eyes and slowly get up, I woke up to a very different
state of mind compared to before the session. Initially I felt a bit dizzy and foggy-headed, but entirely
calm and present in my body, with the sensations of tension and pain much less prominent.
Gradually, I felt increasingly revived and energetic, and more present in the now, a sensation that
lingered for hours after the session.
4.1.3. The Tail and Ears
Our third example from the workshop is the Tail and Ears project [67], see Figures 4 and 5. These
are two design projects involving mechanical body extensions for use in theatre settings. The tail
responds to the swaying of your hips, using an inertial measurement unity, with accelerometer and
gyro, on the hip. The ears are controlled by bend sensors on a glove.
The way the creators describe the background to these two projects is “The lived body is our
experienced body, the body through which we live our lives, which is different from seeing the body as an object
in the world. Designing for the body with the first-person perspective of Merleau-Ponty makes us aware of how
technology is incorporated into the experienced body, and how it thus changes us” [67]. The idea is that the
tail and the ears extend the actor’s body, creating a richer array of expressions. In [68], Svanæs
reported his own first-person experience of wearing the tail that he had created: “One of the most
interesting experiences for me was when I took off the tail after wearing it for an hour, and had a surprising
physical feeling of being tailless”.
Figure 4. Katherine Isbister wearing the Tail.
Figure 4. Katherine Isbister wearing the Tail.
Informatics 2018,5, 8 13 of 26
Informatics 2018, 5, x FOR PEER REVIEW 13 of 26
Figure 5. Engaging with the Ears.
Workshop participants engaged with the tail and ears in social, playful, open-ended ways.
For example, one participant controlled the ears that were placed on another participant’s head.
Here is a first-person account of testing the Tail and Ears at the workshop by Höök, taken from [63]:
I got to try the Ears and Tail at a workshop. At first, I felt mildly embarrassed walking around with
a tail wiggling evocatively behind my back, arousing smiles and embarrassed laughter from my
workshop friends. Once I overcame my discomfort, though, I found the experience strangely
expressive. To make the tail wiggle, I had to move my hips in different patterns, for example: a circular
motion in the sagittal plane, or a smooth sashay from side to side, or a tilt as if I was putting the tail
between my legs. As I did these movements, I could feel the tail as a clear substance, a counterweight,
not too heavy nor too light, changing in response to my movements.
The circular movements of my hips encouraged movements that I associate with a feminine,
sexy, flirtatious way of walking. As the flirtation was directed backwards, behind my back so to speak,
I repeatedly turned my head, cocking it, nodding, smiling, to check out the effect on others behind
me. My whole body posture and movement schema was altered. The feeling I had was that my tail
happily wriggled in a sashay, almost slightly aggressive, “look at me”, cat-like manner.
Putting the tail between my legs put bodily reality and substance into the expression of
cowardice and shame. I could feel my face crumble into a sad embarrassed expression—again looking
behind me to see the effect on others, but now looking under my brows, with a facial expression saying
“excuse me”. The tipping of my hips required a conscious movement that could not be done while
walking, or at least not in the few minutes I wore the tail.
The flapping ears affected me less powerfully—and in some ways serves to show how soma
engagement is not easy to design for. Rather than an experience where my movements immediately
spurred the ears to move, feeding back into my experience of myself, I felt more removed as my hand
made gestures having some effect on the ears which I could not sense or see myself. Perhaps training
in front of a mirror to synchronize these movements with my facial expressions or feeding off the
reaction of others watching my ears might have created a more embodied experience: I am guessing
that this is what the actors had to do in order to make their performance expressive and seamless.
In the workshop, one participant figured out that she could wear the glove while another
participant wore the ears. This became a playful interaction between them where she tried to match
his facial expression by flapping the ears in the right tempo and direction. As it turned out, we all
Figure 5. Engaging with the Ears.
Workshop participants engaged with the tail and ears in social, playful, open-ended ways.
For example, one participant controlled the ears that were placed on another participant’s head.
Here is a first-person account of testing the Tail and Ears at the workshop by Höök, taken from [
63
]:
I got to try the Ears and Tail at a workshop. At first, I felt mildly embarrassed walking around
with a tail wiggling evocatively behind my back, arousing smiles and embarrassed laughter from
my workshop friends. Once I overcame my discomfort, though, I found the experience strangely
expressive. To make the tail wiggle, I had to move my hips in different patterns, for example:
a circular motion in the sagittal plane, or a smooth sashay from side to side, or a tilt as if I was
putting the tail between my legs. As I did these movements, I could feel the tail as a clear substance,
a counterweight, not too heavy nor too light, changing in response to my movements.
The circular movements of my hips encouraged movements that I associate with a feminine, sexy,
flirtatious way of walking. As the flirtation was directed backwards, behind my back so to speak,
I repeatedly turned my head, cocking it, nodding, smiling, to check out the effect on others behind
me. My whole body posture and movement schema was altered. The feeling I had was that my tail
happily wriggled in a sashay, almost slightly aggressive, “look at me”, cat-like manner.
Putting the tail between my legs put bodily reality and substance into the expression of cowardice
and shame. I could feel my face crumble into a sad embarrassed expression—again looking behind
me to see the effect on others, but now looking under my brows, with a facial expression saying
“excuse me”. The tipping of my hips required a conscious movement that could not be done while
walking, or at least not in the few minutes I wore the tail.
The flapping ears affected me less powerfully—and in some ways serves to show how soma
engagement is not easy to design for. Rather than an experience where my movements immediately
spurred the ears to move, feeding back into my experience of myself, I felt more removed as my hand
made gestures having some effect on the ears which I could not sense or see myself. Perhaps training
in front of a mirror to synchronize these movements with my facial expressions or feeding off the
reaction of others watching my ears might have created a more embodied experience: I am guessing
that this is what the actors had to do in order to make their performance expressive and seamless.
In the workshop, one participant figured out that she could wear the glove while another participant
wore the ears. This became a playful interaction between them where she tried to match his facial
Informatics 2018,5, 8 14 of 26
expression by flapping the ears in the right tempo and direction. As it turned out, we all felt we
knew how the ears should be turning to express certain emotions, probably from our experiences of
cats, dogs, horses and other domesticated animals, as well as from various media, theatre, movies
and so on.
4.1.4. GangKlang
Workshop participants also tried out this mobile app that makes use of a person’s walking rhythm
to generate a soundscape [
69
] as they stride along. GangKlang was inspired by work songs, and how
they integrate the singing voice within a repetitive body movement. The sung words underscore and
structure the rhythm of the activity. Fusing mind and body, they absorb the worker’s attention to
the present moment [
70
]. GangKlang aims to re-emphasize the bodily dimension within the walker’s
experience, supporting a first-person perspective by providing functionality for feeling into and
engaging with the movement. The app detects movement phrases through which the user brings
forth the walking process in interaction with contextual conditions and translates peculiarities of
each phrase into characteristics of a soundscape in real time. This makes the user’s movements and
occurring changes in the movement audible. The user’s attention is directed towards the present
walking movement and how its characteristics are influenced by the spatial environment, the others,
and the sonic feedback itself. It provides an experiential way to explore how our walking movements
are an expression of what ‘moves us’.
During the design process of GangKlang and other instantiations of sonic interaction for walking,
it became apparent that sound and interaction designers need to experience from a first-person
perspective the interplay of body movement, sound and mobile context.
Acknowledging the designer’s experiential awareness as an integral part of designing interactive
sound, the mobile App was extended to a rapid prototyping system [
71
]. This prototyping system
enables designers to (a) specify the sonic response to a predefined set of walking movements in an
editor interface, (b) use a mobile App to experience how their design for sonified walking unfolds in
interplay with body movement in mobile situations, and (c) analyze process data that is collected by
the mobile App to reflect on their sonified walking experience afterwards. In this way, the prototyping
system provides a short design-experimentation-feedback loop to explore and understand possibilities
and challenges in the design of interactive sound.
Here is a first-person account of GangKlang by Hajinejad:
Putting on the headphones and positioning the smartphone in my trouser pocket, I felt like preparing
for something important. I started walking and was highly concentrated on listening, similar to the
beginning of an unknown concert. The first steps passed by in silence and increased the suspense,
and then, evolved a regular rhythm of marimba sounds. I placed my attention on the sensation of
moving my legs, how I placed my foot on the ground and listened to the response in the sounds.
Focusing on both at once, my body movement and listening, was quite demanding. In order to return
to a walking pattern as in everyday life, I had to shift my attention to the outside and the route.
I listened carefully as the composition became more complex, and tried not to lose my walking
rhythm. Later, as sustaining flowing sounds joined the percussive ones, my steps felt buoyant and
energized. I started pondering the relationship between my walking movements and the sounds, and
my thoughts drifted away. It was only with the silence phase that nudged me out of my thoughts
and drew my attention to the walking process again.
As these different feedback modi repeated for some cycles, the sounds seemed not as interactive
as in the beginning, and more like meditative background music. The feedback was not exciting
anymore, and I stopped listening consciously. However, the calming sounds had a positive effect on
my thoughts, and the silence phases kept bringing me back from my thoughts. Altogether, the sonic
feedback made me walk with more awareness and yet without judgement. I felt like having walked
with a companion.
Informatics 2018,5, 8 15 of 26
4.1.5. Embodied Encounters Studio
The last example system engaged during the workshop was the Embodied Encounters Studio
[72,73]
.
Hummels has used this system to visit well over 80 inspirators around the world, envisioning and
exploring together hands-on what potential futures can be. The studio consists of an evocative toolbox
with three components: (1) the Embodied Ideation Toolkit: a set of tangible magnetic non-descriptive
objects, (2) Hexn, a modular tangible notation system existing of hexagonal transparent discs with
icons, and (3) Inspiration Cards, small RFID-enhanced laminated cards showing pictures that relate to
content in the database, e.g., slides and videos that are shown on a stationary iPad, see Figure 6.
Informatics 2018, 5, x FOR PEER REVIEW 15 of 26
4.1.5. Embodied Encounters Studio
The last example system engaged during the workshop was the Embodied Encounters Studio
[72,73]. Hummels has used this system to visit well over 80 inspirators around the world, envisioning
and exploring together hands-on what potential futures can be. The studio consists of an evocative
toolbox with three components: (1) the Embodied Ideation Toolkit: a set of tangible magnetic non-
descriptive objects, (2) Hexn, a modular tangible notation system existing of hexagonal transparent
discs with icons, and (3) Inspiration Cards, small RFID-enhanced laminated cards showing pictures
that relate to content in the database, e.g., slides and videos that are shown on a stationary iPad, see
Figure 6.
Figure 6 Embodied Encounters Studio.
The Embodied Encounters Studio was available during the interactivity track at the ACM
SIGCHI annual conference [23]. For the workshop, only the Embodied Ideation Toolkit was used.
The idea behind the design is that playful encounters using these objects allow two or more
participants to brainstorm together, engaging in participatory sense-making, while at the same time,
the encounter is being documented through video, audio and sensor data. The studio supports
nonverbal communication and social coordination in action, instead of a more dialogue-oriented
discussion in which we exchange messages originating in one ‘mind’ that are received by another.
Hummels states: “Embodied Encounters Studio aims at offering a more informal, playful, intimate and
engaging setting, where participants can take a first-person perspective and explore concepts hands-on [74].
Our previous research has shown that bodily engagement seems to drive participants towards concrete ideas
and away from the abstract, and to stimulate interaction based on personal experiences. Moreover, bodily
encounters seem to lower the threshold to merge the perspectives from people with different backgrounds [75].
Here is a first-person account of testing the Embodied Ideation Toolkit at the workshop by Höök
from [63]:
I was immediately drawn to the magnetic objects. Hummels had included old radio antennas that
could be extended, serving as “pointing” devices or as a way to pick up the magnetic objects. I
extended the antenna and started picking up objects with it. Some objects were too heavy to stay on
the antenna and would fall off. Other participants playfully chipped in with their antennas or other
objects to help out, or started dragging around objects, attached to one another through the magnetic
force. The effect on all of us was immediate—we instantly became playful, engaging one another
through joining forces or stealing objects from one another. We touched the objects, explored their
qualities: the space they occupied, their velocity, their affordance for movement, whether they could
be combined into larger forms without breaking apart, and their capacity to sustain play between us.
We touched, fiddled, built, destroyed, explored, and engaged both with ourselves and with one
another. It opened a path to social play.
4.1.6. What Unites these Diverse Design Exemplars?
First, note how there is a diversity of aims in these applications: supporting learning skills in
emotion-regulation and relaxation; understanding your own walking habits; spurring playful
discussion with one another; whole-body movements; or adding to our bodily expressiveness. This
shows how a first-person soma design stance can generate many different applications in diverse
domains. In fact, turning to the literature on soma design, we see a field that is growing, engaging
with one application domain after another (see, e.g., [33,42,53,62,64,76–88]). Some even claim that a
Figure 6. Embodied Encounters Studio.
The Embodied Encounters Studio was available during the interactivity track at the ACM
SIGCHI annual conference [
23
]. For the workshop, only the Embodied Ideation Toolkit was used.
The idea behind the design is that playful encounters using these objects allow two or more
participants to brainstorm together, engaging in participatory sense-making, while at the same time, the
encounter is being documented through video, audio and sensor data. The studio supports nonverbal
communication and social coordination in action, instead of a more dialogue-oriented discussion in
which we exchange messages originating in one ‘mind’ that are received by another. Hummels states:
Embodied Encounters Studio aims at offering a more informal, playful, intimate and engaging setting, where
participants can take a first-person perspective and explore concepts hands-on [
74
]. Our previous research has
shown that bodily engagement seems to drive participants towards concrete ideas and away from the abstract,
and to stimulate interaction based on personal experiences. Moreover, bodily encounters seem to lower the
threshold to merge the perspectives from people with different backgrounds” [75].
Here is a first-person account of testing the Embodied Ideation Toolkit at the workshop by Höök
from [63]:
I was immediately drawn to the magnetic objects. Hummels had included old radio antennas
that could be extended, serving as “pointing” devices or as a way to pick up the magnetic objects.
I extended the antenna and started picking up objects with it. Some objects were too heavy to stay on
the antenna and would fall off. Other participants playfully chipped in with their antennas or other
objects to help out, or started dragging around objects, attached to one another through the magnetic
force. The effect on all of us was immediate—we instantly became playful, engaging one another
through joining forces or stealing objects from one another. We touched the objects, explored their
qualities: the space they occupied, their velocity, their affordance for movement, whether they could
be combined into larger forms without breaking apart, and their capacity to sustain play between
us. We touched, fiddled, built, destroyed, explored, and engaged both with ourselves and with one
another. It opened a path to social play.
4.1.6. What Unites these Diverse Design Exemplars?
First, note how there is a diversity of aims in these applications: supporting learning skills
in emotion-regulation and relaxation; understanding your own walking habits; spurring playful
discussion with one another; whole-body movements; or adding to our bodily expressiveness.
This shows how a first-person soma design stance can generate many different applications in
diverse domains. In fact, turning to the literature on soma design, we see a field that is growing,
Informatics 2018,5, 8 16 of 26
engaging with one application domain after another (see, e.g., [
33
,
42
,
53
,
62
,
64
,
76
88
]). Some even
claim that a slow, soma-based, first-person engagement will be relevant to all forms of interactive
design work [
63
], but perhaps in particular designs with the plethora of upcoming interactive
materials, such as shape-changing interfaces, Internet of Things, sensor networks, and all forms
of movement-based interactions.
But what became more interesting to workshop participants was not so much the uniqueness of
each application, but the similarities between them. When engaging through a first-person perspective,
it was clear that they shared a certain designerly, aesthetic sensitivity clearly coming through in the
design. We ‘recognized’ this similarity, but this was not apparent to us through studying or comparing
their surface form or shape, or the functionality or business model they provided, as might be the
case with a website design or mobile app. The familiar feel of these systems revealed itself only
through engaging yourself, through a first-person experience. A range of insights arose through this
first-person experience.
Apart from the obvious focus on movements of different kinds in all of them, the systems share a
playful approach in which habitual movements from our everyday life are disrupted. Laying down on
the floor and slowly engaging with very small movements, walking differently, brainstorming with
others through manipulating fun objects, or flapping ears or sweeping your tail are all non-habitual
movements. While all are within our capacity as human beings—within our human morphology,
as Sheets-Johnstone would phrase it [
47
]—we might never have moved in this manner ever before,
or perhaps not moved in this particular manner since we were children. Reconnecting with our
moving selves through non-habitual movements connecting us to novel interactions, and thereby
novel experiences, was inherently interesting, making us playful and engaged.
Each design also offered a delightful and engaging experience, creating novel meaning-making
processes. They engaged us somatically, making us connect emotion, thinking, movement, and
expression into one subjectivity. Each interaction experience transported us into a different space,
engaging our senses, awakening our corporeal sensuous perception, and encouraging us to be guided
by our subjective experiences, alone or sometimes together, in order to open to ideas beyond rational
mindsets, to evade well-trodden tracks, concepts and relationships.
This does not mean that any combination of movement, emotion, thinking and expression will
‘work’. This perhaps most clearly comes into focus when the interaction fails—as with the flapping
Ears by Svanaes. As the interaction separated input from output, the meaning-making flow did not
properly connect your first-person experience of moving your hand with the flapping of the Ears.
Instead, others could see the Ears flapping, and you could second-guess what others would see;
engaging, in effect, the wearer in second- and third-person interpretations of what was going on.
Closing the loop—the affective loop [
89
], the feedback loop—is of key importance. In effect, this
is the same argument as provided by Norman and Tognazzini’s critique of mobile phone gestures,
where gestures are not properly connected to feedback at the interface [
90
]. But our insight here goes
beyond providing relevant feedback to make sure learning of a function can be made easy, creating
the correct affordances. Instead, the feedback coupling (when it worked) engaged, on a profound
level, connections between movements, emotional and intellectual experiences. Beyond functional
affordances, aesthetically enhanced actions and sensations are developed in these applications—as ways
to expand the user’s repertoire of movements and orientations, to differentiate and practice new or
forgotten patterns for action and response. As discussed above, we see aesthetics as a path to ‘examine’
and improve on connections between sensation, feeling, emotion and subjective understanding and
values [91]. Each interaction with the systems introduced above transformed our somatic selves.
4.2. First-Person Methods and Material Encounters
Let us now turn to the second set of data: our methods and material encounters. These
were compiled during the discussion at the workshop and in subsequent collaboration during the
construction of this paper.
Informatics 2018,5, 8 17 of 26
Attending to your first-person experience is, on the one hand, easy, as your body is always there
for you, and as Thecla Schiphorst phrased it during the workshop: the body is generous with tips”.
But at the same time, it requires attention and focus. To discern all the small details of an experience,
you might need to slow down, disrupt or engage with some specific practice in order to ‘see’ yourself,
your movement, emotion and the linkages between all the different parts of your first-person presence.
It is sometimes a slow learning process, but at the same time a rewarding process, as it increases your
presence, understanding and aesthetic sensibilities overall. In Section 4.2.1, we describe some example
tactics to achieve a richer first-person perspective.
Second, interaction design is always the study of the socio-digital material: the coming together
of people with designed interactive artefacts, and the behaviors and experiences generated in the
combined material of people, practices, tools and artefacts. It is not enough to attend to your own
soma, you also need to touch, feel and interact with the digital/technical materials that you aim to
shape into the envisioned application. Interaction design always thrives on the affordances of the
technologies and materials used to shape the artefacts. In Section 4.2.2, we turn to some of the tactics
used to engage with technology to attain deep material connoisseurship.
4.2.1. Attending to Your Own Soma
The design material in the exemplars above in a sense consists of two parts: on the one hand the
digitally mediated materials, such as programmable heat pads or bend sensors, and on the other hand
our bodily selves—our somas. Both can be, and are in fact, shaped in and through the interaction. Our
movements change dynamically in response to kinetically dynamic possibilities in our environment.
Those possibilities, of course, include any tools and artifacts we create, cultural practices, and so on.
There is infinite variation in the world; in our felt experience, but also in the variations of the social
and cultural landscape we are shaped by and actively take part in shaping, reinforced and reinstated
through all our everyday acts.
There is, luckily, a path out of the dilemma of the infinite space of possibilities in which design
may shape the world, and that is to engage with your own body, your own movement, your own
felt emotional experience and sense of aesthetics, and let the design process feed off of your own felt
understanding and experience. This is, on the one hand, easy, as your body is always there for you.
You just need to return to it, listen to it, and discern all the different small nuances of movements
and the meaning making they create. On the other hand, such processes are demanding, and they
might require guidance. This is particularly the case, as these methods will also change your body
and perception, increasing your aesthetic sensibility. Here, it is important to make clear that you do
not need to become an expert in yoga or meditation or some other bodily practice to do this. Let us
provide a brief introduction to some of the methods and tactics used by the experts in this field.
Change and Interest.
Directing attention to your own soma relies on the interplay between two
different cognitive processes that Shusterman has developed further from William James: change
and interest [
92
]. By change, he refers to the importance of subdividing the bodily experiences into
more specific body areas or functions and then engaging in activities that shift focus from one area to
another and back, in order to provide a more nuanced and rich perception of fine-grained movements.
The notion of interest, on the other hand, deals with finding means to achieve a sustained attention
towards the part of the body currently under examination. Or as Shusterman phrases it: To reach
precise bodily introspection the key is to direct our focused attention first to one part then another, a clearer sense
of relations of parts to whole can be obtained. This transition of focus, provides sense of change, it also renews
our interest in each new body part” [44].
Lee, Lim and Shusterman have together applied these six tactics to design [
36
]. To provide for
heightened attention and interest to the body, they suggest:
Informatics 2018,5, 8 18 of 26
“1. Questions: Asking questions about different aspects and relations of what we perceive.
2. Division into parts: Subdividing the body and directing our attention to each part, one by one.
3. Contrasts of feeling: Discriminating the different feelings in one part of the body from those
in another.
4. Associative interests: Making the noticing of what we are trying more precisely to feel a key to
something we care about.
5. Avoiding distracting interests: Warding off competing interests to what we are trying to attend
to and feel.
6. Pre-perception: Preparing our attention to notice what we are trying to discriminate in what we
feel” [36].
This method was used in the design process behind the Soma Mat and the Breathing Light
described above.
Disrupting the Habitual—Estrangement.
Another method frequently mentioned, as well as
acted out in the Collaborative Walking exercise above, is to slow down or disrupt a habitual movement to
be able to discern small changes, to feel how different parts of your body are connected, to note how the
movements relate to your emotional experiences, to enjoy sensations or feel pain, to be engaged [
25
].
Defamiliarizing habitual movement patterns is a core principle in many somatic practices [
45
,
46
,
93
],
based on the rationale that only by bringing to the surface what has become automatic and tacit, can
we change deeply ingrained habitual patterns of movement. Often this is achieved by providing
constraints to the habitual pathways in a movement pattern. In terms of movement-based interaction
design, this awareness and playful approach to re-patterning movement possibilities enables us to
explore new kinds of meaningful movements in a structured way by drawing on techniques that have
already been developed and refined in somatic and dance practices [25].
This method was used in the design of, for example, GangKlang.
Somatic Connoisseurship.
Schiphorst has argued for the value of applying somatic connoisseurship
in the design and evaluation of aesthetic soma-based designs [
1
]. The practice of somatic
connoisseurship highlights the significance of somatic facilitation as a role within technological design
processes. The role of somatic connoisseurship characterizes expertise that is developed, expressed,
and passed on through the constantly refining process of soma-based aesthetic practice. This practice
develops expertise that can access and train experiential acuity, including observation, discernment,
synthesis, empathy, and focus. The somatic body-based traditions require techniques that use attention,
observation, and discrimination, applied to the material of experience for the purpose of self-cultivation.
These techniques are developed through training that is tested and validated through the efficacy of
practice. A somatic connoisseur is someone who is skillful in an embodied movement-based practice
that guides, mentors and shares their insights with co-designers and participants, guiding collaborators
in what to attend to, how to move and feel. Somatic connoisseurs may be our co-designers, or they
might not know how to deal with the digital materials engaging researchers in interaction design.
The practice of somatic connoisseurship may inspire transfer of qualities from their body-based practice
to our technologically situated practices. The experts in the workshop reported bringing in somatic
connoisseurs such as choreographers, chefs, or Feldenkrais practitioners.
Laban-Movement-Analysis
. Another path to discerning movement as a language that bridges
aesthetic potential with functional instrumentalization comes from the work of Rudolph Laban.
He created Laban Movement Analysis (LMA) and its notational systems, Motifs and Labanotation [
94
].
Laban Movement Analysis has a rich history of practice that is based upon first-person experiential
training of the moving body in relation to itself and its environment: the Body (what is moving),
Effort (the quality of how the body moves), Space (where the body is moving in relationship to itself,
others and the larger geographical movement that exist in the world), and Shape (which describes
the interaction or relationship between the experiential moving body, others and the artefacts and the
larger world in which it exists). LMA also includes somatic exploration of the body’s developmental
patterns, personal movement signatures, proxemics and cultural movement patterns that influence
Informatics 2018,5, 8 19 of 26
our own and larger group patterns of movement style. LMA is valuable for how it explicitly bridges
the first-person experiential with connection to the external environment, including how we relate
to others. While LMA is considered a movement language born from Western knowledge histories,
Rudolf Laban was influenced by historical teachers of self-cultivation such as Gurdjieff, Delsarte and
Sufism. In HCI, it has been used as a means to connect experience and movement (e.g., [9597]).
In LMA, the dimensions that address the inner experience of a movement touch on the underlying
qualities of a first-person experience. The Effort and Shape dimensions are particularly interesting in
this regard. To really understand the experiential meaning of these dimensions, you need to be trained
in LMA.
4.2.2. Attending to Design Materials and Shaping Concepts
Attending to our own somas is half the story. The other half is attending, with a first-person
perspective, to the design materials we use to create the dynamic gestalt of our designs. While debating
where design ideas come from is too large a topic to deal with here, let us describe a few of the design
tactics the experts in the workshop had found useful during different phases of the design process.
Autobiographical Design
. Almost all the workshop attendees had, at one point or another,
engaged in autobiographical design enquiry to shape their designs. That is, using yourself, your own
experience and desires, in developing the design. This requires taking on all the roles in the design:
we were designers, researchers and users of our own design concepts as they evolve. In those design
processes, we have to repeatedly try different digital and physical materials, simulating interactions
with manually controlled sensory stimuli, and testing them in situ to find the ones that would make
sense. The interactions have to be simulated and acted out in order for us to really feel their impact
on our bodily experiences. Simply imagining what they would be like was not enough to qualify the
experience [
65
]. An interesting framework that helps to structure this process has been proposed by
Trotto and Hummels: the Designing in Skills framework [72,73].
Embodied Sketching
is an umbrella term to describe ideation design activities in the domain of
embodied interaction that leverage the participants’ in-the-moment embodied experience to explore
and design for particularly interesting physical and social activities [
5
]. Examples of embodied
sketching include: bodystorming, where designers envision physical and social activities supported
by technology and physical artefacts; and physical movement sketching—using movement to develop
a design idea, to experience that movement, to transfer that movement to be experienced by others
through imitation, and then to reflect on the movement through discussion. It includes physical
movement as a frame of reference for future design, through inviting designers themselves to explore
and trust their own bodies’ movement capacities as a tool of reference in that work [
98
]. Another
example of embodied sketching is choreography of interaction—an approach where the product comes
into existence through creating activities and movements, i.e., in the choreography of interaction.
The choreography of interaction is the total system of user, product and other elements involved
in interaction. It embodies a trinity of physical involvement, dynamic quality and expressed
meaning [
24
]. There are also sensitizing activities, where designers hone their sensitivity towards
a particular bodily aspect or aesthetic they want to discuss, brainstorm, and ultimately design for, by
engaging with a bodily experience that supports this aspect or aesthetic in an interesting way (perhaps
in a non-conventional one, like with techniques that disrupt our ways of acting or thinking) [
5
].
Lastly, there are co-creation ideation activities, where users play around with (test and appropriate)
a technology prototype to support particular experiences [
99
]. All these ideation activities heavily
involve the moving body and are meant to happen early in the design process, before the construction
of fully-functional prototypes.
Data and Program Code
. Data can be used to shape our designs by means of recent advances in
artificial intelligence and machine learning. Machine learning designates a set of methods able to learn
from examples (data) in order to take decisions (e.g., recognizing a movement, recognizing who is
performing and how) instead of explicitly defining rules that would lead to these decisions. Applying a
Informatics 2018,5, 8 20 of 26
first-person perspective to machine learning allows for shifting the focus from technical computations
to designer intentions. Indeed, as designers we can provide the system with examples of behavior
or idiosyncratic goals and the system could learn from these examples. For example, in the design
of music interaction, a performer-designer can demonstrate to the system personal movements and
gestures, and the machine will learn this performer-specific vocabulary [
100
,
101
]. This is a powerful
tool that can be used by non-technical designers to explore and create designs. As such, non-specialists
used it recently in participatory design workshops in order to transfer high-level body-centered sonic
interaction design ideas into working prototypes [102].
Yet another path is the data-enabled design framework by Bogers and colleagues [
103
]. By using
data specific to each individual, we can imagine designs adapted to each, where a first-person
perspective is at the heart of the interaction, fostering idiosyncratic goals and somaesthetic experience.
4.3. Conceptualizations Arising through Design Research
The various design exemplars discussed above serve as definitions/facts through which we,
as expert design practitioners, can ‘feel’ and share our knowledge. We picked these particular ones as
they were brought to our workshop where we could all interact with them, but there is a whole range
of systems built along the same lines (e.g., [
9
,
12
,
20
,
37
,
43
,
50
,
53
,
58
,
64
66
,
69
,
74
,
82
,
89
,
104
]). While this is
early days, we do not yet know which ones will be the prototypical or canonical exemplars that will
serve as stylistic or orienting concepts to our soma-based design practice.
But we already note how some design elements and experiential qualities in our soma-based
designs are re-used (borrowed, stolen, re-interpreted, transformed) in novel design work. By naming
and articulating those and fine-tuning our articulations, we start forming our own research program,
our own theoretical concepts. This is no small endeavor. As pointed out by Shusterman [
13
], linguistic
tags can be used as a resource to improve perceptual nuances:
“Linguistic tags or descriptions, for example, can make a very vague feeling less difficult to
discriminate by tying that feeling to words, which are much more easily differentiated. James
argues, for instance, that the different names of wines help us discriminate their subtly different
flavours far more clearly and precisely than we could without the use of different names. [
. . .
]
The rich and value-laden associations of words can, moreover, transform our feelings, even our
bodily ones”.
Note how choice of language will not be an innocent choice here, but may shape what we are able
to see and feel. Similarly, as design researchers, our choice of articulations to frame our knowledge and
design insights will shape what we ‘see’. It is important to continue to bring more design exemplars
to shape this space. But, in addition to what Shusterman’s explanation of what linguistic tags can
do, our articulations need to also address the socio-digital materials—both our bodily experiences,
but also how those are guided by and relate to specific design choices.
The articulations we have seen in our field so far, apart from the design exemplars, are reported
in the academic literature as strong concepts (such as biofeedback loops, affective loops, somaesthetic
appreciation, or somaesthetic touch), experiential qualities (such as slow, inwards-turning meditative
experiences, uncomfortable interactions [
105
], scary experiences [
106
] or playful somaesthetics), and
design methods. These strong concepts, experiential qualities and methods should not be seen as
given, rigidly formed rules or patterns for interaction. They are conceptual lenses that help us ‘see’
and ‘feel’ potential design opportunities.
5. Discussion: First-Person Perspectives and Rigor?
Through our engagement with one another’s design exemplars, design practice and in our
analysis, we note how successful soma-based designs all seem to share a structured, careful, thoughtful
first-person engagement. While the experts may employ different strategies to become more
somatically and aesthetically aware, there is a shared understanding that you need to employ structured
Informatics 2018,5, 8 21 of 26
tactics to access your own movements,somatics and aesthetic sensibilities. We slow down our movements;
disrupt the habitual to help us grasp and articulate what is there; we direct our attention to specific
areas (change) and put our sustained attention to it (interest); we playfully engage in movement.
A first-person perspective in design puts emphasis on the individual designers’ (or researchers’)
subjective aesthetic experiences. But this also means taking seriously your own sensations and
experiences as a prerequisite for communication and collaboration, for example within a design
team. Given the “tacit” nature of bodily experiences, in that these kinds of sensations may be hard to
articulate or verbalize, it becomes especially important to create common grounds for sharing such
experiences. By engaging, as a group, in bodily and somatic experiences, these shared experiences can
work as a common ground from which intersubjectively constructed meanings [
107
], or kinaesthetic
empathy [
108
], can be created. In this process, we bring in the (digital and other) design materials
early on, touching, feeling, interacting with them, thereby letting them ‘speak’ back to us [
109
], letting
our design concepts thrive off their affordances.
This is not to say that we ignore what can be learnt through engaging with users outside the
design team. Rigor will also come from testing and exhibiting our designs, as well as letting others
experience and create meaning with them. But, as mentioned above, it is hard to get well-articulated
feedback from end-users, unless they are trained in some aesthetic bodywork practice on the specifics
of some designs. Often, our designs are open to interpretation, and the meaning and practice develops
in dialogue with use, and we need to document and analyze those to provide depth in understanding.
This requires methods for end-user involvement that share the kinds of long-term, deep engagements
employed in participatory design (as it was initially practiced in the Scandinavian countries [110]).
While we emphasize that there is rigor in our practice, this is not in opposition to seeing design
as a creative practice, engaging with our tacit aesthetic sensibilities, fundamentally a studio-based
practice that will render different results depending on which studio and which designers are engaged.
6. Conclusions
Through the lens of the first-person perspective, we have uncovered some of the issues in
addressing aesthetics in soma-based designs. The first-person perspective runs like a thread through
the analysis of our design exemplars, the description of our design methods as well as our material
encounter strategies. Theoretical concepts such as lived body,aesthetic experience,somatics, and
somaesthetics provide some of the theoretical underpinnings for our first-person perspective. As such,
they are shaping our design practice, but they are also being filled with new insights and experiences
through our design work—blending and amalgamating into a new research program on soma-based
design. Our aim is to demystify soma-based design practices, to show that, when it is done as part
of a design research practice, there is rigor to both how the design is done and to the articulations of
gained knowledge.
We believe this work is timely, as we face a new wave of technologies, entering into every walk of
human life (in the era of Internet of Things, ubicomp, and big data—shifting from dialogue-oriented
interaction to faceless interfaces and implicit interaction). There is a window of opportunity where we
make designs harmonize with our basic human condition: our first-person, felt experiences.
But much work remains to be done, which is why we end with a call for action, inviting other
researchers to engage with soma-based interactions. Some of our questions are ancient philosophical
inquiries. We need to keep asking: what is (felt) experience? What is aesthetics? What is the ‘living
body’? As expressed by one of the workshop participants: there are so many things in doing this work
that have confused me”. This is a call for action, for more research and design work, for a rigorous
systematic treatment of aesthetics in soma-based design.
Author Contributions:
All the authors either participated or took part in organising the workshop. All authors
took part in the subsequent discussions and analysis of the First-Person perspective and its importance to their
work. The five systems described came from other, prior, projects involving some of the authors: the GangKlang
application was designed and built by Nassrin Hajinejad and colleagues; the Soma Mat was designed and
Informatics 2018,5, 8 22 of 26
built by Kristina Höök, Martin Jonsson, Anna Ståhl and colleagues; the Ears and Tails were designed and built
by Dag Svanæs and colleagues; the BrightHearts application was designed and built by George Khut; and
the Embodied Encounters Studio was designed and built by Caroline Hummels and colleagues. Likewise,
some of the design methods have been developed by authors: the Somatic Connoisseurship method by
Thecla Schiphorst; a variant of the autobiographical design method by Caroline Hummels; the Embodied
Sketching by Elena Marquez Segura; and the use of data and program code by Baptiste Caramiaux. Kristina Höök
organised and took overall responsibility for writing the manuscript.
Conflicts of Interest: The authors declare no conflicts of interest.
References
1.
Schiphorst, T. Self-evidence: Applying Somatic Connoisseurship to Experience Design. In Proceedings
of the CHI ’11 Extended Abstracts on Human Factors in Computing Systems, Vancouver, BC, Canada,
7–12 May 2011; pp. 145–160.
2.
Page, T. A Forecast of the Adoption of Wearable Technology. Int. J. Technol. Diffus.
2015
,6, 12–29. [CrossRef]
3.
Verbeek, P. Materializing Morality Design Ethics and Technological Mediation. Sci. Technol. Hum. Values
2006,31, 361–380. [CrossRef]
4.
Ferreira, P.; Höök, K. Bodily Orientations Around Mobiles: Lessons Learnt in Vanuatu. In Proceedings of
the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada, 7–12 May 2011;
pp. 277–286.
5.
Segura, E.M.; Vidal, L.T.; Rostami, A.; Waern, A. Embodied Sketching. In Proceedings of the 2016 CHI
Conference on Human Factors in Computing Systems, San Jose, CA, USA, 7–12 May 2016; pp. 6014–6027.
6.
Simon, B. Wii are Out of Control: Bodies, Game Screens and the Production of Gestural Excess. Social Science
Research Network: Rochester, NY. Available online: http://papers.ssrn.com/abstract=1354043 (accessed on
20 September 2016).
7.
Joshua, T.; Karen, T. Envisioning the Future of Wearable Play: Conceptual Models for Props and Costumes as Game
Controllers; The Allen Institute for Artificial Intelligence: Seattle, WA, USA, 2015.
8.
Billinghurst, M.; Clark, A.; Lee, G. A Survey of Augmented Reality. Found. Trends Hum.-Comput. Interact.
2015,8, 73–272. [CrossRef]
9.
Gillies, M. What is Movement Interaction in Virtual Reality for? In Proceedings of the 3rd International
Symposium on Movement and Computing, Thessaloniki, GA, Greec, 5–6 July 2016; pp. 31:1–31:4.
10. Redström, J. Making Design Theory; MIT Press: Cambridge, MA, USA, 2017.
11.
Stolterman, E. The Nature of Design Practice and Implications for Interaction Design Research. Int. J. Des.
2008,2, 55–65.
12.
Hannah, T. What is Somatics? In Bone, Breath & Gesture: Practices of Embodiment; Hanlon-Johnson, D., Ed.;
North Atlantic Books: Berkeley, CA, USA, 1995; pp. 341–352.
13.
Shusterman, R. Thinking through the Body: Essays in Somaesthetics; Cambridge University Press: Cambridge,
UK, 2012.
14.
Hummels, C.; Overbeeke, K.C.J.; Klooster, S. Move to get moved: A search for methods, tools and knowledge
to design for expressive and rich movement-based interaction. Pers. Ubiquitous Comput.
2006
,11, 677–690.
[CrossRef]
15.
Löwgren, J.; Stolterman, E. Thoughtful Interaction Design: A Design Perspective on Information Technology;
MIT Press: Cambridge, MA, USA, 2004.
16. Gaver, B.; Bowers, J. Annotated Portfolios. Interactions 2012,19, 40–49. [CrossRef]
17.
Gaver, W. What Should We Expect from Research Through Design? In Proceedings of the SIGCHI Conference
on Human Factors in Computing Systems, Austin, TX, USA, 5–10 May 2012; pp. 937–946.
18.
Hallnäs, L. On the Foundations of Interaction Design Aesthetics: Revisiting the Notions of Form and
Expression. Int. J. Des. 2011,5, 73–84.
19. Archer, B. The Nature of Research. Co-Des. J. 1995,2, 6–13.
20. Cross, N. Designerly Ways of Knowing; Springer Science & Business Media: London, UK, 2006.
21.
Fallman, D. Design-oriented Human-computer Interaction. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, Ft. Lauderdale, FL, USA, 5–10 April 2003; pp. 225–232.
22.
Christopher Frayling and Royal College of Art. Research in Art and Design; Royal College of Art: London,
UK, 1993.
Informatics 2018,5, 8 23 of 26
23.
Höök, K.; Jonsson, M.; Ståhl, A.; Tholander, J.; Robertson, T.; Marti, P.; Svanaes, D.; Petersen, M.G.; Forlizzi, J.;
Schiphorst, T.; et al. Move to Be Moved. In Proceedings of the 2016 CHI Conference Extended Abstracts on
Human Factors in Computing Systems, San Jose, CA, USA, 7–12 May 2016; pp. 3301–3308.
24.
Klooster, S.; Overbeeke, C.J. Designing products as an integral part of choreography of interaction:
The product’s form as an integral part of movement. In Proceedings of the 1st International Workshop on
Design and Semantics of Form and Movement, New Castle, UK, 2005.
25.
Loke, L.; Robertson, T. Moving and Making Strange: An Embodied Approach to Movement-based Interaction
Design. ACM Trans. Comput.-Hum. Interact. 2013,20, 7:1–7:25. [CrossRef]
26.
Wright, P.; McCarthy, J. Empathy and Experience in HCI. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, Florence, Italy, 5–10 April 2008; pp. 637–646.
27.
Zimmerman, J.; Forlizzi, J.; Evenson, S. Research through Design as a Method for Interaction Design Research
in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’07),
San Jose, CA, USA, 28 April–3 May 2007; pp. 493–502.
28.
Merleau-Ponty, M.; Smith, C. Phenomenology of Perception; Motilal Banarsidass Publisher: Delhi, India, 1996.
29.
Svanæs, D. Kinaesthetic thinking: The tacit dimension of interaction design. Comput. Hum. Behav.
1997
,13,
443–463. [CrossRef]
30.
Svanæs, D. Interaction Design for and with the Lived Body: Some Implications of Merleau-ponty’s
Phenomenology. ACM Trans. Comput. Hum. Interact. 2013,20. [CrossRef]
31. Merleau-Ponty, M. The child’s relations with others. Child. Crit. Concepts Sociol. 2005,3, 102.
32. Dewey, J. Art as Experience; Penguin: New York, NY, USA, 2005.
33.
Beuthel, J.M.; Wilde, D. Wear.x: Developing Wearables That Embody Felt Experience. In Proceedings of the
2017 Conference on Designing Interactive Systems, Edinburgh, UK, 10–14 June 2017; pp. 915–927.
34.
Larssen, A.T.; Robertson, T.; Edwards, J. The Feel Dimension of Technology Interaction: Exploring Tangibles
through Movement and Touch. In Proceedings of the 1st International Conference on Tangible and Embedded
Interaction, Baton Rouge, LA, USA, 15–17 February 2007; pp. 271–278.
35.
Höök, K. Transferring qualities from horseback riding to design. In Proceedings of the 6th Nordic Conference
on Human-Computer Interaction: Extending Boundaries, Reykjavik, Iceland, 16–20 October 2010; pp. 226–235.
36.
Lee, W.; Lim, Y.; Shusterman, R. Practicing Somaesthetics: Exploring Its Impact on Interactive Product
Design Ideation. In Proceedings of the 2014 Conference on Designing Interactive Systems, At Vancouver, BC,
Canada, 21–25 June 2014; pp. 1055–1064.
37.
Löwgren, J. Annotated Portfolios and Other Forms of Intermediate-level Knowledge. Interactions
2013
,20,
30–34. [CrossRef]
38.
Neustaedter, C.; Sengers, P. Autobiographical Design in HCI Research: Designing and Learning Through
Use-it-yourself. In Proceedings of the Designing Interactive Systems Conference, Newcastle Upon Tyne, UK,
11–15 June 2012; pp. 514–523.
39.
Fiore, S.; Wright, P.; Edwards, A. A Pragmatist Aesthetics Approach to the Design of a Technological Artefact.
In Proceedings of the 4th Decennial Conference on Critical Computing: Between Sense and Sensibility,
Aarhus, Denmark, 21–25 August 2005; pp. 129–132.
40.
Forlizzi, J.; Battarbee, K. Understanding Experience in Interactive Systems. In Proceedings of the 5th
Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, Cambridge,
MA, USA, 1–4 August 2004; pp. 261–268.
41. McCarthy, J.; Wright, P. Technology as Experience. Interactions 2004,11, 42–43. [CrossRef]
42.
Petersen, M.G.; Iversen, O.S.; Krogh, P.G.; Ludvigsen, M. Aesthetic Interaction: A Pragmatist’s Aesthetics
of Interactive Systems. In Proceedings of the 5th Conference on Designing Interactive Systems: Processes,
Practices, Methods, and Techniques, Cambridge, MA, USA, 1–4 August 2004; pp. 269–276.
43.
Schiphorst, T. The Varieties of User Experience Bridging Embodied Methodologies from Somatics and
Performance to Human Computer Interaction. 2009. Available online: https://pearl.plymouth.ac.uk/
/handle/10026.1/2177 (accessed on 20 September 2016).
44.
Shusterman, R. Body Consciousness: A Philosophy of Mindfulness and Somaesthetics; Cambridge University
Press: Cambridge, UK, 2008.
45. Alexander, F.M. The Use of the Self. Br. Med. J. 1932,2, 77–78. [CrossRef]
46.
Feldenkrais, M. Awareness through Movement: Health Exercises for Personal Growth; Arkana: London, UK, 1990.
47.
Sheets-Johnstone, M. The Primacy of Movement; John Benjamins Publishing: Amsterdam, The Netherlands, 2011.
Informatics 2018,5, 8 24 of 26
48.
Hug, J. Writing with practice: Body Weather performance training becomes a medium of artistic research.
Theatre Dance Perform. Train. 2016,7, 168–189. [CrossRef]
49.
Wilson, M.L.L.; Resnick, P.; Coyle, D.; Chi, E.H. RepliCHI: The Workshop. In Proceedings of the CHI ’13
Extended Abstracts on Human Factors in Computing Systems, Paris, France, 2 May 2013; pp. 3159–3162.
50.
Nelson, H.G.; Stolterman, E. Design Judgement: Decision-Making in the “Real” World. Des. J.
2003
,6, 23–31.
[CrossRef]
51.
Stappers, P.J. Doing Design as a Part of Doing Research. In Design Research Now; Michel, R., Ed.; Birkhäuser:
Basel, Switzerland, 2007; pp. 81–91.
52.
Zimmerman, J.; Stolterman, E.; Forlizzi, J. An Analysis and Critique of Research through Design: Towards a
Formalization of a Research Approach. In Proceedings of the 8th ACM Conference on Designing Interactive
Systems (DIS ’10), Aarhus, Denmark, 16–20 August 2010; pp. 310–319.
53.
Isbister, K.; Segura, E.M.; Kirkpatrick, S.; Chen, X.; Salahuddin, S.; Cao, G.; Tang, R. Yamove! A movement
synchrony game that choreographs social interaction. Hum. Technol. 2016,12, 74–102. [CrossRef]
54.
Sundström, P.; Höök, K. Hand in hand with the material: Designing for suppleness. In Proceedings of
the SIGCHI Conference on Human Factors in Computing Systems, Atlanta, GA, USA, 10–15 April 2010;
Volume 463.
55.
Frens, J.W. Designing for Rich Interaction: Integrating form, Interaction, and Function; Technische Universiteit
Eindhoven: Eindhoven, The Netherlands, 2006.
56.
Hallnäs, L.; Redström, J. Interaction Design: Foundations, Experiments; Textile Research Centre, Swedish School
of Textiles, Unversity College of Borås and Interactive Institute: Borås, Sweden, 2006. Available online:
http://eprints.sics.se/922/ (accessed on 21 September 2016).
57.
Bowers, J. The Logic of Annotated Portfolios: Communicating the Value of “Research through Design”.
In Proceedings of the Designing Interactive Systems Conference, Newcastle Upon Tyne, UK, 11–15 June 2012;
pp. 68–77.
58.
Höök, K.; Löwgren, J. Strong Concepts: Intermediate-level Knowledge in Interaction Design Research.
ACM Trans. Comput.-Hum. Interact. 2012,19, 23:1–23:18. [CrossRef]
59.
Koskinen, I.; Zimmerman, J.; Binder, T.; Redstrom, J.; Wensveen, S. Design Research through Practice: From the
Lab, Field, and Showroom; Elsevier: Amsterdam, The Netherlands, 2011.
60.
Booth, W.C.; Colomb, G.G.; Williams, J.M. The Craft of Research; University of Chicago Press: Chicago, IL,
USA, 2008.
61.
Oulasvirta, A. Can Computers Design Interaction? In Proceedings of the 8th ACM SIGCHI Symposium on
Engineering Interactive Computing Systems, Brussels, Belgium, 21–24 June 2016; pp. 1–2.
62.
Khut, G.P. Designing Biofeedback Artworks for Relaxation. In Proceedings of the 2016 CHI Conference
Extended Abstracts on Human Factors in Computing Systems, San Jose, CA, USA, 7–12 May 2016;
pp. 3859–3862.
63. Höök, K. Soma Design; MIT Press: Cambridge, MA, USA, 2018.
64.
Höök, K.; Jonsson, M.P.; Ståhl, A.; Mercurio, J. Somaesthetic Appreciation Design. In Proceedings of the 2016
CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, 7–12 May 2016; pp. 3131–3142.
65.
Höök, K.; Ståhl, A.; Jonsson, M.; Mercurio, J.; Karlsson, A.; Johnson, E.-C.B. COVER STORY: Somaesthetic
Design. Interactions 2015,22, 26–33. [CrossRef]
66.
Jonsson, M.; Ståhl, A.; Mercurio, J.; Karlsson, A.; Ramani, N.; Höök, K. The Aesthetics of Heat: Guiding
Awareness with Thermal Stimuli. In Proceedings of the Tenth International Conference on Tangible,
Embedded, and Embodied Interaction, Eindhoven, The Netherlands, 14–17 February 2016; pp. 109–117.
67.
Svanaes, D.; Solheim, M. Wag Your Tail and Flap Your Ears: The Kinesthetic User Experience of Extending
Your Body. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing
Systems, San Jose, CA, USA, 7–12 May 2016; pp. 3778–3779.
68.
Brandslet, S. The Professor Who Misses His Tail. Geminiresearchnews.com, 2015. Available online: https:
//geminiresearchnews.com/2015/12/the-professor-who-misses-his-tail/ (accessed on 16 October 2017).
69.
Hajinejad, N.; Grüter, B.; Roque, L.; Bogutzky, S. GangKlang: Facilitating a movement-oriented walking
experience through sonic interaction. In Proceedings of the 10th Audio Mostly Conference, Norrköping,
Sweden, 4–6 October 2016; pp. 202–208.
70. Wolterstorff, N. Art Rethought: The Social Practices of Art; Oxford University Press: Oxford, UK, 2015.
Informatics 2018,5, 8 25 of 26
71.
Hajinejad, N.; Grüter, B.; Roque, L. Prototyping sonic interaction for walking. In Proceedings of the 19th
International Conference on Human-Computer Interaction with Mobile Devices and Services, Vienna,
Austria, 4–7 September 2017.
72.
Hummels, C.; Trotto, A. Designing in Skills Studio. In Proceedings of the 8th International Conference on
Tangible, Embedded and Embodied Interaction, Munich, Germany, 16–19 February 2013; pp. 357–360.
73.
Trotto, A.; Hummels, C. Engage Me, Do!: Engagement Catalysers to Ignite a (Design) Conversation.
In Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces
(DPPI ’13), Newcastle upon Tyne, UK, 3–5 September 2013; pp. 136–145.
74.
Hummels, C.; van Dijk, J. Seven Principles to Design for Embodied Sensemaking. In Proceedings of the
Ninth International Conference on Tangible, Embedded, and Embodied Interaction, Stanford, CA, USA,
15–19 January 2015; pp. 21–28.
75.
Hummels, C. Embodied Encounters Studio: A Tangible Platform for Sensemaking. In Proceedings of the
2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, San Jose, CA, USA,
7–12 May 2016; pp. 3691–3694.
76.
Aslan, I.; Burkhardt, H.; Kraus, J.; André, E. Hold My Heart and Breathe with Me: Tangible Somaesthetic
Designs. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction, Gothenburg,
Sweden, 23–27 October 2016; pp. 92:1–92:6.
77.
Bergström, I.; Jonsson, M. Sarka: Sonification and Somaesthetic Appreciation Design. In Proceedings of
the 3rd International Symposium on Movement and Computing, Thessaloniki, GA, Greece, 5–6 July 2016;
pp. 1:1–1:8.
78.
Dassen, W.; Alonso, M.B. Aesthetics of Haptics: An Experience Approach to Haptic Interaction Design.
In Proceedings of the 2017 ACM Conference Companion Publication on Designing Interactive Systems,
Edinburgh, UK, 10–14 June 2017; pp. 254–259.
79.
Feltham, F.; Loke, L.; van den Hoven, E.; Hannam, J.; Bongers, B. The Slow Floor: Increasing Creative Agency
While Walking on an Interactive Surface. In Proceedings of the 8th International Conference on Tangible,
Embedded and Embodied Interaction, Munich, Germany, 16–19 February 2014; pp. 105–112.
80.
Isbister, K.; Höök, K. On Being Supple: In Search of Rigor without Rigidity in Meeting New Design and
Evaluation Challenges for HCI Practitioners. In Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems, Boston, MA, USA, 4–9 April 2009; pp. 2233–2242.
81.
Khut, G.P.; Morrow, A.; Watanbe, M.Y. The BrightHearts Project: A New Approach to the Management of
Procedure-Related Paediatric Anxiety; OzCHI Workshop Program: Canberra, Australia, 2011.
82.
Loke, L.; Khut, G.P.; Slattery, M.; Truman, C.; Muller, L.; Duckworth, J. Re-sensitising the body: interactive
art and the Feldenkrais method. Int. J. Arts Technol. 2013,6, 339–356. [CrossRef]
83.
Mailvaganam, A.; Alonso, M.B. Haptic beats: Designing for rich haptic interaction in a music controller.
In Proceedings of the 9th International Conference on Design and Semantics of Form and Movement, Milano,
Italy, 13–17 October 2015.
84.
Pijnappel, S.; Mueller, F. 4 Design Themes for Skateboarding. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, Paris, France, 27 April–2 May 2013; pp. 1271–1274.
85.
Schiphorst, T. Soft(N): Toward a Somaesthetics of Touch. In Proceedings of the CHI ’09 Extended Abstracts
on Human Factors in Computing Systems, Boston, MA, USA, 4–9 April 2009; pp. 2427–2438.
86.
Schiphorst, T.; Chung, W.; Ip, E. Wo.Defy: Wearable Interaction Design Inspired by a Chinese 19th Century
Suffragette Movement. In Proceedings of the 7th International Conference on Tangible, Embedded and
Embodied Interaction, Barcelona, Spain, 10–13 February 2013; pp. 319–322.
87.
Vidyarthi, J.; Riecke, B.E.; Gromala, D. Sonic Cradle: Designing for an Immersive Experience of Meditation
by Connecting Respiration to Music. In Proceedings of the Designing Interactive Systems Conference,
Newcastle Upon Tyne, UK, 11–15 June 2012; pp. 408–417.
88.
Wilde, D. hipDisk: Using sound to encourage physical extension, exploring humour in interface design.
Int. J. Perform. Arts Digit. Media 2008,4, 7–26. [CrossRef]
89.
Höök, K. Affective Loop Experiences—What Are They? In Persuasive Technology; Oinas-Kukkonen, H.,
Hasle, P., Harjumaa, M., Segerståhl, K., Øhrstrøm, P., Eds.; Springer: Berlin/Heidelberg, Germany, 2008;
pp. 1–12.
90.
Norman, D.; Tognazzini, B. How Apple Is Giving Design a Bad Name. Co.Design, 2015. Available online: https:
//www.fastcodesign.com/3053406/how-apple-is-giving-design-a-bad-name (accessed on 16 January 2018).
Informatics 2018,5, 8 26 of 26
91.
Khut, G.P. Development and Evaluation of Participant-Centred Biofeedback Artworks. Ph.D. Thesis, University
of Western Sydney, Sydney, Australia, 2006. Available online: http://researchdirect.westernsydney.edu.au/
islandora/object/uws%3A2425/ (accessed on 13 October 2017).
92. James, W. The experience of activity. Psychol. Rev. 1905,12, 1–17. [CrossRef]
93.
Wilde, D.; Vallgårda, A.; Tomico, O. Embodied Design Ideation Methods: Analysing the Power of Estrangement.
In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17), Denver, CO,
USA, 6–11 June 2017; pp. 5158–5170.
94. Davies, E. Beyond Dance: Laban’s Legacy of Movement Analysis; Routledge: New York, NY, USA, 2006.
95.
Loke, L.; Larssen, A.T.; Robertson, T. Labanotation for Design of Movement-based Interaction. In Proceedings
of the Second Australasian Conference on Interactive Entertainment, Sydney, Australia, 23–25 November
2005; pp. 113–120.
96.
Mentis, H.M.; Laaksolahti, J.; Höök, K. My Self and You: Tension in Bodily Sharing of Experience. ACM Trans.
Comput.-Hum. Interact. 2014,21, 20:1–20:26. [CrossRef]
97.
Sundström, P. Exploring the Affective Loop. Stockholm University, 2005. Available online: http://eprints.
sics.se/98/ (accessed on 20 September 2016).
98.
Tobiasson, H.; Hedman, A.; Gulliksen, J. Less Is Too Little—More Is Needed: Body-Motion Experience as a Skill in
Design Education; Digitala Vetenskapliga Arkivet: Uppsala, Sweden, 2014; pp. 1327–1341.
99.
Segura, E.M. Co-creating Embodied Sketches Playing as a Method to Design with Children. In Proceedings of
the 12th International Conference on Advances in Computer Entertainment Technology, Iskandar, Malaysia,
16–19 November 2015.
100.
Fiebrink, R.; Caramiaux, B. The Machine Learning Algorithm as Creative Musical Tool. In Oxford Handbook
on Algorithmic Music; Oxford University Press: Oxford, UK, 2016.
101.
Françoise, J. Motion-Sound Mapping by Demonstration. UPMC, 2015. Available online: http://hal.upmc.fr/
tel-01161965/document (accessed on 20 September 2016).
102.
Caramiaux, B.; Altavilla, A.; Pobiner, S.G.; Tanaka, A. Form Follows Sound: Designing Interactions from
Sonic Memories. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing
Systems, Seoul, Korea, 18–23 April 2015; pp. 3943–3952.
103.
Bogers, S.; Frens, J.; van Kollenburg, J.; Deckers, E.; Hummels, C. Connected Baby Bottle: A Design Case
Study Towards a Framework for Data-Enabled Design. In Proceedings of the 2016 ACM Conference on
Designing Interactive Systems, Brisbane, QLD, Australia, 4–8 June 2016; pp. 301–311.
104.
Hobye, M.; Löwgren, J. Touching a Stranger: Designing for Engaging Experience in Embodied Interaction.
Int. J. Des. 2011,5, 31–48.
105.
Benford, S.; Greenhalgh, C.; Giannachi, G.; Walker, B.; Marshall, J.; Rodden, T. Uncomfortable Interactions.
In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, USA,
5–10 May 2012; pp. 2005–2014.
106.
Mueller, F.F.; Pell, S.J. Technology Meets Adventure: Learnings from an Earthquake-interrupted Mt. Everest
Expedition. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous
Computing, Heidelberg, Germany, 12–16 September 2016; pp. 817–828.
107.
Schutz, A. The Phenomenologasy of the Social World; Northwestern University Press: Evanston, IL, USA, 1967.
108.
Fogtmann, M.H.; Grønbæk, K.; Ludvigsen, M.K. Interaction technology for collective and psychomotor
training in sports. In Proceedings of the 8th International Conference on Advances in Computer
Entertainment Technology, Lisbon, Portugal, 8–11 November 2011.
109.
Schon, D.A. The Reflective Practitioner: How Professionals Think in Action; Basic Books: New York, NY,
USA, 1984.
110.
Bødker, S.; Ehn, P.; Sjögren, D.; Sundblad, Y. Cooperative Design—Perspectives on 20 years with the
Scandinavian IT Design Model. In Proceedings of the NordiCHI 2000, Stockholm, Sweden, 23–25 October 2000.
©
2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).
... Therefore, I chose to focus on their firstperson perspective to understand their experiences. (Additionally, previous research by Höök et al. (2018) presents the need to embrace the first-person perspective in body-based design.) I used qualitative methods like semi-structured interviews and structured observation to capture the nuances in each dancer's personal experience. ...
... Another examples is Höök et al. (2018)'s work unpacking and explaining the strong first-person perspective through: soma-based design exemplars; methodologies like disrupting the habitual and autobiographical design; and theoretical underpinnings like Merleau-Ponty (1996)'s phenomenology of the body and definition of the firstperson perspective, Dewey (2005)'s aesthetic experience, and Shusterman (2008)'s somaesthetics. With this philosophical framing, Höök et al. engaged in a conference workshop, interacting with some participants' design exemplars, and discussing approaches for attending to the designer's bodily self while working with design materials. ...
Thesis
Design methodologies for gesture- and movement-based interaction take divergent perspectives, either quantifying movement from an external point of view or supporting inner sensation exploration and description. However, professional movers, like dancers, define poses and phrases using both descriptions from choreographers and personal, bodily cues. I argue that dance style transitions offer researchers insights into dancers’ experiences questioning and balancing the externally codified and internally not-possible-to-be-codified parts of their training. After employing qualitative methods, I present the challenges and strategies of the dance style transition and discuss how structuring technology around movement substrates could support the mentality change necessary for dancers to access new types of movement. I then describe the ImproviGrid System and discuss how output modality impacts dancer use of cues during improvisation, in order to understand how to design better for movement exploration, a challenge of the dance style transition. I conclude with greater implications of user-designed movement methodologies, reframing them as users creating choreographed gestures rather than researchers eliciting gestures.
... Their gestural vocabulary combines nuanced musical gestures with dance-like larger body movements. During a performance, these instruments deliver an embodied listening experience to the audience while simultaneously emphasizing the felt, bodily experience of music-making for the performer (Engelsrud, 2005;Höök et al., 2018;Mah et al., 2021). ...
Article
Full-text available
Inclusive musical instruments benefit from incorporating wearable interfaces into digital musical instrument design, creating opportunities for bodily felt experiences and movement-based interactions. In this article, we discuss the evolution of our inclusive design approach behind the design and performance practices of three wearable musical instruments. By focusing on the embodied, somatic, and tacit dimensions of movement-based musical interaction, we evaluate these case studies, combining the third and first-person perspectives. The design and implementation of the wearable sensing, utilizing the additive manufacturing techniques, are discussed for each instrument and its performer in specific cases of musical expression. This article further discusses how our approach integrates music performance as a crucial step into design and evaluation, utilizing these performance practices and such collaborative settings for improved diversity and inclusion. Finally, we examine how our design approach evolves from user-centered design to more participatory practices, offering people with diverse abilities a shared music performance space.
... Other researchers have focused on soma-based design and aesthetics of the interaction [15]. Similar to movement-based interaction design, somaesthetics focus on the instrumentality of the body [30] and extend this framework with the social, emotional, 1 https://www.uio.no/ritmo/english/research/labs/fourms/software/musicalgesturestoolbox/ and aesthetic aspects and design considerations. ...
Conference Paper
Full-text available
Movement-based musical interfaces support performers’ music and movement expressions by drawing from expertise and creative practices of both disciplines. In this work, we qualitatively and quantitatively analyze the movement interaction of participants with Bodyharp, a movement-based musical instrument. This wearable instrument offers musical affordances that allow performers to extend beyond small gestural spaces. Its wearable design encourages the performers to move while creating music and to express while using their bodies. Data was collected from twenty participants’ interactions, reflections, and compositions with Bodyharp. Video recordings of the experiment were annotated and qualitatively analyzed to reveal which performed gestures directly contribute to sound production and modification and which gestures accompany these musical actions. For a subset of participants, Musical Gestures Toolbox was used to further quantify the gestures. Using the Laban Movement Analysis framework, we observed participants’ use of space and body in their interaction with a movement-based musical instrument and how their backgrounds in music or movement (based on participants’ self-reported experiences) influenced the interaction. Our results offer design practices for creating new interactions at the intersection of music and dance.
... Engaging with the felt experience of touching and being touched, both by humans and machines, is one viable path to understanding the semantics of digital touch [39]. Such research has enriched our understanding through, for example, identifying linguistic terms to convey tactile sensitivities [31,45], or meaningful terminology to describe haptic stimuli [68] which in turn informs the development of haptic technologies. ...
Article
Full-text available
We report on a soma design process, where we designed a novel shape-changing garment—the Soma Corset. The corset integrates sensing and actuation around the torso in tight interaction loops. The design process revealed how boundaries between the garment and the wearer can become blurred, leading to three flavours of cyborg relations. First, through the lens of the monster, we articulate how the wearer can adopt or reject the garment, resulting in either harmonious or disconcerting experiences of touch. Second, it can be experienced as an organic “other”-with its own agency-resulting in uncanny experiences of touch. Through mirroring the wearer’s breathing, the garment can also be experienced as a twisted version of one’s own body. We suggest that a gradual sensitisation of designers-through soma design and reflection on the emerging human-technology relations-may serve as a pathway for uncovering and articulating novel, machine-like, digital touch experiences.
Conference Paper
Disseminating dance in online spaces provides an opportunity for kinesthetic knowledge to reach broader audiences. However, the transmission of dance in online spaces also primarily relies on visual and aural modalities that cannot fully capture the nuanced physical sensations of a kinesthetic experience. In recent years there has been an influx of interactive online dance resources; yet there is little analysis of how these works effectively translate kinesthetic knowledge to online audiences. We bring together research in dance film and interaction design practices to explore kinesthetic transmission in online spaces and conduct analyses on interactive digital dance resources. Based on our literature review and analyses of these dance resources we propose the I-TEC design framework for kinesthetic transmission. In this framework, Instructional, Translational, Exploratory, and Contextual interactions are brought together to expose the multiple embodiments, perspectives, and translations of kinesthesia.
Article
Full-text available
Atendiendo a la necesidad de integrar las sensibilidades somáticas a la enseñanza del diseño, este artículo ofrece un recorrido por una actividad realizada en un taller cuyo propósito fue introducir las leyes de la Gestalt a través de cultivar la sensibilidad somática. En lugar de seguir una ruta exclusivamente visual para aprender las leyes de la Gestalt, pedimos a los estudiantes que utilizaran sus sentidos y una serie de materiales como vehículos para representar, a través de sus diseños, las sensaciones corporales de su experiencia interior. Al aplicar los pasos propuestos, observamos que ciertos patrones de respuestas que informaban sobre el movimiento, la intensificación y la dispersión de la conciencia estaban asociados a leyes de la Gestalt específicas. Este artículo informa sobre las primeras aproximaciones de este enfoque, sugiriendo vías que podrán ser exploradas por los educadores en el ámbito del diseño a medida que los estudiantes aumentan su sensibilidad somática, tales como la integración de restricciones o la inclusión de interacciones que comprometan a todo el cuerpo con los materiales.
Conference Paper
Full-text available
Embodied design ideation practices work with relationships between body, material and context to enliven design and research potential. Methods are often idiosyncratic and due to their physical nature not easily transferred. This presents challenges for designers wishing to develop and share techniques or contribute to research. We present a framework that enables designers to understand, describe and contextualise their embodied design ideation practices in ways that can be understood by peers, as well as those new to embodied ideation. Our framework developed over two conference workshops provides a frame for discussion of embodied design actions that leverage the power of estrangement. We apply our framework to eight embodied design ideation methods. Our contribution is thus twofold: (1) a framework to understand and leverage the power of estrangement in embodied design ideation, and (2) an inspirational catalogue demonstrating the diversity of ideas that embodied design ideation methods can foster.
Conference Paper
Full-text available
In this paper we discuss the design process and results from a design exploration on the use of thermal stimuli in body awareness exercises. A user-study was performed on an interactive prototype in the form of an interactive heat mat. The paper brings forth an alternative understanding of heat as a design material that extends the common understanding of thermal stimuli in HCI as a communication modality to instead bring the aesthetic and experiential properties to the fore. Findings account for felt body experiences of thermal stimuli and a number of design qualities related to heat as a design material are formulated, pointing to experiential qualities concerning the felt body, subjectivity and subtleness as well as material qualities concerning materiality, inertia and heat transfer.
Article
This research aims to quantify the current market size for wearable technology, and determine why this market has struggled over the past decade. These are products which are worn on the body and enhanced using electronics. Forecasts have been made as to how this wearable technology is likely to develop in terms of market size and product design or function. It is predicted that in five years the wearable technology market will be several times larger than it is currently, and entertainment devices will overtake fitness to become the largest product category. Medical devices will be used to reduce healthcare costs by monitoring patients within their own home and wearable technology will allow businesses to improve customer relations and productivity.
Conference Paper
Sounds play a substantial role in the experience of movement activities such as walking. Drawing on the movement inducing effects of sound, sonic interaction opens up numerous possibilities to modify the walker's movements and experience. We argue that designing sonic interaction for movement activities demands an experiential awareness of the interplay of sound, body movement and use situation, and, propose a prototyping method to understand possibilities and challenges related to the design of mobile sonic interaction. In this paper, we present a rapid prototyping system that enables non-expert users to design sonic interaction for walking and to experience their design in the real-world context. We discuss the way this prototyping system allows designers to experience how their design ideas unfold in mobile use and affect the walking.
Book
A leading MIT social scientist and consultant examines five professions--engineering, architecture, management, psychotherapy, and town planning--toshow how professionals really go about solving problems.
Conference Paper
Technological developments are expanding the interaction possibilities for the haptic domain, allowing designers to program dynamic haptic feedback. While haptic sensations are emerging in design practice, the relation between programmable haptics and the user experience remains underexplored. This study presents a research-through-design approach on the design of a push button and a knob with multiple haptic sensations. Interaction design of these haptic sensations started from an experience perspective building on material qualities. We reflect on the design process and present future directions for research.
Conference Paper
Physical discomfort can be highly personal, difficult to discern from the outside, challenging to effectively communicate. Yet communicating discomfort can be of great value. We present a method for developing wearables that transfer one person's discomfort to another: a modified fashion ideation process that enables a person to bring their hidden embodied experiences into wearable form. Using five complementary foci, the method seeks to simulate rather than replicate; to support people to find abstracted expressions for their lived experiences of discomfort, with which to negotiate shared understanding. The resulting wearables support empathic engagement with how another person might feel. Physical discomfort can be highly personal, difficult to discern from the outside, challenging to effectively communicate. Yet communicating discomfort can be of great value. We present a method for developing wearables that transfer one person's discomfort to another: a modified fashion ideation process that enables a person to bring their hidden embodied experiences into wearable form. Using five complementary foci, the method seeks to simulate rather than replicate; to support people to find abstracted expressions for their lived experiences of discomfort, with which to negotiate shared understanding. The resulting wearables support empathic engagement with how another person might feel.
Conference Paper
Sedentary lifestyle and resulting health problems have increased interest in design strategies to supports people's motivation for physical activity. In our experience-driven approach, we focus on renewing walkers' sensibility to embodied aesthetic qualities of the processing activity itself. In this paper, we present our steps to design sonic interaction tailored towards re-emphasizing the bodily dimension within walker's experience. We designed and developed a mobile application that makes the user's walking movement audible. Using our mobile application, users' walking movements trigger sounds that evolve into a dynamic soundscape changing in real time. We discuss initial findings on how listening to a walking-generated soundscape influences walkers' experience in terms of attention focus.