Available via license: CC BY 4.0
Content may be subject to copyright.
Improvisieren
Playing with
Virtual Realities.
A Practice-
based-Research
Experiment
in Dancing with
Technology
Einav Katan-Schmid
95 Improvisieren
Playing with Virtual Realities.
A Practice-based-Research Experiment in Dancing with Technology
1 Experiential Point of View: Emergence by Chance
‘Playing with Virtual Realities’ (‘PwVR’) is an interdisciplinary
practice-based research project (Nelson 2013) that was born by
chance inside the experimental zone of the Interdisciplinary
Laboratory, an Excellence Cluster of Humboldt University of Berlin.
The experimental zone, itself a research experiment, is the open
working space of the Interdisciplinary Laboratory, where scholars
from approximately forty different disciplines are clustered in
working groups of interdisciplinary research collaborations. One of
the aims of the experimental zone is to accelerate accidental
interactions between the laboratory’s multi-disciplinary scholars.
The emergence of ‘PwVR’ followed the pattern set by these circum-
stances. One day in December 2016, I was writing at my table and
observed Christian Stein, a computer scientist, linguist, gamer, and
co-founder of the research group ‘gamelab.berlin’, playing an arcade
game (Space Pirate Trainer), while using the VR technology HTC
VIVE in the other corner of the room. At the time I saw him, I was
working on a lecture presentation which dealt with my research
on dance and embodied cognition. The presentation had as its
subject the design of spatiality within dance movements. Following
from Maurice Merleau-Ponty’s Phenomenology of Perception and his
work on the phenomenology of human motility and spatiality
(1945), my argument emphasized the cognitive processes of design-
ing motility in relation to imaginary spatial instructions in For-
sythe’s ‘Improvisation Technologies’. Dealing with Forsythe’s
example of conducting the body in space, I claimed that his dancing
embodies an imaginary score as a motor-motivation. As I argued,
while dancing he concentrates on developing precision of motility
by integrating imagination (not visible visions) and perceptual
affordance within the current environment (Katan-Schmid 2017).
Occupied by my development of this argument, I saw Christian
Stein playing within VR technology, his vision so absorbed by the
game that he was oblivious to the physical surroundings in the
studio space. He wore goggles and used controllers in order to shoot
and to protect himself from drones that hunted him in a virtual
Einav Katan-Schmid
universe (as I later learnt). His body was alert, playful, and alive and
his agile movements covered the space. His motility was precisely
articulated. It appeared to me as if his movements brought to life an
imaginary world which was a creation of his private mind. In
practice, however, Stein was playing a game and the images inside
the VR guided his activities. His movements followed a realm he
could actually see and his alert reactions were immersed in a
present vision. Yet, to my eyes, Stein looked as if he were dancing.
I felt joy at seeing a fellow researcher move in this scientific
environment. Without reflecting on my philosophical views and
without thinking of the potential for a mutual research project, I
left my working table and started to improvise dancing movements
with him. When he left the virtual setting, I told him about my
playful experience, in which he participated without knowing. By
meeting a good sport who loves to play, by having the appropriate
academic environment for experimental explorations, and by
following my influential first encounter with a new engaging
interaction, the intermedial and interdisciplinary research collabo-
ration of dancing within VR technology ‘Playing with Virtual
Realities’ was born.
2 Experimental Points of View: Systemizing Inter-
disciplinary, Intermedial, Experiential, and Reflective
Explorations
Following from the Aristotelian definitions of the different five types
of wisdom, Western culture traditionally distinguishes between
artistic practices as a form of technical know-how to scientific knowl-
edge as a type of theoreticalepistemological—know-that (Aristotle
1994–2009, Chatzichristodoulou/Crossley 2016). The traditional
outlook on experimentations follows the clear distinction between
‘episteme’ and ‘techne’ as well; scientific research experiments are
traditionally empiric and aim to observe the natural world through
measuring causalities (Radder 2003: 2), while, in contrast, experi-
mental art is reputed to be ‘experiential’, and therefore not reflec-
tive. The division between the poles of a rigidly planned and
measured scientific experiment on the one side, and experiential—
lived through-experimentalism on the other side, can be seen for
example in Lydia Goehr’s sharp distinction between ‘experiment’
and ‘experimental’. Her analysis follows Theodor Adorno’s ana lysis
of the works of Francis Bacon, the founder of the scientific method
in modern science, and John Cage, a leading voice of American
experimentalism in art (Shultis 1998). ‘PwVR’ is a complex experi-
mentation in between those traditions and poles. The project deals
97 Improvisieren
Playing with Virtual Realities.
A Practice-based-Research Experiment in Dancing with Technology
with the experiential media of VR technology and of dancing, and
situates these as a source for academic explorations which are
theoretically driven. As a form of research, ‘PwVR’ plays with both
theoretical research and aesthetic practices, offering another
possibility of experimenting as well as another outlook on the
symbiosis between theory and practice in both artistic media and in
academic research.
The research project ‘PwVR’ united two dancers, a philosopher/
choreographer, two theatre scholars/dramaturges, two computer
scientists/gamers, and two media experience designers to co-explore
how the embodied practice of dancing can interact with HTC Vive,
a virtual reality headset developed by HTC and Valve Corporation.1
The venture was initiated by chance, but its establishment as a
research project systemizes an exchange between the scholars, the
artists, the artistic practice, the theoretical questioning, and the
technology. The core of our research is practice-based. We deal
with VR technology and with creative techniques (gaming, danc-
ing, and performing in front of an audience) as experiential sources
for embodied explorations and reflections. So far, the project has
had three interdisciplinary weekly workshops of inquiry into pos-
sible exchanges of dancing with VR technology, which resulted in
a performance publication and a further symposium with experts
from across disciplines.2 The project’s name implies our intentions
for exchange and the core of our investigation. Accordingly, VR
technology is our medium of research, but the play is not merely
with the technology, but with a diversity of bodies of knowledge
and with the virtual surplus each practice generates. Practically,
using the medium of VR as a metaphor for virtual perceptions
which are provoked by a technology, our hypothesis and enquiry
dealt with the epistemologies as they are generated by practices
and techniques (Mauss 1994, Rheinberger 1997, Miller et al. 2008,
Hoel 2012).
An inquiry into virtual reality as a metaphor for a private vision
and for epistemology was primarily performed through practice-
based knowledge in dance. It followed my philosophical investi-
gation on visual imageries as made-up instructions for movements
in dancing (Katan-Schmid 2016, 2017). In Forsythe’s ‘Improvisation
Technologies’, for example, dancers playfully deconstruct geo-
metrical patterns within their bodies and their kinespheres. In the
CD-ROM publication Improvisation Technologies: A Tool for the
Analytical Dance Eye, Forsythe draws imaginary lines in order to
1 Research trailer of ‘PwVR’: https://www.youtube.com/watch?v=YcGdKuUhMf4
(Accessed: July 30, 2018)
2 To watch the complete choreographic work online:
https://www.youtube.com/watch?v=2r_LOG7TaQA (Accessed: July 30, 2018)
Einav Katan-Schmid
explain the invisible imageries, with which his body interacts
(Forsythe/Haner 2012). In the CD-ROM, his imageries are ani-
mated and assume a visible graphic shape. The annotations illus-
trate how his dance movements are produced in relation to made-up
triggers for decision-making, which are clear and immediately
available to him. However, those annotations are interpretations by
Paul Kaiser, the annotator, after Forsythe was filmed. Playing with
the new possibilities of the VR technology, we could play dierently
with the investigation of mental imageries during the time of their
development by the dancers as they dance. The VR technology we
used initiated a new explorative question: “what if dancers can see
the score of their dancing?” As a metaphor, the VR became a space
for the dancer to step into their own minds.
Staging dancing with VR as a performance-publication enabled
us to work systematically on the alternation between hypotheses
and their fulfillments. We could experiment with relationships
between the visions of dancers, their bodily movements, and their
expressivity. Following cultural examples of dance notations and
annotations (Laban 1956, Forsythe/Haner 2012, deLahunta/
Hennermann 2013), we looked first for the opportunity to draw the
score of dancing. We used the VR application Tilt Brush by Google,
which was designed for drawing in a three-dimensional envi-
ronment. The dancers wore the VR gear and used the controllers
in order to draw lines, which (supposedly) illustrate their visions.
Additionally, following on from the initial chance of seeing the
gamer as dancer while he was playing with the application Space
Pirate Trainer in the experimental zone, we further explored how
dancers move in relation to an immersive virtual environment in an
arcade game. While the dancers were moving in the virtual space,
we projected the point of view of their gaze (their goggles function
as a camera), so the other scholars in the room could see what the
dancers see. After each session of exploring dancing within VR
application, we interviewed all the participants, dancers, as well as
the other scholars, and documented those interviews on video. The
questionnaire always included the same six questions, which led to
reflections on bodily awareness, imaginary interactions, and
decision-making, as those followed my initial theory on embodied
cognition in dancing.3 In addition to the fixed set of questions, we
started each day of the workshop with a warm-up exercise, sharing
3 The six questions we asked are:
How did you feel your body in the experience?
How did you interact with the image in the VR?
Did you follow an image which was not present in the VR?
What was your main instruction for decision-making?
How did the experience aect your emotions?
Do you have further reflections and thoughts which you’d like to share?
99 Improvisieren
Playing with Virtual Realities.
A Practice-based-Research Experiment in Dancing with Technology
our thoughts by answering two daily questions, which were
inspired by former experiences and discussions between us.4
We used the questionnaire as a platform to observe and reflect on
our actions. The dancers, Nitsan Margaliot and Lisanne Goodhue,
reflected, for example, on their movements in Space Pirate Trainer
as a reaction to the movement of the virtual spaceships, rather than
a conscious instance of decision-making. Alternatively, for Good-
hue, at first, seeing the virtual environment in Tilt Brush was
initially a paralyzing experience; she either followed her gaze and
forgot to dance, or danced while closing her eyes. Another layer
of analysis dealt with the gazes of the other scholars as external
viewers. Ramona Mosse, for example, a dramaturge in the research
group, and I described our lack of interest in what the dancers do
in Space Pirate Trainer, as long as we could see the projection of
their experiences. Regarding the first experiences of Goodhue with
the VR application Tilt Brush, the scholars in the room found her
bodily posture sensitive and expressive, although she was paralyzed
by the virtual environment and did not move much. When she
danced with her eyes closed, without knowing how to define it,
I felt that her dance was beautiful, but the projection of the images
from the goggles was not compelling. After Goodhue reflected
on her decision to close her eyes, we understood the lack of
directed gaze and agency in the images that were projected. The
questionnaires allowed us to share our visions. They also enabled
us to exchange knowledge and to expand the practice we created
in terms of design. Sabiha Ghellal, for instance, a media experience
designer, found the experience in Tilt Brush ambiguous, while
the choices of the dancers in Space Pirate Trainer are prescribed
by the setting of the virtual space (Ghellal 2017). Relying on the
defi nition of expression by the American pragmatist John Dewey
(Dewey 1934: 62),5 I explained t he lack of interest in watching the act i-
vity of the dancers as losing the appearance of a development of
meaning. Together, we concluded that, in order to suit dancing,
the experiences within Tilt-Brush are too ambiguous and therefore
need a clear structure which would allow the dancers to incor-
porate their visions as motivations for movements and to integrate
them within their decision-making. We also realized that the inte-
4 A few examples of these questions are:
What moves you?
When are you the most immersed in what you do?
What is your best technique?
How do you use virtual spheres?
Are there virtual spheres in your life which are not technological?
5 “To express is to stay by, to carry forward in development, to work out to completion”
(Dewey 1980:62).
Einav Katan-Schmid
Fig. 1, 2. Nitsan Margaliot plays/dances within Space Pirate Trainer.
Meik Ramey and Norbert Schröck help to operate the technology.
101
Playing with Virtual Realities.
A Practice-based-Research Experiment in Dancing with Technology
Fig. 3, 4. Nitsan Margaliot and Lisanne Goodhue interact within Tilt Brush as
virtual drawing and as dance scores. Meik Ramey and Norbert Schröck
help to operate the technology.
Einav Katan-Schmid
gration of the gamers in Space Pirate Trainer was too immersive on
one side of our experience, and it is thus necessary to deconstruct
interaction with the game and to play with other information out-
side of the virtual environment. The experiences, reflections, and
analysis led us to compose a new practice which interacts dancing
and playing with the VR technology on an equal basis. Our mixed
exchanges also led us to question our intentions and to recon-
sider definitions such as ‘immersion’ and ‘expressivity’ in order to
analyze and to glean explanations of why and when they appear
as experiential phenomena.
In ‘PwVR’, we explored bodies of knowledge via both their media
of configuration (VR technology, dance, dramaturgy, per formance
studies, philosophy, and theories of experience design) and their
agencies of knowing (the researchers and the practitioners involved
in the project). The questions we asked, and our intermedial explo-
rations, emphasized our assumption and approach that the knowl-
edge we interact with is not external to us. A core motivation of
the research experiment was to co-explore our knowledge systems.
We aimed to use the technology to visualize tacit invisible elements
of our knowing (like mental imageries in dance), and we used the
resulting discussions among ourselves to illuminate our perceptual
experiences with reflections and explanations. On the one hand,
our interdisciplinary ambition was to discover and encounter the
knowledge of our peers, and, on the other, the aim was to reflect on
our own processes of knowing, and to understand how our prac-
tices shape them.
During the research process, I was aware that encountering
knowledge is a regulative idea for a continuous investigation, rather
than a mission to achieve its ultimate comprehensive result. Knowl-
edge is possessed and embodied by humans and media and is thus
fragmentary and contextual. The act of knowing can be neither
absolute, nor eternal (Chatzichristodoulou/Crossley 2016: 283). The
introspective mission of encountering one’s own patterns and
habits of thoughts, and the aspiration to understand other people’s
perspectives form part of the challenges of sciences, media design,
artistic practices, and philosophies, as well as political concerns,
and our practice could not aspire to give a stable answer to those
challenges. Moreover, thus far our practice had been highly problem -
atic. However, these problems have been creatively directing the
research, since the collaborative work seeks to find mutual solutions.
The group’s joint eorts resulted in advanced insights we could not
reach alone. In the early stage of our research, we were able to
ascertain that the VR technology did not reveal the imageries of the
103 Improvisieren
Playing with Virtual Realities.
A Practice-based-Research Experiment in Dancing with Technology
dancers, although there were times when the dancers did identify
with the technology.6 The VR technology created a reversed realm
for interaction; the score became visible while the body was in visi-
ble, and the dancers had to imagine their own physical form. We
found that we needed to layer the activity of the dancers with
instructions that are expressed both in the VR and in the dancers’
imagi nation toward their bodily feelings. In practice, we created a
layered exchange between the sensory information in the VR and
the bodily feelings the dancers initiate. In the research, theoretical
knowledge induced practical problems, while another layer of
theoretical thinking oered a direction for solutions. The pattern of
the process was: theoretical hypo thesis and questioning → problems
within practice → reflecting on our instinctive feelings regarding
those problems → theo retical explanations of our instinctive feel-
ings → a new hypothesis for how to balance the practice → chang-
ing the practice → problem considered is solved (and new research
question emerges). Thus, we used the exchange between theory and
practice in order to advance both.
Returning to the distinction between the rigidity of experiments
and the indefiniteness of experimentalism, these poles fall within
the tension of knowledge in any media of knowing, which aims to
create attentive honesty by bracketing old definitions (Husserl 1997:
184). Broadly, the wish to transgress traditions and to move beyond
a mere confirmation of hypotheses and former knowing is a com-
mon challenge for experiments in both science and art. Hans-Jörg
Rheinberger advocates experimental culture, arguing that its
concept “should allow historians of science to write the history of
research domains free of the burden of disciplinary history” (Rhein-
berger 2008: 22). Ludger Schwarte (2012: 187) defines the beginning
of aesthetic experimentalism “when the parameters of a given
aesthetic praxis are broken, suspended or transcended” (translated
in: De Assis 2015: 7). Correspondingly, although dealing with Bacon
the empiricist and Cage the artist as the two poles of ‘experiment’
and ‘experimentalism’, neither Adorno nor Goehr understand the
first pole as merely scientific and the other as solely artistic. Instead,
they try “to capture the sense of what is lost in experiments when
they become too controlled and of what is lost in experimentalism
when it travels too superficially under the naturalizing banner of
freedom from human constraint” (Goehr 2015: 37—38). According to
Goehr, Adorno’s criticism on Cage’s experimentalism and on
Bacon’s scientific ambition to observe the world as it objectively is,
is based on doubts regarding whether a natural mode of experience
6 Margaliot observed that when the technology was disconnected he felt as if his own
brain did not function.
Einav Katan-Schmid
is possible in a world in which rigid cultural structures have already
been performed (Goehr 2015: 37).
Institutionally ‘PwVR’ cannot be considered as an act of resis-
tance, since it has both the context of the Interdisciplinary Labo-
ratory, in which experiments for finding new interdisciplinary
methodologies are encouraged, and also stands side by side with
current interdisciplinary and intermedial research fashions (Miller
et al. 2008, Chatzichristodoulou/Crossley 2016, Aquilina/Sarco-
Thomas 2017). Never theless, I emphasize the revolutionary stance
of ‘PwVR’ in avoiding hierarchical attitudes toward types of know-
ing. We consider the reflective knowing of dancers as equal to the
knowledge of the other scholars, and relate the experience of every-
one as relevant perspectives for discussing and analyzing problems
and possi bilities. We attempt to grasp the knowledge incorporated
by artistic practices and technology by interfering and playing with
our observations, practical habits, and theoretical assumptions.
This eort acknowledges Adorno’s criticism on the impossibility of
performing genuine experiences within a world of pre-defined
categories, without accepting his pessimistic voice as our answer.
Instead of aiming to liberate ourselves from the constraints of our
knowledge systems, or to surrender to the doom of our own firm-
ness, we played with our knowing as sources for new experiences
and for new knowledge. Our task for know ing is to remain open-
minded to the knowledge of others and to notice how it interferes,
aects, and extends our hypotheses, habits, and perceptions.
Acknowledgments: ‘Playing with Virtual Realities’ has been
realized by the ‘gamelab.berlin’ as part of ‘Image Knowledge
Gestaltung’, Cluster of Excellence of Humboldt-University
of Berlin, funded by the German Research Foundation (DFG).
The project has been realized in collaboration with ‘Institute
for Games’, Hochschule der Medien Stuttgart, and with ‘the
Department of Theatre and Performance Studies’, Free Uni-
versity of Berlin. The project is in association with the interna-
tional network of Performance Philosophy. The premiere
and the symposium of ‘P wVR’ took place at DOCK 11 Berlin,
January25—28, 2018.
Project director and choreographer: Einav Katan-Schmid
Dancers-researchers: Lisanne Goodhue, Nitsan Margaliot
Technical assistance-researchers: Meik Ramey, Norbert Schröck
Creative team-researchers: Sabiha Ghellal, Ramona Mosse,
Christian Stein, Thomas Lilge.
105 Improvisieren
Playing with Virtual Realities.
A Practice-based-Research Experiment in Dancing with Technology
Aristotle (1994–2009): Nichomachean Ethics.
Translated by W. D. Ross. Cambridge, MA:
The Internet Classics Archive. http://classics.
mit.edu/Aristotle/nicomachaen.html
(accessed August 10, 2017).
Aquilina, Stefan and Sarco-Thomas, Malaika
(2017): Interdisciplinarity in the Performing
Arts: Contemporary Perspectives. Malta: Malta
University Press.
Chatzichristodoulou, Maria and Crossley, Mark
(2016): “Falling Between’: Opportunities
and Challenges for Intermediality” [Editorial
Introduction], in: Research in Drama Educa-
tion: The Journal of Applied Theatre and Perfor-
mance 21(3), pp. 277–292.
de Assis, Paulo (2015): “Introduction“, in: Paulo de
Assis (ed.): Experimental Anities in Music.
Leuven: Leuven University Press, pp. 7–13.
deLahunta, Scott and Hennermann, Célestine
(eds) (2013): Motion Bank: Starting Points &
Aspirations. Frankfurt: Motion Bank/ The
Forsythe Company.
Dewey, John (1980): Art as Experience. New York
NY: Perigee Books.
Forsythe, William and Haner, Nik (2012):
William Forsythe: Improvisation Technologies:
A Tool for the Analytical Dance Eye. Karlsruhe
DE: ZKM digital arts edition, Zentrum für
Kunst und Medientechnologie Karlsruhe.
Ghellal, Sabiha (2017): The Interpretative Role of
an Experiencer: How to Design for Meaninul
Transmedia Experiences by Contrasting Ambig-
uous Vs. Prescribed Qualities. Aalborg: Aalborg
Universitetsforlag. https://doi.org/10.5278/vbn.
phd.tech.00006 (accessed August 10, 2017).
Goehr, Lydia (2015): “Explosive Experiments and
the Fragility of the Experimental”, in: Paulo
de Assis (ed.): Experimental Anities in Music.
Leuven: Leuven University Press, pp. 15–41.
Hoel, Aud Sissel (2012): “Technics of Thinking”,
in: Aud Sissel Hoel and Ingvild Folkvord (eds):
Ernst Cassirer on Form and Technolo.
London: Palgrave Macmillan, pp. 65–91.
Husserl, Edmund (1997): Psychological and Tran-
scendental Phenomenolo and the Confronta-
tion with Heideer (1927–1931) (ed. and trans.
by Thomas Sheehan and Richard E. Palmer).
London: Kluwer Academic publishers.
Katan-Schmid, Einav (2016): Embodied Philosophy
in Dance: Gaga and Ohad Naharin’s Movement
Research. London: Palgrave Macmillan.
Katan-Schmid, Einav (2017): “Dancing Metaphors:
Creative Think ing within Bodily Movements“,
in: Dan-Eugen Ratiu and Connell Vaughan
(eds): Proceedings of the European Society for
Aesthetics, Vol. 9. Fribourg, CHE: European
Society for Aesthetics.
Laban, Rudolf (1956): Laban’s Principles of Dance
and Movement Notation. 2. edition (ed. and
annotated by Roderyk Lange). London:
MacDonald and Evans.
Mauss, Macel (1994): “Techniques of the Body”,
in: Jonathan Crar y and Sanford Kwinter (eds):
Incorporations. New York: Zone Books.
Merleau-Ponty, Maurice (1945): Phénoménologie
De La Perception. Paris: Librairie Gallimard.
Miller, Thaddeus R.; Baird, Timothy D.;
Littlefield, Caitlin M.; Kofinas, Gary; Chapin,
F.Stuart; Redman, Charles L. (2008): “Episte-
mological pluralism: reorganizing interdisci-
plinary research“, in: Ecolo and Society 13(2)
46. URL: http://www.ecologyandsociety.org/
vol13/iss2/art46/
Nelson, Robin (2013): Practice as Research in the
Arts: Principles, Protocols, Pedagogies, Resis-
tances. London: Palgrave Macmillan.
Radder, Hans (2003): “Toward a More Developed
Philosophy of Scientific Experimentation”, in:
Hans Radder (ed): The Philosophy of Scientific
Experimentation. Pittsburgh: University of
Pittsburgh Press, pp. 1–18.
Rheinberger, Hans-Jörg (1997): Toward a Histor y
of Epistemic Things: Synthesizing Proteins in the
Test Tube. Stanford, CA: Stanford University
Press.
Rheinberger, Hans-Jörg (2008): Epistemic Objects/
Technical Objects, in: Epistemic Objects
Research Colloquium. Max Planck Institute
for the History of Science, Berlin, May 16–17,
2008, pp. 21–28. http://institut-kunst.ch/
wp-content/uploads/2016/05/epistemic-
objects.pdf (accessed August 10, 2017).
Schwarte, Ludger (2012): “Experimentelle
Ästhetik: Arbeit an den Grenzen des Sinns”,
in: Zeitschrift für Ästhetik und Allgemeine
Kunst wissenschaft 57(2), pp. 185–95.
Shultis, Christopher (1998): Silencing the Sounded
Self: John Cage and the American Experimental
Tradition. Hanover and London: University
Press of New England.
VR Technology and Applications Cited
HTC Vive, HTC Corporation.
www.htcvive.com
(accessed July 30, 2018).
Space Pirate Trainer, I-Illusions.
http://www.spacepiratetrainer.com/
(accessed July 30, 2018).
Tilt Brush, Google VR.
https://www.tiltbrush.com/
(accessed July 30, 2018).
Performance Publication
Katan-Schmid, Einav; Goodhue, Lisanne; Marga-
liot, Nitsan; Ramey, Meik; Schröck, Norbert;
Ghellal, Sabiha; Mosse, Ramona; Stein,
Christian; Lilge, Thomas (2018): Playing with
Virtual Realities. Berlin: January 26, DOCK11.
https://www.youtube.com/watch?v=2r_
LOG7TaQA (accessed July 30, 2018).
Literature
