Conference Paper

Exploring the Virtuality Continuum Frontiers: Multisensory and Magical Experiences in Interactive Art

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

This article is an effort to approach certain aspects of the evolution of user interface design, as well as the design and aesthetics of interactive works of art. Interactive works attempt to connect the viewer with the work and invite him to deploy his senses and his body. But many researchers and artists are concerned by the fact that interface technologies tend to put us out of touch with ourselves, so they try to create interactions that take place in our periphery, drawing our attention rather than demanding it. Taking as a starting point the sense of touch, whose importance has been widely analysed in philosophy, phenomenology and aesthetics, we describe the modalities of active touch and active sensing. We then propose that with the use of physical materials and tangible interfaces in interactive artworks we can achieve more magical experiences that engage us in a multi sensorial way. As an example we are describing the concepts around the mixed reality installation VitRails.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Such are the experiences people observe when entering Josef Rehrl Schule, a school for Deaf, hard-of-hearing, and hearing children and personnel. Our three installations, integrated in the school's entrance hall, celebrate all senses and especially full-body spatial interaction; the natural way humans traverse, explore, and experience the built environment [20]. ...
... As a way of comprehending and managing people's reactions to novel technology, De Jongh Hepworth introduces four magicalexperience elements (surprise, unordinary, unnatural, and exciting) [11]. On a similar note, Georgakopoulou et al. blend magic (unexpected, non-ordinary behaviour of physical objects) with their mixed reality (MR) tangible interface [20]. ...
... The use of this metaphor encapsulates a set of particular attributes that could foster this desired experience. Specifically, we reflect on the characteristics of our designs based on De Jongh Hepworth's four ingredients for magical experiences [11], and both Georgakopoulou et al. [20] and Subbotsky's definition of magic (anything breaching the laws of physical reality) [44]. De Jongh Hepworth's Manuscript submitted to ACM magical experiences' ingredients are unordinary (different from prior experiences), surprise (something unexpected), unnatural (to do something one cannot), exciting (something one anticipates) [11]. ...
Conference Paper
An ocular-centric fixation governs the majority of examples in both traditional and media architecture. Stepping away from this predominance, the three technology-augmented installations described in this contribution focus primarily on multi-sensory spatial interactions. To trigger curiosity and establish engagement, we make use of magic as a metaphor to encourage playful exploration of the installations' possibilities and speculation on their modus-operandi enigmas. This combination elevates the exhibition space as it appoints the human body as the key constituent of the experience. Placed at the entrance hall of an integration school (for Deaf, hard-of-hearing and hearing children and staff), our installations are aimed at raising population awareness by highlighting this population's particularities. In this paper, we detail the constellation of magical design characteristics of our artifacts and propose a set of considerations that could bring forth magical media architecture experiences.
... In their work, they introduced the concept of the reality-virtuality continuum. They proposed the concept of a virtuality continuum in the context of visual displays, but their ideas have since been adopted and extended to fit all domains of research around virtual and mixed reality, whether scientific or artistic (Georgakopoulou et al., 2019). ...
Article
Full-text available
Shaping both the environment and the embodiment of the users in that virtual world, VR offers designers and cognitive scientists the unprecedented potential to virtually explore a vast set of interactions between persons, and persons and their environment. By design, VR tools offer a formidable opportunity to revisit the links between body movement and lived experiences, and to experiment with them in a controlled, yet engaging and ecologically valid manner. In our multidisciplinary research-creation project we ask, how can we design (virtual) environments that specifically encourage interactions between multiple persons and that allow designers, scientists, and participants (users or “immersants”) to explore the very process of interaction itself? Building on our combined experience with dance improvisation research and interactive virtual spatial design, we document a multi-user VR experience design approach we name Shared Diminished Reality (SDR), where immersants are co-present and able to move together while their bodies and the environment are represented in a minimalist way. Our working hypothesis is that non-anthropomorphic embodiment of oneself and one’s partner(s), combined with open-ended exploration, focuses the user’s attention on the quality of the interaction and encourages playfulness and creativity. We present the articulations VR platform and its design history, as well as design evaluations of SDR in a laboratory setting and through a mixed reality performance, interrogating the impact of our minimalist approach on user experience and on the quality of the interaction. Our results suggest that minimizing (self and other) representation in Shared Diminished Reality positively impacts relational dynamics, induces playful creativity, and fosters the will to move and improvise together.
... What is apparent from our empirical work, and supported by others as above (Georgakopoulou et al., 2019;Kumari et al., 2018;Rasmussen, 2013), is that as interaction design extends multi-dimensionally into the physical environment including through augmentation and tangible user interfaces, there are myriad new opportunities to create surprise, delight, excitement International Journal of Design Vol. 14 No. 3 2020 T-Shifting Identities and Practices: Interaction Designers in the Fourth Industrial Age and apparent magic. Consequently, we say, there is a greater need, and responsibility to make available, simple explanations of how things work, while also avoiding the word 'magic' due to its inherent fuzziness. ...
Article
Full-text available
We report findings from our two-year research study to investigate the practices, processes and roles of professional creatives working on interaction design and wider digital design projects. The study contributes insights from interviews conducted to support the development of 13 high profile industry case studies involving 21 of their creators. Through thematic analysis of interview transcripts we constructed key themes of project scope, design stances, skills sets and studio practice. We discuss these as representative of the perpetual shifting of the cornerstones of how designers have traditionally understood and embodied their own and peers’ roles and combinations of competencies. This, we argue, is challenging perceptions and expectations around designers’ traditional “T-shape” organisation of skills and knowledge. The article goes on to identify areas of emerging design practice brought about by rapid technological changes associated with the fourth industrial age that warrant further research. These include anticipatory design and personalisation, branded interactions and magic technology. The article concludes by calling for wide sharing of designers’ stories as a pragmatic resource to demonstrate and communicate emerging practices that support the development of graduates and other designers entering this rapidly-changing field.
Chapter
An end-use’s experience of any software is typically influenced by the interface presented by the application to the user. For Mixed Reality Environments such as Augmented and Virtual Reality, the user interface is highly visual, and a poor interface can significantly degrade the user experience. Adequate attention is required when designing or creating interfaces and user experience within Mixed Reality Environments as traditional interface design goals and specifications often need to be adjusted. Furthermore, for mixed reality environments on Mobile devices there are additional interface constraints and considerations that would considerably improve the user experience when properly addressed. This research paper discusses the evolution(s) of user interface(s) and user experience of Augmented and Virtual Reality applications on Mobile devices and contributes a framework for improving user interfaces and experience when using Mixed Reality Environments.
Conference Paper
Full-text available
For millennia, magicians have designed illusions that are perceived as real regardless of their impossibility, inducing a sense of wonder in their audience. This paper argues that video game designers face the same design challenge - crafting believable and engaging illusions - and that the practice of magic provides an untapped wealth of design principles and techniques for game designers. To support this claim, the paper introduces two key principles of magic, affording perceived causal relations and forcing perceived-free choice. It then presents techniques to create and exploit these effects and discusses their parallels and applications in game design, encouraging game designers and researchers to further explore the field of magic for testable theories and applicable techniques.
Conference Paper
Full-text available
We present Tangible Drops, a visio-tactile display that for the first time provides physical visualization and tactile feedback using a planar liquid interface. It presents digital information interactively by tracing dynamic patterns on horizontal flat surfaces using liquid metal drops on a programmable electrode array. It provides tactile feedback with directional information in the 2D vector plane using linear locomotion and/or vibration of the liquid metal drops. We demonstrate move, oscillate, merge, split and dispense-from-reservoir functions of the liquid metal drops by consuming low power (450 mW per electrode) and low voltage (8--15 V). We report on results of our empirical study with 12 participants on tactile feedback using 8 mm diameter drops, which indicate that Tangible Drops can convey tactile sensations such as changing speed, varying direction and controlled oscillation with no visual feedback. We present the design space and demonstrate the applications of Tangible Drops, and conclude by suggesting potential future applications for the technique.
Article
Full-text available
Aristotle's treatment of tactility is at odds with the hierarchical order of psyche's faculties. Touching is the commonest and lowest power; it is possessed by all sentient beings; thinking is, on the contrary, the highest faculty that distinguishes human beings. Yet, while Aristotle maintains against some of his predecessors that to think is not to sense, he nevertheless posits a causal link between practical intelligence and tactility and even describes noetic activity as a certain kind of touch. This essay elucidates Aristotle's analysis of the sense of touch in De anima and argues that tactility provides a paradigm for sensitivity in general and in particular for the reflexivity of sensation whereby the senses disclose not only what they are sensing but also that they are sensing. This feature, it is argued, has epistemological and ontological consequences. The sense of touch testifies to the physical presence of material beings and provides an empirical verification of substance's essential feature, namely, self-reference.
Conference Paper
Full-text available
In this paper we give an overview of the work done on the methodology of using smart material interfaces as it appears in the literature until now. We address the opportunities offered by smart materials as they have been exploited by other researchers who created smart material interfaces. We do so by surveying smart materials by kind and by looking at how they have been exploited.
Book
Full-text available
In children, magical thinking has traditionally been viewed as an immature form of thinking that is destined to diminish with age. With some exceptions, the study of magical thinking and magical beliefs in adults has mostly remained on the fringes of psychology, along with the study of such topics as superstitions, anomalistic beliefs, and parapsychology. In this book, I argue that the role of magical thinking in child development and in adult life should be reconsidered. In children, magical thinking is an important part of cognitive development. In adults, magical thinking and magical beliefs assist individuals as they struggle with situations that are beyond rational control. There is evidence that suggestive techniques used in politics, commercial advertising, and psychotherapies target magical thinking and magical beliefs. In this book, the mechanisms and development of magical thinking and beliefs throughout the lifespan are discussed.
Article
Full-text available
In this paper we discuss Augmented Reality (AR) displays in a general sense, within the context of a Reality-Virtuality (RV) continuum, encompassing a large class of "Mixed Reality" (MR) displays, which also includes Augmented Virtuality (AV). MR displays are defined by means of seven examples of existing display concepts in which real objects and virtual objects are juxtaposed. Essential factors which distinguish different Mixed Reality display systems from each other are presented, first by means of a table in which the nature of the underlying scene, how it is viewed, and the observer's reference to it are compared, and then by means of a three dimensional taxonomic framework, comprising: Extent of World Knowledge (EWK), Reproduction Fidelity (RF) and Extent of Presence Metaphor (EPM). A principal objective of the taxonomy is to clarify terminology issues and to provide a framework for classifying research across different disciplines.
Conference Paper
Full-text available
Advances in the creation of computational materials are transforming our thinking about relations between the physical and digital. In this paper we characterize this transformation as a "material turn" within the field of interaction design. Central to theorizing tangibility, we advocate supporting this turn by developing a vocabulary capable of articulating strategies for computational material design. By exploring the term texture, a material property signifying relations between surfaces, structures, and forms, we demonstrate how concepts spanning the physical and digital benefit interaction design. We ground texture in case study of the Icehotel, a spectacular frozen edifice. The site demonstrates how a mundane material can be re-imagined as precious and novel. By focusing on the texture of ice, designers craft its extension into the realm of computational materiality. Tracing this process of aligning the physical and digital via the material and social construction of textures speaks back to the broader field of interaction design. It demonstrates how the process of crafting alliances between new and old materials requires both taking seriously the materialities of both, and then organizing their relation in terms of commonalities rather than differences. The result is a way of speaking about computational materials through a more textured lens.
Conference Paper
Full-text available
In this paper, we seek to identify interesting sources of metaphor for tangible user interfaces (TUIs). We begin by doing a systematic exploration of the design space that results from constructing simple TUI devices. From this we argue that a new set of metaphors are needed for this domain. From usability tests of simple tangible devices, we suggest that magic and paranormal phenomena could be a fruitful place to look for new metaphors for TUIs.
Article
Full-text available
Active sensing systems are purposive and information-seeking sensory systems. Active sensing usually entails sensor movement, but more fundamentally, it involves control of the sensor apparatus, in whatever manner best suits the task, so as to maximize information gain. In animals, active sensing is perhaps most evident in the modality of touch. In this theme issue, we look at active touch across a broad range of species from insects, terrestrial and marine mammals, through to humans. In addition to analysing natural touch, we also consider how engineering is beginning to exploit physical analogues of these biological systems so as to endow robots with rich tactile sensing capabilities. The different contributions show not only the varieties of active touch--antennae, whiskers and fingertips--but also their commonalities. They explore how active touch sensing has evolved in different animal lineages, how it serves to provide rapid and reliable cues for controlling ongoing behaviour, and even how it can disintegrate when our brains begin to fail. They demonstrate that research on active touch offers a means both to understand this essential and primary sensory modality, and to investigate how animals, including man, combine movement with sensing so as to make sense of, and act effectively in, the world.
Thesis
Dans des situations où la perception visuelle est fortement contrainte ou déficiente, il est nécessaire de rendre perceptible l'information dans une modalité non visuelle, tout en prenant en compte des capacités sensorielles et mnésiques humaines. Par exemple, un non-voyant, souhaitant prendre connaissance d'un itinéraire, devra le parcourir de façon non visuelle et le mémoriser. Cependant, outre l'aspect matériel, la mise en œuvre de solutions alternatives (non visuelles) demeure confrontée aux capacités cognitives de l'utilisateur (compréhension, mémorisation, intégration de plusieurs informations, etc.). L'objet de cette thèse est de contribuer à la conception de techniques d'interactions permettant d'optimiser la transmission non visuelle d'informations. A ces fins, j'ai exploré l'apport de la multimodalité comme moyen d'optimisation permettant d'outrepasser les limites de la mémorisation. Je me suis concentré sur l'étude des techniques d'interaction basées sur les modalités auditives et tactiles, en limitant au maximum l'utilisation de la parole, afin de concevoir des techniques pour des environnements différents (flexibilité), d'optimiser l'utilisation de canaux perceptifs (exploitation des propriétés du son dans des messages audio pour transmettre plus d'informations, par exemple), d'éviter de limiter mes techniques par la barrière de la langue ou de sa compréhension et enfin, pour explorer d'autres solutions que la synthèse vocale seule. Les travaux de ma thèse ont mené à la conception, à l'implémentation et à l'évaluation de techniques d'interaction multimodale non visuelle, en réponse à différents contextes, dont, en particulier, ceux de la transmission d'informations de type , (couple de coordonnées) et (séquence de couples direction-distance). Pour parvenir à concevoir mes interactions, j'ai, tout d'abord, effectué une revue de la littérature, afin d'en extraire les principaux facteurs de conception de techniques d'interaction dédiées à la transmission non visuelle d'information. Puis, j'ai organisé ces facteurs sous la forme d'un cadre d'analyse, sur lequel je me suis appuyé pour concevoir chacune de mes techniques. Trois expériences distinctes ont permis d'évaluer l'influence de facteurs de conception sur l'efficacité des interactions et la satisfaction des utilisateurs vis-à-vis des techniques. Je peux, notamment, citer l'implication des utilisateurs (actif ou passif), la présence d'aides explicites", la transmission de plusieurs informations en parallèle et la modalité principale utilisée et, le type de codage dans lequel est encodée l'information.
Conference Paper
Actuated Interfaces are receiving a great deal of interest from the research community. The field can now present a range of point designs, illustrating the potential design space of Actuated Interfaces. However, despite the increasing interest in Actuated Interfaces, the research carried out is nevertheless primarily preoccupied with the technical challenges and potential application areas, rather than how users actually approach, experience, interpret and understand Actuated Interfaces. Based on three case studies, investigating how people experience Actuated Interfaces, we point to; magic, movement and ambiguity as fruitful perspectives for understanding users' experiences with Actuated Interfaces. The three perspectives are employed to reflect upon opportunities and challenges, as well as point to open questions and relevant areas for future research for Actuated Interfaces.
Conference Paper
The field of interaction design is littered with examples of artefacts, which seemingly do not adhere to well-known physical causalities and our innate expectations of how artefacts should behave in the world, thereby creating the impression of a magic reality; where things can float in mid-air, the usually inanimate TV can become animate, two separate objects can become physically connected, and we can move objects with our mind. The paper presents Subbotsky's [21] four types of magical causalities: mind-over-matter magic, animation magic, nonpermanence magic and sympathetic magic, as a way to reflect upon the magical realities constructed by technological artefacts.
Conference Paper
This paper is a description of design experiments performed to investigate the use of magical experiences in interaction design of consumer electronics. Magical experiences are usually associated with a passive audience watching a magician, but it is interesting to see if interaction can also be a magical experience. Using inspiration from stage magic, industrial design, the consumer electronics company Bang & Olufsen, and related research in interaction design a set of key topics for developing magical experiences are proposed. The key topics are used to develop three design experiments. The design experiments are tested and the results are analyzed and discussed.
De l'âme, GF Flammarion
  • Phdthesis
PhdThesis, Université de Toulouse, Toulouse. Aristote, 1993. De l'âme, GF Flammarion. Paris.
  • J.-P Balpe
Balpe, J.-P. (Ed.), 2000. L'art et le numérique, Les cahiers du numérique, volume 1, Hermès, Paris.
Materializing Interaction
  • M Coehlo
Coehlo, M., 2013. Materializing Interaction, PhD Thesis, MIT.
La Dioptrique, GF Flammarion
  • Descartes
Descartes, 1966. La Dioptrique, GF Flammarion. Paris.
A taxonomy for and analysis of tangible interfaces. Personal and Ubiquitous Computing 8
  • K Fishkin
Fishkin, K., 2004. A taxonomy for and analysis of tangible interfaces. Personal and Ubiquitous Computing 8. Springer-Verlag. pages 347-358.
Microphysics of Hardware or the Gigantism of the Infinitesimally Small
  • D Ginosatis
  • D Kavvathas
  • A Golemi
Ginosatis, D., Kavvathas, D., Golemi, A., 2012. Microphysics of Hardware or the Gigantism of the Infinitesimally Small. In Marshall McLuhan and Walter J. Ong International Centenary Symposium. SSRN Electronic Journal.
  • J Guez
Guez, J., 2015. Illusions entre le réel et le virtuel (IRV) comme nouvelles formes artistiques : présence et émerveillement, Thèse, Esthétique, Science et Technologie des Arts, Université Paris 8, Paris.
Reality-based interaction: a framework for post-wimp interfaces
  • R J Jacob
  • A Girouard
  • L M Hirsheld
  • M S Horn
  • O Shaer
  • E T Solovey
  • J Zigelbaum
Jacob, R. J., Girouard, A., Hirsheld, L. M., Horn, M. S., Shaer, O., Solovey, E. T. and Zigelbaum, J., 2008. Reality-based interaction: a framework for post-wimp interfaces. In Proc. conf. SIGCHI on Human factors in computing systems, CHI '08, pages 201-210.
Fragile and magical: materiality of computational technology as design material. Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility
  • H Landin
Landin, H., 2005. Fragile and magical: materiality of computational technology as design material. Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility. ACM. pages 117-120.
Le corps. GF Flammarion
  • E Levine
  • P Touboul
Levine, E., Touboul, P., 2015. Le corps. GF Flammarion. Paris.
L'inscription corporelle de l'esprit, éditions du Seuil
  • F Varela
  • E Thompson
  • E Rosch
Varela, F., Thompson, E., Rosch, E., 1993. L'inscription corporelle de l'esprit, éditions du Seuil. Paris.
AloïsRiegl: Art, Value, and Historicism
  • H Zerner
Zerner, H., 1976. AloïsRiegl: Art, Value, and Historicism. Daedalus, 105(1), 177-188. Retrieved from http://www.jstor.org/stable/20024392
De l'âme, GF Flammarion
  • Aristote
Aristote, 1993. De l'âme, GF Flammarion. Paris.