No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Hybrid Virtual Modeling for Multisensory Interaction Design
Abstract
The Synth-A-Modeler compiler enables portable, rapid and modular prototyping of audio-haptic models for single degree-of-freedom (d.o.f.) haptic interaction. This paper introduces hybrid modeling in Synth-A-Modeler, which mixes traditionally disparate virtual modeling paradigms from the computer music literature: digital waveguide models, mass-interaction models, and modal synthesis models. Hybrid modeling enables more salient adjustment of model parameters. Using the open-source Synth-A-Modeler toolchain, the resulting models can be compiled for a wide variety of programming languages and haptic devices, and the models are made openly available to the research and music communities.
... Vibrotactile haptics can also be experimented with in HSP simply by connecting audio signals to the FireFader˜object. • Synth-A-Modeler [9,11] is another tool for creating physics-based models. Table 9.1 summarizes the Synth-A-Modeler objects referred to in the rest of the chapter. ...
... This kind of thinking is also helpful when designing virtual instruments for haptic interaction. The authors are working on this endeavor not only by prototyping, iterating, and refining interaction designs into music compositions, but also by expanding and honing the content available in the Open-Source Haptics for Artists repository [6,7,9,11]. ...
Digital musical instruments yielding force feedback were designed and employed in a case study with the Laptop Orchestra of Louisiana. The advantages of force feedback are illuminated through the creation of a series of musical compositions. Based on these and a small number of other prior music compositions, the following compositional approaches are recommended: providing performers with precise, physically intuitive, and reconfigurable controls, using traditional controls alongside force-feedback controls as appropriate, and designing timbres that sound uncannily familiar but are nonetheless novel. Video-recorded performances illustrate these approaches, which are discussed by the composers.
... A third wave of mass-interaction tools have appeared in the last decade, driven by open-source initiatives: HSP (hap- Screenshot of the Ruratae environment, allowing dynamic creation/playing of 3D sounding mass-interaction models tic signal processing) [6] provided a first means for audio rate simulation in Max/MSP, whereas Synth-a-modeler [10] provides a Faust-based [58] engine allowing compilation for a variety of targets and platforms. It has since been extended with a modelling user interface and bridges allowing for interconnection between mass-interaction, waveguide and modal synthesis elements [8]. Recent developments have yielded new prototypes for 3D mass-interaction frameworks with audio and haptic capabilities [47,73] (Fig. 5), the migen toolkit for efficient simulation in Max/MSP [48], as well as Ruratae [1] a system offering a novel approach to sound-producing 3D mass-interaction models (Fig 6). ...
The nature of human/instrument interaction is a long-standing area of study, drawing interest from fields as diverse as philosophy, cognitive sciences, anthropology, human–computer-interaction, and artistic creation. In particular, the case of the interaction between performer and musical instrument provides an enticing framework for studying the instrumental dynamics that allow for embodiment, skill acquisition and virtuosity with (electro-)acoustical instruments, and questioning how such notions may be transferred into the realm of digital music technologies and virtual instruments. This paper offers a study of concepts and technologies allowing for instrumental dynamics with Digital Musical Instruments, through an analysis of haptic-audio creation centred on (a) theoretical and conceptual frameworks, (b) technological components—namely physical modelling techniques for the design of virtual mechanical systems and force-feedback technologies allowing mechanical coupling with them, and (c) a corpus of artistic works based on this approach. Through this retrospective, we argue that artistic works created in this field over the last 20 years—and those yet to come—may be of significant importance to the haptics community as new objects that question physicality, tangibility, and creativity from a fresh and rather singular angle. Following which, we discuss the convergence of efforts in this field, challenges still ahead, and the possible emergence of a new transdisciplinary community focused on multisensory digital art forms.
Over the past four years the authors have conducted classes and workshops in design of systems for real-time digital synthesis of sound and haptic response. In response to current trends in Interaction Design education focusing on visual feedback and touchscreen interactions, the classes were developed to provide foundations for design students to leverage the potential of non-visual modes of interaction and provide them with tools and skills to develop complex multimodal, embodied experiences. These classes have been held in various institutions, have formed the basis for short workshops at conferences as well as provided tools for collaborations with external partners. In this paper we describe for the first time the extent of the project and reflect on this particular field of Interaction Design education.
A single axis force feedback device known as a haptic paddle has been implemented as a teaching tool at several universities with different designs and emphases. Presented here is a low-cost haptic paddle design that increases the ease with which students can assemble and perform virtual environment experiments over that of previously presented designs, without decreasing the haptic performance below acceptable levels. We present a frequency domain system identification and Z-Width performance characterization of the new design alongside a comparison to the previous Rice University haptic paddle design. Lastly, we discuss the benefits of new real-time hardware, an easy-to-use Field-Programmable Gate Array (FPGA), implemented with the haptic paddle.
Open innovation has become the latest management buzzword. We would have to look long and hard to find an executive who does not agree with the basic premise of open innovation that success in coming up with creative new value to offer to the marketplace depends upon connecting effectively with other companies. Executives realize they simply cant do it alone. Virtually all of them now buy into the wisdom of Bill Joys observation that there are always more smart people outside your company than within.
This paper describes the LEMUR GuitarBot, a robotic musical instrument composed of four independent MIDI controllable single-stringed movable bridge units. Design methodology, development and fabrication process, control specification and results are discussed.
A haptic interface serves as an ideal context and platform for teaching both system dynamics and embedded control. At The University of Michigan, a traditional undergraduate mechanical engineering course in systems dynamics and a new undergraduate electrical engineering course in embedded control systems have been equipped with instructional modules based on two new single-axis haptic interface devices. The iTouch motor is a low-budget, single axis, voice-coil based haptic device intended for teaching system dynamics fundamentals. Students gain hands-on experience by assembling these motors from scratch, performing experiments, and comparing actual to theoretically predicted dynamic response. A second device called "The Box" features higher torque output and robustness for the embedded control systems course. Both device designs are presented and contrasted and results following from their introduction into the curriculum are discussed. The uses of these devices to rapidly prototype various research projects and integrate undergraduate students into a research program are also briefly discussed.
We describe a system for constructing computer models of several aspects of physical interaction behavior, by scanning the response of real objects. The behaviors we can successfully scan and model include deformation response, contact textures for interaction with force-feedback, and contact sounds. The system we describe uses a highly automated robotic facility that can scan behavior models of whole objects. We provide a comprehensive view of the modeling process, including selection of model structure, measurement, estimation, and rendering at interactive rates. The results are demonstrated with two examples: a soft stu#ed toy which has significant deformation behavior, and a hard clay pot which has significant contact textures and sounds. The results described here make it possible to quickly construct physical interaction models of objects for applications in games, animation, and e-commerce.
Open Innovation describes an emergent model of innovation in which firms draw on research and development that may lie outside their own boundaries. In some cases, such as open source software, this research and development can take place in a non-proprietary manner. Henry Chesbrough and his collaborators investigate this phenomenon, linking the practice of innovation to the established body of innovation research, showing what's new and what's familiar in the process. Offering theoretical explanations for the use (and limits) of open innovation, the book examines the applicability of the concept, implications for the boundaries of firms, the potential of open innovation to prove successful, and implications for intellectual property policies and practices. The book will be key reading for academics, researchers, and graduate students of innovation and technology management.
form only given. Courses involving a laboratory component are conspicuously missing from the recent upsurge in online courses, limiting student access to hands-on educational experiences. We present a low-cost, open-hardware haptic device, called Hapkit, designed to provide a hands-on laboratory experience in an introductory online haptics course. Hapkit is a one-degree-of-freedom kinesthetic haptic device that allows users to input motions and feel programmed forces. Based on previous haptic paddle designs, Hapkit consists of a motor, magneto-resistive sensor and rotating magnet, laser-cut acrylic structure, friction drive, force-sensitive resistor, and custom Arduino-based controller board. The CAD files, parts list and costs, assembly instructions, and sample software are posted at http://hapkit.stanford.edu so the public can both recreate Hapkit and modify its design. The corresponding online haptics course, piloted in Autumn 2013, will be available as a self-paced course at http://hapticsonline.stanford.edu. In this demonstration, conference attendees will be able to assemble and program Hapkits.
It might be easy to imagine that physical models only rep-resent a small portion of the universe of appropriate force feedback controllers for haptic new media; however, we argue the contrary in this work, in which we apply creative physical model design to re-examine the science of feed-back stability. For example, in an idealized analog haptic feedback con-trol system, if the feedback corresponds to a passive phys-ical model, then the haptic control system is guaranteed to be stable, as we show. Furthermore, we argue that it is in fact necessary that the feedback corresponds to a pas-sive physical model. Otherwise, there exists a passive user-haptic device transfer function that can drive the feedback control system unstable. To simplify the mathematics, we make several assumptions, which we discuss throughout the paper and reexamine in an appendix. The work implies that besides all of the known advan-tages of physical models, we can argue that we should em-ploy only them for designing haptic force feedback. For example, even though granular synthesis has traditionally been implemented using signal modeling methods, we ar-gue that physical modeling should still be employed when controlling granular synthesis with a haptic force-feedback device.
The FireFader is a simple haptic force-feedback device that is optimized for introducing musicians to haptics. It is based upon a single-degree-of-freedom potentiometer fader coupled to a DC motor, also known as a “motorized fader.” A light is connected in parallel with the motor to help communicate the force's strength visually. The FireFader consists of only open-source hardware and open-source software elements. Consequently, it is relatively easy for users to repurpose it into new projects involving varying kinds and numbers of motors and sensors.
An open-source device driver for the FireFader allows it to be linked to a computer via USB so that the computer can perform the feedback control calculations. For example, the computer can simulate the acoustics of a virtual musical instrument to concurrently synthesize sound and calculate the motor force as a function of the fader position. The serial connection over USB increases the latency of the control signal compared to embedded implementations, but the serial connection facilitates easier programming via the computer, and the force feedback can be automatically disabled when the user is not touching the fader. Some new devices derived from the FireFader design are presented.
Dynamic interaction with the environment is fundamental to the process of manipulation. This paper describes an approach to the design of ‘interaction controllers’ and contrasts this with an approach to servo design. The need for ‘coupled stability’ and ‘interactive behaviour’ specifications is motivated. A necessary and sufficient condition to ensure the stability of a linear manipulator coupled at a single interaction port to a linear, passive environment is found. It is shown that this condition may be extended to a broad class of active environments, and a simple test for coupled stability, based on the root locus, is presented. Two interaction controller designs for a linear manipulator are presented. Interactive behaviour is analysed in the frequency domain by comparison with the behaviour of a target model. A simple controller shows strong stability properties, but poor behaviour; a more sophisticated design shows that it is possible to achieve desirable behaviour while retaining stability guarantees.
In The Music Machine, Curtis Roads brings together 53 classic articles published in Computer Music Journal between 1980 and 1985, providing a cohesive survey of the major developments in computer music and in the related technology during the last decade. The book includes interviews with major figures in the field and articles devoted to composition, artificial intelligence, and the popular Music Instrument Digital Interface (MIDI). Roads has written an overview of each of the book's seven parts, highlighting the major topics and placing the various articles in a thematic and historical context.
The ACROE and the ICA laboratory are two associated institutions running a program of research, development, creation and pedagogy in the field of computer for music and animated image synthesis. ACROE was founded in 1976 by Claude Cadoz, Annie Luciani and Jean-Loup Florens, in the Institut National Polytechnique of Grenoble (INP-G) with the support of French Ministry of Culture. The ICA (Artistic Creation and Computer) was initially a team, belonging successively to the LCP (Laboratoire de la Communication Parlée - INP-G), from 1976 to 1985, to the LIFIA (Informatique Fondamentale et Intelligence Artificielle - IMAG), from 1985 to 1995, and to the CLIPS (Communication Langagière et Interaction Personne-Système - IMAG). Since 1999, ICA is itself a laboratory of INP-G. ICA is in charge of the scientific research part of the global program.
We will present in the first part of this paper a musical work, pico..TERA, recently created by the author, which is a 290-second piece entirely generated by one (non real-time) run of a complex physical model "orchestra". With this work, we will show how a musical piece can be created, from its micro- to its complete macro-structure by the means of a physically-based formalism. Then we will identify the concepts within the piece that introduce this new dimension of "musical construction". From a theoretical analysis of the specific examples and applications in the piece, we will induce and introduce a general conceptual framework for musical composition through physical modeling and will explain in the last section the fundamental "paradigm shift" to which it corresponds.
Active force-feedback holds the potential for precise and rapid controls. A high performance device can be built from a surplus disk drive and controlled from an inexpensive microcontroller. Our new design, The Plank has only one axis of force-feedback with limited range of motion. It is being used to explore methods of feeling and directly manipulating sound waves and spectra suitable for live performance of computer music.
As an innovative approach to teaching the laboratory component of an undergraduate course on dynamic systems, we present the haptic paddle: a low-cost, single-axis, force-feedback joystick. Using the paddle, students not only learned to model and analyze dynamic systems, but by using their sense of touch, they were able to feel the effects of phenomena such as viscous damping, stiffness, and inertia. Feeling the dynamics, in addition to learning the underlying physics, improved students' understanding and added an element of fun to the course. In this paper, we describe the purpose and design of the haptic paddle, present examples of how the paddle was integrated into laboratory exercises, and show the results of student evaluations. 1.
The missing link: modal synthesis, Representations of musical signals
- Jean-Marie Adrien
New Computational Paradigms for Computer Music, chapter FAUST: An Efficient Functional Approach to DSP Programming
- Y Orlarey
- D Fober
- S Letz
An introduction to the Synth-A-Modeler compiler: Modular and open-source sound synthesis using physical models
- E Berdahl
- J Smith
The Missing Link: Modal Synthesis, chapter Representations of Musical Signals
- J.-M Adrien
Berdahl and J. Smith III. An introduction to the Synth-A-Modeler compiler: Modular and open-source sound synthesis using physical models
- E Berdahl
- J Smith
- Berdahl E.
Creation nets: Harnessing the potential of open innovation
- J Brown
- Brown J.