Edgar Berdahl’s research while affiliated with Louisiana State University and other places

This paper describes a human subject study that compared the limits at which humans could communicate information through pursuit tracking gestures versus pointing (i.e. tapping) gestures. These limits were measured by estimating the channel capacity of the human motor-control system for pursuit tracking versus pointing along a single axis. A human-computer interface was built for this purpose, consisting of a touch strip sensor co-located with a visual display. Bandwidth-limited Gaussian noise signals were used to create targets for subjects to follow, enabling estimation of the channel capacity at bandwidth limits ranging from 0.12 Hz to 12 Hz. Results indicate that for lower frequencies of movement (from 0.12 Hz to 1 Hz or 1.5 Hz), pointing gestures with such a sensor may tend to convey more information, whereas at higher frequencies (from 2.3 Hz or 2.9 Hz to as high as 12 Hz), pursuit tracking gestures will afford higher channel capacities. In this work, the direct comparison between pursuit tracking and pointing was made possible through application of the Nyquist sampling theorem. This study forms a methodological basis for comparing a wide range of continuous sensors and human capacities for controlling them. In this manner, the authors are aiming to eventually create knowledge useful for theorizing about and creating new kinds of computer-based musical instruments using diverse, ergonomic arrangements of continuous sensors.

The design of a Spatially Distributed Vibrotactile Actuator Array (SDVAA) is presented. It employs a multitude of vibrotactile actuators in order to communicate a larger amount of information than is possible using a single actuator. The SDVAA is currently being used for applications in music-to-vibrotactile sensory augmentation. While prior related projects have focused more on sensory substitution, this project aims only to add to a person’s experience by augmenting a multimedia presentation with vibrotactile feedback. Because haptic perception is fundamentally different than auditory perception, it makes sense to rearrange the information transmitted to the haptic senses. For example, while auditory perception is limited to approximately the range 20 Hz to 20 kHz, tactile perception is limited primarily to the range 0 Hz to 800 Hz (if not higher). Accordingly, one approach being considered for converting auditory signals to vibrotactile signals is pitch shifting. Project results relating to music composed specifically for the SDVAA as well as general musical vibrotactile prototyping concepts will be presented.

Actuated instruments can be created in a variety of ways. An interesting class of actuated instruments are feedback-controlled acoustic musical instruments. A robust way to create these is to employ pairs of collocated sensors and actuators, and then to use the feedback control to simulate virtual physical systems. In the linear and time-invariant case, this means that the feedback control functions can be designed to be positive real. Accordingly, the physical properties of the instrument can become adjustable via the feedback control. An interesting case arises when the number of actuators and sensors differ. The actuators and sensors can however still potentially be collocated, which will result in the individual transfer functions from the actuators to the sensors (e.g. the mobilities) in being positive real. Under some special cases, stable feedback control can still be attained for a wide variety of feedback gains. The Feedback Guitar serves as an interesting case study for this. It has one actuator, which is approximately collocated with six piezoelectric sensors, one for each string. Using any non-negative linear combination of the sensor signals, an approximately positive real mobility can be obtained, which can enable stable feedback control for a wide variety of feedback gains.

Digital musical instruments yielding force feedback were designed and employed in a case study with the Laptop Orchestra of Louisiana. The advantages of force feedback are illuminated through the creation of a series of musical compositions. Based on these and a small number of other prior music compositions, the following compositional approaches are recommended: providing performers with precise, physically intuitive, and reconfigurable controls, using traditional controls alongside force-feedback controls as appropriate, and designing timbres that sound uncannily familiar but are nonetheless novel. Video-recorded performances illustrate these approaches, which are discussed by the composers.

This paper demonstrates Invisible, a critical digital artwork as performance in a conceptual framework derived from performance studies. Invisible exemplifies how digital art can reflect and influence critical thinking by focusing on three key features of performance studies: constitutive, epistemic, and critical. This intersects with Human-Computer Interaction (HCI) in a digital art context, which addresses inspirational roles of digital art.

Using a musical instrument that is augmented with haptic feedback, a performer can be enabled to haptically interact with a modal synthesis-based sound synthesizer. This subject is explored using the resonators object in Synth-A-Modeler. For synthesizing a single mode of vibration, the resonators object should be configured with a single frequency in Hertz, a single T60 exponential decay time in seconds, and a single equivalent mass in kg. Changing the mass not only changes the level of the output sound, it also changes how the mode feels when touched using haptic feedback. The resonators object can further be configured to represent a driving-point admittance corresponding to arbitrarily many modes of vibration. In this case, each mode of vibration is specified by its own frequency, decay time, and equivalent mass. Since the modal parameters can be determined using an automated procedure, it is possible to (within limits) approximately calibrate modal models using recordings of sounds that decay approximately exponentially. Various model structures incorporating the resonators object are presented in a variety of contexts. The musical application of these models is demonstrated alongside presentation of compositions that use them.

A wide variety of electronic sensors can be used for designing new digital musical instruments and other human-computer interfaces. However, presently human abilities for continuously controlling such sensors are not well quantified. The field of information theory suggests that a human together with a user interface can be modeled as a communication channel. Previously, Fitts' Law used a discrete communications channel to model information conveyed by a human pointing at discrete targets. In contrast, the present work employs a continuous communications channel to model a human continuously controlling an analog-valued sensor. The Shannon-Hartley theorem implies that the channel capacity (e.g., HCI throughput) can be estimated by asking human subjects to perform gestures that match idealized, bandlimited Gaussian “target gestures” across a range of bandwidths. Then, the signal-to-noise ratio of the recorded gestures determines the channel capacity (e.g., HCI throughput). This approach is tested on human users alternately operating simple analog sensors. Suggestions are made for creating knowledge about user interfaces that could potentially transmit an enhanced amount of information to a computer.