David Gerhard

University of Manitoba | UMN

Despite the benefits of learning an instrument, many students drop out early because it can be frustrating for the student, expensive for the caregiver, and loud for the household. Virtual Reality (VR) and Extended Reality (XR) offer the potential to address these challenges by simulating multiple instruments in an engaging and motivating environme...

The development of effective and usable software for spatial computing platforms like virtual reality (VR) requires an understanding of how these devices create new possibilities (and new perils) when it comes to interactions between humans and computers. Virtual Reality Usability Design provides readers with an understanding of the techniques and...

Current definitions of immersion describe its relationship to presence and allow for relative comparisons between the immersive qualities of Virtual Reality (VR) systems, but lack the ability to describe the immersion supported by a system as an absolute quantity. In this paper, we present an abstract model of perception, defining sensory units as...

The Royal Canadian Mounted Police (RCMP), like all public safety personnel (PSP), are frequently exposed to potentially psychologically traumatic events that contribute to posttraumatic stress injuries (PTSI). Addressing PTSI is impeded by the limited available research. In this protocol paper, we describe the RCMP Study, part of the concerted effo...

Les membres de la Gendarmerie royale du Canada (GRC), comme l’ensemble du personnel de la sécurité publique, sont fréquemment exposés à des événements potentiellement traumatiques sur le plan psychologique qui contribuent à la survenue de blessures de stress post­traumatique. Le peu de travaux de recherche disponibles sur le sujet limite l’étude de...

Background Although movement variability and long-range correlations (LRCs) have been assessed in relation to neuropathology and aging during walking, to date only a few studies have investigated these aspects in subjects of different skill levels during prolonged overground running. Research question What effect does skill level and run duration...

This paper presents a novel controller-centric text input system that allows for chording key selection using existing consumer VR hardware. The design for a layout is discussed, consisting of a one-handed alphabetic keyboard. We argue that the use of existing handheld controllers and the inclusion of a functional one-handed layout would allow for...

The question of how to arrange harmonically related pitches in space is a historical research topic of computational musicology. The primitive of note arrangement is linear in 1-D, in which ordered ascending pitches in one direction correspond to increasing frequencies. Euler represented harmonic relationships between notes with a mathematical latt...

Many VR “simulated walking locomotion systems” have been developed to allow for movement in virtual environments in a manner similar to how people move in the real world. Similarly, several solutions that could be classified as “inverse treadmills” (in which the motion of running is translated to devices attached to the foot) exist. However, in the...

Many VR “simulated walking locomotion systems” have been developed to allow for movement in virtual environments in a manner similar to how people move in the real world. Similarly, several solutions that could be classified as “inverse treadmills” (in which the motion of running is translated to devices attached to the foot) exist. However, in the...

It is noted that emergency vehicles no-longer apply caution when approaching an intersection. The assumption is that, the automated system should already be aware of the emergency vehicles arrival at the intersection. Considering the assumption, emergency vehicles tend to proceed through traffic intersections without visual confirmation of road pri...

The question of how to arrange harmonically related pitches in space is a historical research topic of computational musicology. The primitive of note arrangement is linear in 1-D, in which ordered ascending pitches in one direction correspond to increasing frequencies. Euler represented harmonic relationships between notes with a mathematical latt...

In this paper we present a collection of virtual interfaces used to augment a MIDI keyboard synchronized in physical and virtual space. Several virtual interfaces are developed and evaluated. Some utilize the tactility offered by the keyboard's surface, while others rely on the improved presence offered by the keyboard. An evaluation of these virtu...

From a research perspective, detailed knowledge about stride length (SL) is important for coaches, clinicians and researchers because together with stride rate it determines the speed of locomotion. Moreover, individual SL vectors represent the integrated output of different biomechanical determinants and as such provide valuable insight into the c...

As virtual reality gains popularity, technology which better integrates the user’s physical experience in the virtual environment is needed. Researchers have shown that by including real physical objects to interact with, the experience can be made significantly more convincing and user-friendly. To explore physically connecting the user to the vir...

The concept of musical isomorphism is known as the arrangement of musical note constructs in an identical shape regardless of the beginning root note. The research of musical isomorphism exists for over three-hundred years. In the 1700s, Euler tried to present harmonic relationships between notes with a mathematical lattice called Tonnetz, and sinc...

As virtual reality gains popularity, technology which better integrates the user's physical experience in the virtual environment is needed. Researchers have shown that by including real physical objects to interact with, the experience can be made significantly more convincing and user-friendly. To explore physically connecting the user to the vir...

The Mixed Reality MIDI Keyboard is a prototype designed to augment virtual reality experiences through the inclusion of a physical interface which aligns the user's senses with the virtual environment. It also serves as a platform on which the uses of virtual reality in music interaction and art installations can be experimented with. The main prob...

In this paper, we propose a novel feature set for instrument classification which is based on the information rate of the signal in the time domain. The feature is extracted by calculating the Shannon entropy over a sliding short-time energy frame and binning statistical features into a unique feature vector. Experimental results are presented, inc...

The research of musical isomorphism has been around for hundreds of years. Based on the concept of musical iso-morphism, designers have created many isomorphic keyboard-based instruments. However, there are two major concerns: first, most instruments afford only a single pattern per interface. Second, because note actuators on isomorphic instrument...

The concept of a tone helix has been studied in tone theory and harmonic analysis from a variety of different perspectives. A tone helix represents harmonic relationships between tones in an attempt to model the perception of pitch and harmony in a single form. This paper presents a framework whereby previous helical tone representations can be con...

Poster of Conference Paper

This paper presents a novel stance phase detection procedure based on observations from a foot-mounted inertial measurement unit (IMU). A frequency-tracking algorithm from the field of audio analysis was applied to the inertial signal to obtain information about gait cycle duration. Afterwards, this information was used to determine the stance phas...

It is possible to position equal-tempered discrete notes on a flat hexagonal grid in such a way as to allow musical constructs (chords, intervals, melodies, etc.) to take on the same shape regardless of the tonic. This is known as a musical isomorphism, and it has been shown to have advantages in composition, performance, and learning. Considering...

Many contemporary computer music systems can emulate aspects of composers’ behaviour, creating and arranging structural elements traditionally manipulated by composers. This raises the question as to how new computer music systems can act as effective tools that enable composers to express their personal musical vision–if a computer is acting as a...

This paper presents a novel approach for extracting swimming performance parameters from accelerometer data using techniques traditionally applied to audio analysis. The recorded acceleration data is treated as sampled audio data, with the stroke rate (one of the main parameters to extract) treated as the fundamental frequency. A pitch detection al...

INTRODUCTION: In recent years, several studies have tried to estimate stride length (SL) using body-mounted IMUs and reported promising results. However, many studies have focused on estimating SL for walking and, to the best of our knowledge, no study has explicitly investigated the concurrent validity of a single foot-mounted IMU to estimate SL f...

In this paper, we propose a novel feature set for instrument classification which is based on the information rate of the signal in the time domain. The feature is extracted by calculating the Shannon en-tropy over a sliding short-time energy frame and binning statistical features into a unique feature vector. Experimental results are presented, in...

A new Short-Time Fourier Transform (STFT) pipeline is presented which uses a two-pass method to adapt the size of the analysis window to the period of the signal, which is assumed in this case to be at least pseudo-periodic. The pipeline begins with pitch estimation, followed by upsampling, to construct a new analysis window that matches the length...

One challenge relating to the creation of adaptive music involves generating transitions between musical ideas. This paper proposes a solution to this problem based on a modification of the Q-Learning framework described by Reese, Yampolskiy and Elmaghraby. The proposed solution represents chords as states in a domain and generates a transition bet...

In 2007, most of the accelerometer chips sold in the US were installed in cars. Today, accelerometers are in every phone, every computer, every video game controller and in fitness watches and smart bands. They have also been used in countless movement-based, and installation artworks. This paper describes how the accelerometer moved from automotiv...

An isomorphic layout can be used to position pitches on a grid of hexagons. This has many beneficial musical prop-erties such as consistent fingering and spatial harmonic consistency. A Unified Isomorphic Layout (UIL) format is presented in order to create a common specification for describing hexagonal isomorphic layouts. The UIL format provides a...

Musix (an iOS application) and Rainboard (a physical device) are two new musical instruments built to overcome limitations of existing isomorphic instruments. Musix was developed to allow experimentation with a wide variety of different isomorphic layouts to assess the advantages and disadvantages of each. The Rainboard consists of a hexagonal arra...

This presentation will outline the results of a recent research project at the University of Regina funded by the President’s Teaching and Learning Scholars Grant Program. This project explored the Apple iPad as a teaching and learning device in the area of music composition and performance. During a special course held in the Fall of 2012, the res...

Many traditional and new musical instruments make use of an isomorphic note layout across a uniform planar tessellation. Recently, a number of hexagonal isomorphic keyboards have become available commercially. Each such keyboard or interface uses a single specific layout for notes, with specific justifications as to why this or that layout is bette...

Many applications in music information retrieval require the analysis of the harmonic structure of a music piece. In Western music, the harmonic structure can be often be well illustrated by the chord structure and sequence. This chapter presents a technique of disambiguation for chord recognition based on a priori knowledge of probabilities of voi...

Over the past few years, a number of computer-based orchestral conducting systems have been designed and implemented. However, only a few of them have been developed to help a user learn and practice musical conducting gestures. This paper is intended to address research related to this area. It utilizes an infrared baton and an acceleration sensor...

In this paper, we discuss our experience in offering a usability course with projects taken from an active open source software development project. We describe what was done in the class inside the larger context of the usability of open source software. We conclude with an invitation for others to adopt this model and use it for their own purpose...

This paper presents a technique of disambiguation for chord recognition based on a-priori knowledge of probabilities of chord voicings in the specific musical medium. The main motivating example is guitar chord recognition, where the physical layout and structure of the instrument, along with human physical and temporal constraints, make certain ch...

NEEDS REWRITE We present an audio browsing and edit- ing paradigm that incorporates the \focus plus context" visual inter- action metaphor. A traditional waveform is displayed in full, and an area of focus is dynamically re-calculated to provide maximum detail in- focus and minimum detail in-context. The interaction metaphor also si- multaneously r...

We have constructed a poker classification system which makes informed betting decisions based upon three defining features ex- tracted while playing poker: hand value, risk, and aggressiveness. The system is implemented as a poker player agent, and as such, the goals of the classifier are not only to correctly determine whether each hand should be...

We present an audio browsing and editing paradigm that incorporates the "focus plus context" visual interaction metaphor. A traditional waveform is displayed in full, and an area of focus is dynamically re-calculated to provide maximum detail in-focus and minimum detail in-context. The interaction metaphor also simultaneously re-scales a frequency-...

In this paper we discuss the use of sound spatialization in western classical music and attempt to explain a potential reason why it is not found in current popular music. We argue that the barriers limiting the use of musical sound spatialization are nearly gone and attempt to facilitate the return by providing a computer interface for musical spa...

In this paper we will report on the use of real-time sound spatialization in Challenging Bodies, a trans-disciplinary performance project at the University of Regina. Using well-understood spatialization techniques mapped to a custom interface, a computer system was built that allowed live spatial control of ten sound signals from on-stage performe...

Instruments like the piano or guitar have a long tradition in many cultures such that non-musicians who encounter them understand that the piano keys can be pressed and the guitar strings can be plucked. Users of computer-based sound synthesis tools must use parameter names and inter-face feedback to develop a model of the available sound space of...

Talking and singing seem disparate, but there are a range of human utterances that fall between them, such as poetry, chanting, and rap music. This paper presents research into differentiation between talking and singing, development of feature-based analysis tools to explore the continuum between talking and singing, and evaluating human perceptio...

A multimedia composition can be regarded as tracing a path through a very high-dimensional parameter space. Hepting and Gerhard (2004) described an approach for managing parameter spaces for multimedia composition that allows an artist to explore both individual compositional elements, like music and animation, and integrated compositions in a more...

Unlike fixed-pitch instruments such as the piano, human singing can stray from a target pitch by as much as a semitone while still being perceived as a single fixed note. This paper presents a study of the difference between tar- get pitch and actualized pitch in natural singing. A set of 50 subjects singing the same melody and lyric is used to com...

We propose an interactive composition technique to facilitate the exploration of complex parameter spaces. More computer-aided composition than algorithmic composition, human aes-thetic judgment is an integral part of the technique. From a set of central parameters, the composer chooses a mapping which allows the generation of both audio and video...

Although the capability to create whole animations by computer has greatly enhanced the artist’s repertoire, it does not by itself address the issue that Whitney [1980] encountered while producing his film, Arabesque, where the “concept was not deeply ‘mined’ for its fullest possibilities.” Systematic exploration of alternatives is usually costly b...

This paper describes an approach to match visual and acoustic parameters to produce an animated musical expression. Music may be generated to correspond to animation, as described here; imagery may be created to correspond to music; or both may be developed simultaneously. This approach is intended to provide new tools to facilitate both collaborat...

Although many artists have worked to create associations between music and animation, this has traditionally be done by developing one to suit the pre-existing other, as in visualization or sonification. The approach we employ in this work is to enable the simultaneous development of both music and sound from a common and rather generic central par...

Pitch extraction (also called fundamental frequency estimation) has been a popular topic in many fields of research since the age of computers. Yet in the course of some 50 years of study, current techniques are still not to a desired level of accuracy and robustness. When presented with a single clean pitched signal, most techniques do well, but w...

This paper discusses the merit of using silence as a cue to the rhythmic structure of human utterances, specifically speech and song. Identifying the rhythm of a human utterance applies to the problem of music information retrieval, specifically in the instance where a human musical utterance (i.e. song) is used as a query in a musical database. Cu...

The automatic discrimination between acoustical categories has been an increasingly interesting problem in the fields of computer listening, multimedia databases, and music information retrieval. A system is presented which automatically generates classification models, given a set of destination classes and a set of a priori labeled acoustic event...

Automatic audio signal classification is one of the general research areas in which algorithms are developed to allow computer systems to understand and interact with the audio environment. Human utterance classification is a specific subset of audio signal classification in which the domain of audio signals is restricted to those likely to be enco...

this paper. To investigate the perceptual differences between talking and singing, human subjects were exposed to a corpus of singing and talking sounds, and asked first to classify each sound on a scale between speaking and singing, and then to indicate the characteristics of the sounds that lead to their judgements. The subject responses indicate...

In this paper we present the collection and annotation process of a corpus of human utterance vocalizations used for speech and song research. The corpus was collected to fill a void in current research tools, since no corpus currently exists which is useful for the classification of intermediate utterances between speech and monophonic singing. Mu...

Human speech and song seem disparate, but a range of utterances between speech and song are evident, such as poetry, chant, and rap, which have features of both singing and speaking. This work seeks to identify and characterize the perceptual features relevant for a fuzzy classification of utterances between speech and singing. The speech-ness or s...

There are several modern methods to visualize sounds, from the oscilloscope and the spectrometer, to colour organs and strobe lights. Phase space is a relatively new way to visualize sounds. Originally used to better understand the chaos of strange attractors and other non-periodic systems, it can be used to observe the regularity of periodic and p...

Audio signal classification consists of extracting physical and perceptual features from a sound, and of using these features to identify into which of a set of classes the sound is most likely to fit. The feature extraction and classification algorithms used can be quite diverse depending on the classification domain of the application. This paper...

Relative pitch perception is the identification of the relationship between two successive pitches without identifying the pitches themselves. Absolute pitch perception is the identification of the pitch of a single note without relating it to another note. To date, most pitch algorithms have concentrated on detecting the absolute pitch of a signal...

This paper is a proposal to do research work on features, techniques and strategies for human utterance classification (assigning individual human vocal utterances to one (or more) of a group of possible classes including speech, song, and others). This is a sub-problem of audio signal classification (assigning individual sounds to one (or more) of...

Audio signal classification (ASC) consists of extracting relevant features from a sound, and of using these features to identify into which of a set of classes the sound is most likely to fit. The feature extraction and grouping algorithms used can be quite diverse depending on the classification domain of the application. This paper presents backg...

Computer music analysis is investigated, with specific reference to the current research fields of automatic music transcription, human music perception, pitch determination, note and stream segmentation, score generation, timefrequency analysis techniques, and musical grammars. Human music perception is investigated from two perspectives: the comp...

This pager presents an image compression system based on the wavelet transform. The compression system includes preprocessing to remove unimportant information, wavelet transform of the image, thresholding of the wavelet coefficients, zerotree coding of the thresholded coefficients, and finally arithmetic coding of the zerotree code. Post-processin...

As the increase of the sales of Wii game consoles, the Wii Remote is able to used as a common input device for a com-puter system. Some software is developed for acquiring the data from the Wii Remote for further processing. This paper presents a Wii-based gestural interface for computer-based conducting systems. It employs the infrared camera in a...

With the increase of sales of Wii game consoles, it is be-coming commonplace for the Wii remote to be used as an alternative input device for other computer systems. In this paper, we present a system which makes use of the infrared camera within the Wii remote to capture the gestures of a conductor using a baton with an infrared LED and battery. O...