Alejandro Rosa-Pujazón’s research while affiliated with University of Malaga and other places

In this paper, we present a system for the detection of fast gestural motion by using a linear predictor of hand movements. We also use the proposed detection scheme for the implementation of a virtual drumkit simulator. A database of drum-hitting motions is gathered and two different sets of features are proposed to discriminate different drum-hitting gestures. The two feature sets are related to observations of different nature: the trajectory of the hand and the pose of the arm. These two sets are used to train classifier models using a variety of machine learning techniques in order to analyse which features and machine learning techniques are more suitable for our classification task. Finally, the system has been validated by means of the Kinect application implemented and the participation of 12 different subjects for the experimental performance evaluation. Results showed a successful discrimination rate higher than 95 % for six different gestures per hand and good user experience.

In this paper, an implementation of a virtual reality based application for drumkit simulation is presented. The system tracks user motion through the use of a Kinect camera sensor, and recognizes and detects user-generated drum-hitting gestures in real-time. In order to compensate the effects of latency in the sensing stage and provide real-time interaction, the system uses a gesture detection model to predict user movements. The paper discusses the use of two different machine learning based solutions to this problem: the first one is based on the analysis of velocity and acceleration peaks, the other solution is based on Wiener filtering. This gesture detector was tested and integrated into a full implementation of a drumkit simulator, capable of discriminating up to 3, 5 or 7 different drum sounds. An experiment with 14 participants was conducted to assess the system's viability and impact on user experience and satisfaction.

In this paper, an augmented reality application for drumkit simulation is presented. The system is capable of classifying any percussive sounds produced by the user from an everyday desktop environment, e.g. clapping, snapping, stroking different objects with a pencil, etc., recognizing up to six different classes of drum hits. These different types of user-generated sounds will subsequently be associated to predefined drumkit sounds, resulting in a natural and intuitive audio interface for drummers and percussionists, which only requires a computer with a built-in microphone. A set of audio features and classification techniques are evaluated for the implementation of the aforementioned system.

This paper presents a low-cost add-on sensor that has been developed to provide other systems with music listening and note detection capabilities. This sensor has been integrated as part of an intelligent system for solfeo learning. The system is capable of listening the performance of a user as well as assessing the correctness of both the rhythmic pattern and intonation in the performance, and presenting practice exercises. Concretely, the system is based on the Lego Mindstorms NXT 2.0 robot. The system designed implements a training application with three training modalities, depending on whether the user wishes to improve his musical reading (rhythm and intonation) or listening abilities.

In this chapter, the use of advanced human computer interfaces to create innovative interaction paradigms for music applications (music creation, music manipulation, music games, etc.) is explored. The advances in the design and implementation of sensing technologies have provided the means to create new ways to interact with computers in a more natural way than the conventional computer framework with a mouse and a keyboard. More involving ans immersive experiences can be offered to the user than to these technologies. However, there is no silver bullet: each kind of sensing technology excels at some fields and lack others. Each application will demand its very own selection of sensors and the development of an adequate interaction metaphor. In this chapter, some of the most commonly used technologies for motion sensing are presented with as special focus on the possibilities of a 3D camera sensor (i.e. kinect) with regard to the design of human computer interfaces for musin interaction. We will present our findings in the studies we have conducted using these devices to develop augmented instruments. These include a drumkit stimulator or a virtual theremin. Additionally, the use of this type of interface for other music applications will be discussed. A description of the technical issues that need to be addressed to successfully implement these interaction paradigms is also given.