
Haruhiro KatayoseKwansei Gakuin University · Department of Human System Interaction
Haruhiro Katayose
Ph.D (Engineering)
About
122
Publications
18,478
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
1,004
Citations
Introduction
Additional affiliations
April 1997 - March 2001
April 2002 - present
April 2002 - present
Education
April 1986 - March 1991
Osaka Univsersity
Field of study
- Engineering Science
Publications
Publications (122)
To enhance an audience's immersive experience of a media artwork, it is important to provide the audience with more than one channel through which they can take the initiative to participate in the artwork. It is also crucial to provide a natural interaction between the artwork and its audience. Based on these ideas, we developed an interactive med...
In this paper, we present a Java-based framework for supporting the development of music-related software, called the CrestMuse Toolkit. Although various efficient software for dataflow visual programming of sound and music processing have been developed, they are not necessarily suitable for exploiting conventional programming skills and experienc...
These are the proceedings of the 9th International Conference on Advances in Computer Entertainment ACE 2012). ACE has become the leading scientific forum for dissemination of cutting-edge research results in the area of entertainment computing. Interactive entertainment is one of the most vibrant areas of interest in modern society and is amongst...
Previous studies have suggested that professional musicians comprehend features of music-derived sound even if the sound sequence lacks the traditional temporal structure of music. We tested this hypothesis through behavioral and functional brain imaging experiments. Musicians were better than nonmusicians at identifying scrambled pieces of piano m...
Designing the behavioral patterns of video game agents (Non-player character: NPC) is a crucial aspect in developing video games. While various systems that have aimed at automatically acquiring behavioral patterns have been proposed and some have successfully obtained stronger patterns than human players, those patterns have looked mechanical. Whe...
This paper introduces a multi-media performance system called "MoPH" that allows the user to enjoy playing graphics and sounds using an iPad. The computer graphics and sounds are generated based on the movement of "objects" that are subject to physical laws. Users of the MoPH define and interact with the object using the multi-touch functions of an...
Research into music generation and into emulating human musical competence has attracted much attention in the field of computer science. In general, the results of academic research should be verified by assessing `objective effectiveness', which is often represented by a `recognition ratio'. Although `objective effectiveness' is also a requiremen...
Designing behavior patterns of video game agents (COM players) is a crucial aspect of video game development. While various systems aiming to automatically acquire behavior patterns has been proposed and some have successfully obtained stronger patterns than human players, the obtained behavior patterns looks mechanical. We present herein an autono...
The VirtualPhilharmony (VP) system conveys the sensation of conducting an orchestra to a user (a conductor). VPs performances are created through interaction between the conductor and orchestra, exactly like real performance. "Concertmaster function has been already implemented by incorporating the heuristics of conducting an orchestra. A precisely...
A new non-photorealistic rendering method using a depth map has been developed, on the basis of which a real-time cartoon-like video converter called Takumi was created. Many systems have been proposed that convert a given image into a non-photorealistic one. Images that these conventional methods render lack in perspective, due to processing being...
Various attempts at automatic music composition systems have been made, but they have not addressed the issue of how the user can edit a composed piece. In this paper, we propose a human-in-the-loop music composition system, in which the manual editing stage is integrated into the composition process. This system first generates a musical piece bas...
First Person Shooters are among the most played computer video games. They combine navigation, interaction and collaboration in 3D virtual environments using simple input devices, i.e. mouse and keyboard. In this paper, we study the possibilities brought by these games for musical interaction. We present the Couacs, a collaborative multiprocess ins...
Musical performance requires motor skills to coordinate the movements of multiple joints in the hand and arm over a wide range of tempi. However, it is unclear whether the coordination of movement across joints would differ for musicians with different skill levels and how inter-joint coordination would vary in relation to music tempo. The present...
Various attempts at automatic music composition systems have been made but they have
not dealt with the issue of how to edit the composed piece by the user. In this paper, we
propose an automatic music composition system, called OrpheusBB, where the user edits
the composed piece after the automatic composition. The feature of this system is to rege...
The Performance Rendering Contest (Rencon) is an an-nual international competition in which entrants present computer systems they have developed for generating ex-pressive musical performances, which audience members and organizers judge. Recent advances in performance-rendering technology have brought with them the need for a means for researcher...
This paper introduces a multi-media performance system called "MoPH" that allows the user to enjoy playing graphics and sounds using an iPad. The computer graphics and sounds are generated based on the movement of "objects" that are subject to physical laws. Users of MoPH define and interact with the object using the multi-touch functions and accel...
In this study, we investigated brain activity under social pressure using Solomon Asch’s experimental paradigm. We replicated the group situation using the MORI technique and recorded the brain activity using functional near-infrared spectroscopy (fNIRS). Eight Japanese healthy males in groups of four participated in this study. The Asch experiment...
Equations of motion for inverse and forward dynamics computations. The file contains complete equations of motion used for performing inverse and forward dynamics analyses. The upper extremity was assumed as four interconnected rigid links (upper-arm, forearm, hand, and finger).
Production of a variety of finger-key touches in the piano is essential for expressive musical performance. However, it remains unknown how expert pianists control multi-joint finger and arm movements for manipulating the touch. The present study investigated differences in kinematics and kinetics of the upper-limb movements while expert pianists w...
VirtualPhilharmony" (V.P.) is a conducting interface that enables users to perform expressive music with conducting action. Several previously developed conducting interfaces do not satisfy users who have conducting experience because the feedback from the conducting action does not always correspond with a natural performance. The tempo scheduler,...
Music performance is processing to embody musical ideas in concrete sound, giving expression to tempo and dynamics and articulation to each note. Human compe-tence in music performance rendering is enhanced and fostered by supplementing a lack of performance skill and musical knowledge using computers. This paper in-troduces a performance design en...
This study evaluates a virtual Japanese drum system that delivers the drum sound through headphones along with haptic stimulation via vibrators on the abdomen. Two psychological experiments, which were conducted using subjective ratings and a functional brain imaging method, were performed to investigate the factors necessary to improve the perceiv...
This paper proposes a method of giving humanoid robots a natural humanlike walk, which we call the extended-knee walk. Unlike the bent-knee walk of most humanoid robots to date, this walk includes a period in which the knee is fully extended. A parallel mechanism is used in the legs and a method of calculating the walk trajectory copes with the dif...
This paper describes a music performance design system focusing on phrasing, the design and development of an intuitive interface to assist music performance design system. The proposed interface has an editor to control the parameter curves of ¿dynamics¿ and ¿tempos¿ of hierarchical phrase structures, and supports analysis mechanisms for hiera...
One kind of pleasure that jam sessions bring is deciding a melody or an accompaniment while mutually predicting what the other
participants are going to play. We propose a jam session system, called BayesianBand, which provides this kind of musical pleasure through sessions with computers. With this system, the chord progression in
a session is not...
This paper describes a singing design method based on morphing, the design and development of an intuitive interface to assist
morphing-based singing design. The proposed interface has a function for real-time morphing, based on simple operation with
a mouse, and an editor to control the singing features in detail. The user is able to enhance singi...
Technologies recreating piano performance in the form of CG animation are eagerly anticipated by people working in various fields, such as content production, music education, etc. Nonetheless, much of the past research has dealt with the mechanical finger movements in piano practice support systems and performance support GUIs, etc. and there has...
In this study, we note chordal structure as the factors determining the musical mood, and suggested the model for calculating the mood scores (dissonance, tension and modality) from the chords. The model was implemented for the system which is describing the mood scores as various color patch.
This paper describes an automatic chord voicing system using the Bayesian network. Automatic chord voicing is not easy because it is necessary to decide tension notes and inversions by taking into account both musical simultaneity and sequentiality. To solve this problem, we construct a chord voicing model based on the Bayesian network. This model...
Technologies recreating piano performance in the form of CG animation are eagerly anticipated by people working in various fields, such as content production, music education, etc. Nonetheless, much of the past research has dealt with the mechanical finger movements in piano practice support systems and performance support GUIs, etc. and there has...
Technologies recreating piano performance in the form of CG animation are eagerly anticipated by people working in various fields, such as content production, music education, etc. Nonetheless, much of the past research has dealt with the mechanical finger movements in piano practice support systems and performance support GUIs, etc. and there has...
Many studies have focused on the relationship between playing video games and brain activity. Most of these studies have reported that regions of the prefrontal cortex are deactivated when video games are played. However, it is natural to assume that the effect on humans varies with the player's age, attitude, or mastery level. In this paper, we fo...
Behavior and strategy of computers(COM) have recently attracted considerable attention with regards to video games, with the development of hardware and the spread of entertainment on the Internet. Previous studies have reported strategy-acquisition schemes for board games and fighting games. However, there have been few studies dealing with the sc...
Various studies on music information retrieval have been conducted. Most of them used low-level features such as spectral and cepstral features, which are effective to some extent but have a limit because they do not directly or clearly correspond to any musical aspects. In this paper, we design new audio features to be extracted from bass lines.Be...
We introduce the CrestMuse Performance Expression Data- base (CrestMusePEDB), a music database that describes mu- sic performance expression and is available for academic re- search. While music databases are being provided as MIR technologies continue to progress, few databases deal with performance expression. We constructed a music expres- sion...
We propose new audio features that can be extracted from bass lines. Most previous studies on content-based music information retrieval (MIR) used low-level features such as the mel-frequency cepstral coefficients and spectral centroid. Musical similarity based on these features works well to some extent but has a limit to capture fine musical char...
Rencon (Performance Rendering Contest) is an annual international competition at which entrants present the computer systems they have developed for generating expressive musical performances and audience members and organizers judge the performances. Recent advances in performance rendering technology have brought with them the need for a means fo...
We developed a computational model for automatically voic-ing chords based on a Bayesian network. Automatic chord voicing is difficult because it is necessary to choose extended notes and inversions by taking into account musical simultane-ity and sequentiality. We overcome this difficulty by inferring the most likely chord voicing using a Bayesian...
One of the advantages of case-based systems is that they can generate expressions even if the user doesn't know how the system applies expression rules. However, the systems cannot avoid the problem of data sparseness and do not permit a user to improve the expression of a certain part of a melody directly. After discussing the functions required f...
We describe an interactive multimedia application called crossing colorful communication (C.C.C), players of which can enjoy collaborative and interactive media control by speaking or making sound. Players of C.C.C. fire visual bullets, the direction and the size of which are determined by the pitch and the intensity of each player's voice. When bu...
This paper introduces jPop-E (java-based PolyPhrase En-semble), an assistant system for the Pop-E performance rendering system. Using this assistant system, MIDI data including expressive tempo changes or velocity control can be created based on the user's musical intention. Pop-E (PolyPhrase Ensemble) is one of the few machine systems devoted to c...
Recently commercial breaks are often inserted just before the climax of stories in TV programs in Japan. One article reports that commercials inserted just before the climax will dissatisfy the audience and discourage their concentration. In this research, we study the effects of the timing of commercial breaks by the measurement of brain activity...
This paper proposes a musical performance-rendering model, called Pop-E, for expressive ensembles: homo-phonic and polyphonic music. Polyphonic music has to be synchronized after the independent expression of each voice. Pop-E gives phrasing expressions to each part and has a means to align the timings of the parts according to the group structure...
This paper describes our preliminary investigation on mu- sical brain activities using fNIRS. We conducted two sets of experiments on musical activities: 1) a comparison of ac- tivities in subjects hearing music but not paying attention to it, listening to music actively, and playing music using an interactive performance interface called iFP, and...
We held the third meeting at the 19th Annual Conference of JSAI in the series of near-future challenge of Establishing Case-Based Design Support and Assessment Framework. In this Research Notes, following the presentations and discussions made at the meeting, we investigate how we bridge the gap between our design support technology and the contrib...
This paper describes an environment for editing and performing interactive media art/entertainment. The design background
is to provide artistic/entertainment pieces, in which multiple people can participate without special sensory equipment. This
paper introduces a gesture input function using color tags in the image and some matching functions to...
We assume that there are various musical groupings of perceptions according to the degree of schemata and there are two dominant music grouping schemata; (a) accent-oriented grouping schema and (b) phrasing schema (musical expression referred to as the Rainbow type). In order to verify these hypotheses, we investigated how listeners’ groupings chan...
There is a challenge in designing a system for timbre design that is engaging for new users and enables experienced users
to intuitively design a diverse range of complex timbres. This paper discusses some of the issues involved in achieving these
aims and proposes that a timbre can be intuitively represented as an image. The design of TimbrePainte...
Trading card games have become very popular among children all over the world. This paper describes an example of enhancing a trading card game involved with Augmented Reality technologies, the players of which can experience monster summoning and battles, as if they are happening on their own arms.
Demands for multimedia contents are increasing.
Mix-down (track down) occurs at the final stage of commercial music production. Selecting effectors for each sound track and setting the parameters of each effecter balances the sound from each track in stereo (mix-down design). The mix-down process greatly influences the final sound quality. Recently, professional mixing tasks have been done on di...
This paper describes a performance interface called iFP that enables players to play music as if he or she had the hands of
the virtuoso. iFP is a tapping-style musical interface and refers to a pianist’s expressiveness described in a performance
template. The paper describes the scheduler that allows a player to mix her/his own intension with the...
Demand for music databases is increasing for the stud- ies of musicology and music informatics. Our goal is to construct databases that contain deviations of tempo, and dynamics, start-timing, and duration of each note. This paper describes a procedure based on hybrid use of DP Matching and HMM that efficiently extracts deviations fromMIDI-formatte...
Rencon is an annual international event that started in 2002. It has roles of (1) pursuing evaluation methods for systems whose output includes subjective issues, and (2) providing a forum for researches of several fields related to musical expression. In the past, Rencon was held as a workshop as- sociated with a musical contest that provided a fo...
This paper describes an approach for playing expressive music, as it refers to a pianist's expressiveness, with a tapping-style interface. MIDI-formatted expressive performances played by pianists were first analyzed and transformed into performance templates, in which the deviations from a canonical description was separately described for each ev...
To seek evaluation methods for systems that gener-ate expressive performance, Rencon (CONtest for performance RENdering systems) was started in 2002. Performance Rendering is a typical system whose output is interpreted subjectively. Further-more, this research concerns not only computer sci-ence but also musicology, psychology, and cogni-tion, Ren...
Rencon, CONtest for performance RENdering systems, started in 2002. Since we have not had evaluation methods for such systems whose output is interpreted subjectively, furthermore performance rendering is a research of not only computer science but also musi-cology, psychology, and cognition, Rencon has roles of (1) pursuing evaluation methods for...
Design is one of the typical tasks which are realized with both intelligence and sense. When demands for contents are increasing in this decade, design assistance with computers is one of the most important information technology services. The design assistance method known as Kansei Engineering, which combines multivariate analysis of impression a...
Evaluating computer music research from the system point of view has not been established because of the subjective nature of music. In order for such research areas to evolve and prosper, we believe the evaluation method based on objective understanding is essential. We started and propel RENCON (performance rendering concours), a world-wide-landm...
This paper introduces a piano performance system called “INSPIRATION”, receiving rough keyboard input and translating it into musically correct performance in real time. This system changes the scale notes and chord notes, maintaining the performer’s intention, expressed as the melody contour, the chord density, and the input timing. The system ena...
In this paper the authors describe the development of a session system which follows human performance and creates an improvisational accompaniment. One of the features of this system is that it can use acoustic instruments such as pianos and guitars instead of MIDI instruments which are common in conventional session systems. In a session system,...
A set of procedures to generate scat singing by manipulating a small set of seed voices using a speech manipulation system called STRAIGHT [Kawahara et al., Speech Commun. 27, 187–207 (1999)] is proposed. F0 adaptive spectral smoothing based on a second‐order cardinal spline combined with an F0 extractor based on a fixed‐point analysis from filter...
This paper describes a motion capturing system for human arm
motion in real time; directly, simply and precisely by integrating two
types of sensor; gyroscopes and accelerometers. A new sensor fusion
technique is proposed which enlarges the merits of each sensor and
compensates for their deficiencies. Its potential precision has been
verified by ex...
Recently a field called interactive art, where performers control multimedia interactively, has been established and new musical instruments called hyper instruments have been developed. Most of hyper instruments are focussed on western musical instruments which have been given much attention to standardization. The number of hyper instruments rega...
In recent years, the researches for conducting the human motion capture and recognition are prosperously carried out in the field of man-machine interface, sign-language recognition, dancing behavior description and so force. Image processing techniques and magnetic sensing devices are widely used, but the former exhibits some disadvantages due to...