Douglas Turnbull

• Ithaca College

Singing songs can be an engaging and effective activity when learning a foreign language. In this paper, we describe a multi-language karaoke application called SLIONS: Singing and Listening to Improve Our Natural Speaking. When developing this application, we followed a user-centered design process which was informed by conducting interviews with...

In this paper, we explore the task of local music event recommendation. Many local artists tend to be obscure long-tail artists with a small digital footprint. That is, it can be hard to find social tag and artist similarity information for many of the artists who are playing shows in the local music community. To address this problem, we explore u...

We explore the use of personalized radio to facilitate the discovery of music created by local artists. We describe a system called MegsRadio.fm that produces a customizable stream of music by both local and well-known (non-local) artists based on seed artists, tags, venues and/or location. We hypothesize that the more popular artists provide conte...

Digital storage of personal music collections and cloud-based music services (e.g. Pandora, Spotify) have fundamentally changed how music is consumed. In particular, automatically generated playlists have become an important mode of accessing large music collections. The key goal of automated playlist generation is to provide the user with a cohere...

Searching for relevant content in a massive amount of multimedia information is facilitated by accurately annotating each image, video, or song with a large number of relevant semantic keywords, or tags. We introduce game-powered machine learning, an integrated approach to annotating multimedia content that combines the effectiveness of human compu...

Automatically generated playlists have become an impor-tant medium for accessing and exploring large collections of music. In this paper, we present a probabilistic model for generating coherent playlists by embedding songs and social tags in a unified metric space. We show how the embedding can be learned from example playlists, pro-viding the met...

A computer-vision system predicts music genre tags by making use of content-based image analysis, suggesting that we can learn some notion of artists' similarity on the basis of visual appearance alone.

We present the Beat-Sync-Mash-Coder, a new tool for semi-automated real-time creation of beat-synchronous music mashups. We combine phase vocoder and beat tracker technology to automate the task of synchronizing clips. Freeing the user from this task allows us to replace the traditional audio editing paradigm of the Digital Audio Workstation with a...

The task of automatically annotating music with text tags (referred to as autotagging) is vital to creating a large-scale semantic music discovery engine. Yet for an autotagging system to be successful, a large and cleanly-annotated data set must exist to train the system. For this reason, we have collected a data set, called Swat10k, which consist...

In developing automated systems to recognize the emotional content of music, we are faced with a problem spanning two disparate domains: the space of human emotions and the acoustic signal of music. To address this problem, we must develop models for both data collected from humans describing their perceptions of musical mood and quantitative featu...

This paper surveys the state of the art in automatic emo-tion recognition in music. Music is oftentimes referred to as a "language of emotion" [1], and it is natural for us to categorize music in terms of its emotional associations. Myriad features, such as harmony, timbre, interpretation, and lyrics affect emotion, and the mood of a piece may also...

We are interested in automatically calculating music similarity based on the visual appearance of artists. By collecting a large set of promotional photographs featuring artists and using a state-of-the-art image annotation system, we show that we can successfully annotate artists with a large set of (genre) tags. This suggests that we can learn so...

Tags are useful text-based labels that encode semantic information about music (instrumentation, genres, emo- tions, geographic origins). While there are a number of ways to collect and generate tags, there is generally a data sparsity problem in which very few songs and artists have been accurately annotated with a sufficiently large set of releva...

In the process of automatically annotating songs with de- scriptive labels, multiple types of input information can be used. These include keyword appearances in web docu- ments, acoustic features of the song's audio content, and similarity with other tagged songs. Given these individ- ual data sources, we explore the question of how to aggre- gate...

When attempting to annotate music, it is important to con- sider both acoustic content and social context. This pa- per explores techniques for collecting and combining multi- ple sources of such information for the purpose of building a query-by-text music retrieval system. We consider two representations of the acoustic content (related to timbre...

Associating labels with online products can be a labor- intensive task. We study the extent to which a standard \bag of visual words" image classier can be used to tag products with useful information, such as whether a sneaker has laces or velcro straps. Using Scale Invariant Feature Transform (SIFT) image descriptors at random keypoints, a hierar...

We present "Herd It", a competitive, online, multi-player game that has the implicit benefit of collecting tags for music. We describe Herd It's user-centered design process and demonstrate that the game can collect both musical and social data. This data can be used to build machine learning models that automatically associate music with tags. Her...

We compare five approaches to collecting tags for music: conducting a survey, harvesting social tags, deploying anno- tation games, mining web documents, and autotagging audio content. The comparison includes a discussion of both scala- bility (financial cost, human involvement, and computational resources) and quality (the cold start problem & pop...

We present a computer audition system that can both annotate novel audio tracks with semantically meaningful words and retrieve relevant tracks from a database of unlabeled audio content given a text-based query. We consider the related tasks of content-based audio annotation and retrieval as one supervised multiclass, multilabel problem in which w...

We apply a new machine learning tool, kernel combination, to the task of semantic music retrieval. We use 4 different types of acoustic content and social context feature sets to describe a large music corpus and derive 4 individual kernel matrices from these feature sets. Each kernel is used to train a support vector machine (SVM) classifier for e...

Games based on human computation are a valuable tool for collecting semantic information about images. We show how to transfer this idea into the music domain in order to collect high-quality semantic information about songs. We present Listen Game, a online, multiplayer game that measures the semantic relationship between music and words. In the n...

We improve upon query-by-example for content-based audio information retrieval by ranking items in a database based on semantic similarity, rather than acoustic similarity, to a query example. The retrieval system is based on semantic concept models that are learned from a training data set containing both audio examples and their text captions. Us...

Query-by-semantic-description (QBSD) is a natural paradigm for retrieving content from large databases of music. A ma- jor impediment to the development of good QBSD systems for music information retrieval has been the lack of a cleanly- labeled, publicly-available, heterogeneous data set of songs and associated annotations. We have collected the C...

A musical boundary is a transition between two musical segments such as a verse and a chorus. Our goal is to au- tomatically detect musical boundaries using temporally- local audio features. We develop a set of difference fea- tures that indicate when there are changes in perceptual as- pects (e.g., timbre, harmony, melody, rhythm) of the mu- sic....

A musically meaningful vocabulary is one of the keystones in building a computer audition system that can model the semantics of audio content. If a word in the vocabulary is inconsistently used by human annotators, or the word is not clearly represented by the underlying acoustic repre- sentation, the word can be considered as noisy and should be...

A musically meaningful vocabulary is one of the keystones in building a computer audition system that can model the semantics of audio content. If a word in the vocabulary is not clearly represented by the underlying acoustic representation, the word can be considered noisy and should be removed from the vocabulary. This paper proposes an approach...

We propose a query-by-text system for modeling a het- erogeneous data set of music and words. We quantitatively show that our system can both annotate a novel song with semantically meaningful words and retrieve relevant unla- beled songs from a database given a text-based query. We explain two feature extraction methods useful for summa- rizing th...

This paper explores the automatic classification of audio tracks into musical genres. Our goal is to achieve human-level accuracy with fast training and classification. This goal is achieved with radial basis function (RBF) networks by using a combination of unsupervised and supervised initialization methods. These initialization methods yield clas...

We explore the automatic classification of audio tracks into music genres using radial basis function (RBF) networks. We explore unsupervised and supervised methods for initializing the parameter of the radial basis functions. Using gradient descent during training of the networks, we find that we can significantly improve classification accuracy....

Despite the recent focus on semantic image/video annotation and retrieval, rela-tively little work has been done on semantic audio annotation and retrieval. We show that the supervised mutli-class nave Bayes model, which has successfully been used for image annotation, can be used to model the semantics of audio data. This model, as opposed a host...

We present a query-by-example system for content-based music information retrieval by ranking items in a database based on semantic similarity, rather than acoustic similar-ity, to a query example. The retrieval system is based on semantic concept models that are learned from the CAL-500 data set containing both audio examples and their text captio...

We are interested in automatically calculating music similar-ity based on the visual appearance of artists. By collecting a large set of promotional photographs featuring artists and using a state-of-the-art image annotation system, we show that we can successfully annotate artists with a large set of (genre) tags. This suggests that we can learn s...

Abstract We present a computer audition system that can both annotate novel audio tracks with semantically meaningful words and use a se- mantic query to retrieve relevant tracks from database of unlabeled audio content. We consider the related tasks of content-based au- dio annotation and retrieval as one supervised multi-class problem in which we...

TO BE WRITTEN LAST: (1) Develop human computa- tion games that collect semantic labels (i.e., semantically relevant words) for music. (2) This technique used to collect semantic information about images. (3) Describe Listen Game as a multiplayer, online game with realtime feedback. (4) Produces high-quality semantic labels for music. (5) Using this...