Mark Gotham’s research while affiliated with Durham University and other places

We introduce a project that revives a piece of 15th-century Korean court music, Chihwapyeong and Chwipunghyeong, composed upon the poem Songs of the Dragon Flying to Heaven. One of the earliest examples of Jeongganbo, a Korean musical notation system, the remaining version only consists of a rudimentary melody. Our research team, commissioned by the National Gugak (Korean Traditional Music) Center, aimed to transform this old melody into a performable arrangement for a six-part ensemble. Using Jeongganbo data acquired through bespoke optical music recognition, we trained a BERT-like masked language model and an encoder-decoder transformer model. We also propose an encoding scheme that strictly follows the structure of Jeongganbo and denotes note durations as positions. The resulting machine-transformed version of Chihwapyeong and Chwipunghyeong were evaluated by experts and performed by the Court Music Orchestra of National Gugak Center. Our work demonstrates that generative models can successfully be applied to traditional music with limited training data if combined with careful design.

Automatic harmonic analysis of music has recently been significantly improved by AugmentedNet, a convolutional recurrent neural network for predicting Roman numeral labels. The original network was trained on a combination of computer encodings of digital scores and human harmonic analyses thereof. Learning from these pairs, the system predicts new harmonic analyses for unseen examples. However, for much music, no score symbolic is available. For this study, we adjusted AugmentedNet for a direct application to audio signals (represented either by chromagrams or semitone spectra). We also implemented and compared further modifications to the network architecture: adding a preprocessing block designed to learn pitch spellings, increasing the network size, and adding dropout layers to avoid over-fitting. A thorough statistical analysis helped to identify the best among the proposed configurations and has shown that some of the optimization steps significantly increased the classification performance. We find that this adapted AugmentedNet can reach similar accuracy levels when faced with audio features as it achieves with the “cleaner” symbolic data on which it was originally trained.

Deep learning has recently established itself as a cluster of methods of choice for almost all classification tasks in music information retrieval. However, despite very good classification performance, it sometimes brings disadvantages including long training times and higher energy costs, lower interpretability of classification models, or an increased risk of overfitting when applied to small training sets due to a very large number of trainable parameters. In this paper, we investigate the combination of both deep and shallow algorithms for recognition of musical genres using a transfer learning approach. We train deep classification models once to predict harmonic, instrumental, and segment properties from datasets with respective annotations. Their predictions for another dataset with annotated genres are used as features for shallow classification methods. They can be trained over and again for different categories, and are particularly useful when the training sets are small, in a real world scenario when listeners define various musical categories selecting only a few prototype tracks. The experiments show the potential of the proposed approach for genre recognition. In particular, when combined with evolutionary feature selection which identifies the most relevant deep feature dimensions, the classification errors became significantly lower in almost all cases, compared to a baseline based on MFCCs or results reported in the previous work.KeywordsMusical genre recognitionDeep neural networksTransfer learningInterpretable featuresEvolutionary feature selection

While it is encouraging to see renewed attention to 'openness' in academia, that debate (and its interpretation of the F.A.I.R. principles) is often rather narrowly defined. This paper addresses openness in a broad sense, asking not so much whether a project is open, but how open and to whom. I illustrate these ideas through examples of my own ongoing projects which to seek to make the most of a potential symbiosis between academic and wider musical communities. Specifically, I discuss how these communities can both benefit from – and even work together on building – highly accessible and interoperable corpora of scores and analyses when ambitious openness is factored into decision making from the outset.

AugmentedNet is a new convolutional recurrent neural network for predicting Roman numeral labels. The network architecture is characterized by a separate convolu-tional block for bass and chromagram inputs. This layout is further enhanced by using synthetic training examples for data augmentation, and a greater number of tonal tasks to solve simultaneously via multitask learning. This paper reports the improved performance achieved by combining these ideas. The additional tonal tasks strengthen the shared representation learned through multitask learning. The synthetic examples, in turn, complement key transposition , which is often the only technique used for data augmentation in similar problems related to tonal music. The name 'AugmentedNet' speaks to the increased number of both training examples and tonal tasks. We report on tests across six relevant and publicly available datasets: ABC, BPS, HaydnSun, TAVERN, When-in-Rome, and WTC. In our tests, our model outperforms recent methods of functional harmony, such as other convolutional neural networks and Transformer-based models. Finally, we show a new method for reconstructing the full Roman numeral label, based on common Roman numeral classes, which leads to better results compared to previous methods.