No file available
To read the file of this research,
you can request a copy directly from the authors.
Classifying Songs with EEG
Preprints and early-stage research may not have been peer reviewed yet.
Abstract
This research study aims to use machine learning methods to characterize the EEG response to music. Specifically, we investigate how resonance in the EEG response correlates with individual aesthetic enjoyment. Inspired by the notion of musical processing as resonance, we hypothesize that the intensity of an aesthetic experience is based on the degree to which a participants EEG entrains to the perceptual input. To test this and other hypotheses, we have built an EEG dataset from 20 subjects listening to 12 two minute-long songs in random order. After preprocessing and feature construction, we used this dataset to train and test multiple machine learning models.
ResearchGate has not been able to resolve any citations for this publication.
As an emerging sub-field of music information retrieval (MIR), music imagery information retrieval (MIIR) aims to retrieve information from brain activity recorded during music cognition–such as listening to or imagining music pieces. This is a highly inter-disciplinary endeavor that requires expertise in MIR as well as cognitive neuroscience and psychology. The OpenMIIR initiative strives to foster collaborations between these fields to advance the state of the art in MIIR. As a first step, electroencephalography (EEG) recordings of music perception and imagination have been made publicly available, enabling MIR researchers to easily test and adapt their existing approaches for music analysis like fingerprinting, beat tracking or tempo estimation on this new kind of data. This paper reports on first results of MIIR experiments using these OpenMIIR datasets and points out how these findings could drive new research in cognitive neuroscience.
Music perception involves complex brain functions. The relationship between music and brain such as cortical entrainment to periodic tune, periodic beat, and music have been well investigated. It has also been reported that the cerebral cortex responded more strongly to the periodic rhythm of unfamiliar music than to that of familiar music. However, previous works mainly used simple and artificial auditory stimuli like pure tone or beep. It is still unclear how the brain response is influenced by the familiarity of music. To address this issue, we analyzed electroencelphalogram (EEG) to investigate the relationship between cortical response and familiarity of music using melodies produced by piano sounds as simple natural stimuli. The cross-correlation function averaged across trials, channels, and participants showed two pronounced peaks at time lags around 70 and 140 ms. At the two peaks the magnitude of the cross-correlation values were significantly larger when listening to unfamiliar and scrambled music compared to those when listening to familiar music. Our findings suggest that the response to unfamiliar music is stronger than that to familiar music. One potential application of our findings would be the discrimination of listeners' familiarity with music, which provides an important tool for assessment of brain activity.
Most humans have a near-automatic inclination to tap, clap, or move to the beat of music. The capacity to extract a periodic beat from a complex musical segment is remarkable, as it requires abstraction from the temporal structure of the stimulus. It has been suggested that nonlinear interactions in neural networks result in cortical oscillations at the beat frequency, and that such entrained oscillations give rise to the percept of a beat or a pulse. Here we tested this neural resonance theory using MEG recordings as female and male individuals listened to 30 s sequences of complex syncopated drumbeats designed so that they contain no net energy at the pulse frequency when measured using linear analysis. We analyzed the spectrum of the neural activity while listening and compared it to the modulation spectrum of the stimuli. We found enhanced neural response in the auditory cortex at the pulse frequency. We also showed phase locking at the times of the missing pulse, even though the pulse was absent from the stimulus itself. Moreover, the strength of this pulse response correlated with individuals’ speed in finding the pulse of these stimuli, as tested in a follow-up session. These findings demonstrate that neural activity at the pulse frequency in the auditory cortex is internally generated rather than stimulus-driven. The current results are both consistent with neural resonance theory and with models based on nonlinear response of the brain to rhythmic stimuli. The results thus help narrow the search for valid models of beat perception.
Significance
We demonstrate that cortical oscillatory activity in both low (<8 Hz) and high (15–30 Hz) frequencies is tightly coupled to behavioral performance in musical listening, in a bidirectional manner. In light of previous work on speech, we propose a framework in which the brain exploits the temporal regularities in music to accurately parse individual notes from the sound stream using lower frequencies (entrainment) and in higher frequencies to generate temporal and content-based predictions of subsequent note events associated with predictive models.
The ability to perceive a regular beat in music and synchronize to this beat is a widespread human skill. Fundamental to musical behaviour, beat and meter refer to the perception of periodicities while listening to musical rhythms and often involve spontaneous entrainment to move on these periodicities. Here, we present a novel experimental approach inspired by the frequency-tagging approach to understand the perception and production of rhythmic inputs. This approach is illustrated here by recording the human electroencephalogram responses at beat and meter frequencies elicited in various contexts: mental imagery of meter, spontaneous induction of a beat from rhythmic patterns, multisensory integration and sensorimotor synchronization. Collectively, our observations support the view that entrainment and resonance phenomena subtend the processing of musical rhythms in the human brain. More generally, they highlight the potential of this approach to help us understand the link between the phenomenology of musical beat and meter and the bias towards periodicities arising under certain circumstances in the nervous system. Entrainment to music provides a highly valuable framework to explore general entrainment mechanisms as embodied in the human brain.
Following the success in Music Information Retrieval (MIR), research is now steering towards understanding the relationship existing between brain activity and the music stimuli causing it. To this end, a new MIR topic has emerged, namely Music Imagery Information Retrieval, with its main scope being to bridge the gap existing between music stimuli and its respective brain activity. In this paper, the encephalographic modality was chosen to capture brain activity as it is more widely available since of-the-shelf devices recording such responses are already affordable unlike more expensive brain imaging techniques. After defining three tasks assessing different aspects of the specific problem (stimuli identification, group and meter classification), we present a common method to address them, which explores the temporal evolution of the acquired signals. In more detail, we rely on the parameters of linear time-invariant models extracted out of electroencephalographic responses to heterogeneous music stimuli. Subsequently, the probability density function of such parameters is estimated by hidden Markov models taking into account their succession in time. We report encouraging classification rates in the above-mentioned tasks suggesting the existence of an underlying relationship between music stimuli and their electroencephalographic responses.
Objective: The frequency-following response (FFR) is a neurophonic potential used to assess auditory neural encoding at subcortical stages. Despite the FFR’s empirical and clinical utility, basic response properties of this evoked potential remain undefined.
Design: We measured FFRs to speech and nonspeech (pure tone, chirp sweeps) stimuli to quantify three key properties of this potential: level-dependence (I/O functions), adaptation and the upper limit of neural phase-locking.
Study sample: n = 13 normal-hearing listeners.
Results: I/O functions showed FFR amplitude increased with increasing stimulus presentation level between 25 and 80 dB SPL; FFR growth was steeper for tones than speech when measured at the same frequency. FFR latency decreased 4–5 ms with decreasing presentation level from 25 and 80 dB SPL but responses were ∼2 ms earlier for speech than tones. FFR amplitudes showed a 50% reduction over 6 min of recording with the strongest adaptation in the first 60 s (250 trials). Estimates of neural synchronisation revealed FFRs contained measurable phase-locking up to ∼1200–1300 Hz, slightly higher than the single neuron limit reported in animal models.
Conclusions: Findings detail fundamental response properties that will be important for using FFRs in clinical and empirical applications.
Reading the human mind has been a hot topic in the last decades, and recent research in neuroscience has found evidence on the possibility of decoding, from neuroimaging data, how the human brain works. At the same time, the recent rediscovery of deep learning combined to the large interest of scientific community on generative methods has enabled the generation of realistic images by learning a data distribution from noise. The quality of generated images increases when the input data conveys information on visual content of images. Leveraging on these recent trends, in this paper we present an approach for generating images using visually-evoked brain signals recorded through an electroencephalograph (EEG). More specifically, we recorded EEG data from several subjects while observing images on a screen and tried to regenerate the seen images. To achieve this goal, we developed a deep-learning framework consisting of an LSTM stacked with a generative method, which learns a more compact and noise-free representation of EEG data and employs it to generate the visual stimuli evoking specific brain responses.
OurBrain2Image approach was trained and tested using EEG data from six subjects while they were looking at images from 40 ImageNet classes. As generative models, we compared variational autoencoders (VAE) and generative adversarial networks (GAN). The results show that, indeed, our approach is able to generate an image drawn from the same distribution of the shown images. Furthermore, GAN, despite generating less realistic images, show better performance than VAE, especially as concern sharpness. The obtained performance provides useful hints on the fact that EEG contains patterns related to visual content and that such patterns can be used to effectively generate images that are semantically coherent to the evoking visual stimuli.
The experience of musical rhythm is a remarkable psychophysical phenomenon, in part because the perception of periodicities, namely pulse and meter, arise from stimuli that are not periodic. One possible function of such a transformation is to enable synchronization between individuals through perception of a common abstract temporal structure (e.g., during music performance). Thus, understanding the brain processes that underlie rhythm perception is fundamental to explaining musical behavior. Here, we propose that neural resonance provides an excellent account of many aspects of human rhythm perception. Our framework is consistent with recent brain-imaging studies showing neural correlates of rhythm perception in high-frequency oscillatory activity, and leads to the hypothesis that perception of pulse and meter result from rhythmic bursts of high-frequency neural activity in response to musical rhythms. High-frequency bursts of activity may enable communication between neural areas, such as auditory and motor cortices, during rhythm perception and production.
Towards Music Imagery Information Retrieval: Introducing the OpenMIIR Dataset of EEG Recordings from Music Perception and Imagination
- Sebastian Stober
Stober, Sebastian, et al (2015). "Towards Music Imagery
Information Retrieval: Introducing the OpenMIIR Dataset
of EEG Recordings from Music Perception and
Imagination." ISMIR.