Conference PaperPDF Available

Behavioral and Neurophysiological Effects of Singing and Accompaniment on the Perception and Cognition of Song

Authors:

Abstract and Figures

In order to further investigate the effects of singing and accompaniment on the processing of language, a classroom experiment reported earlier was followed by an EEG experiment, using the same materials. 24 participants listened to four songs, each in one of four versions: spoken, sung a cappella, complete (sung with accompaniment), or vocalized (sung a cappella on 'lala'). During listening, EEG was measured, and after each song, a questionnaire was filled out. Behavioral results suggest that singing supports cued word recall, even after just one exposure, and focus on the lyrics. Furthermore, an accompaniment supports positive affect and appreciation of voice quality, and decreases seriousness. A preliminary EEG data analysis reveals that out-of-key notes elicit a slightly larger ERAN and N400 than in-key notes, a smaller P2 and N5, and a larger P600, followed by a larger late negativity ('N1400'). However, the larger ERAN and P2 are not visible in all conditions; the larger N400 is only significant in the condition complete, the larger P600 predominantly in the condition vocalized, and the late negativity only in the condition a cappella. These differences lead to the conclusion that the processing of in-key and out-of-key notes interacts with the presence of interpretable lyrics, indicating that music might affect the meaning of words or vice versa. The interaction between the processing of these notes and the presence of an accompaniment is more difficult to interpret.
Mean recall scores per condition. Time window 100-175; 175-225; 100-300. In the two time windows associated with the ERAN, the only difference which reaches significance is a small one between a cappella iks and a cappella ooks in the time-window 100-175. Three electrodes show a significant negativity for ooks (p < 0.05, see figure 4). However, visual inspection of the waveforms revealed that the ERANS often peak earlier and the P2s later. Hence, a post hoc analysis of the time windows 100-300 showed a significant negativity for all ooks compared to all iks (p < 0.05), which seems mainly due to a cappella ooks, where the P2 is almost absent. Time window 300-400; 400-500. In the time-window 300400, the waveforms of both a cappella and complete show a negative tendency for ooks. This difference is significant only for 5 centro-parietal electrodes in the condition complete (p < 0.05, see Figure 4, bottom). In the condition vocalized on the other hand, ooks show a positive tendency in this time window, which becomes significant between 400 and 500 on eleven centro-parietal electrodes (p < 0.05, see figure 4, bottom). In the conditions a cappella and complete on the other hand ooks show a positive shift in this time window. Time windows 600-900; 500-1000. Within the time window 600-900 all conditions together (p < 0.05), and the condition vocalized (p < 0.05) show a significant positivity for ooks across the whole scalp. In the complete condition 3 electrodes show such a significant positivity (p < 0.05). As the difference curves indicated that the P600 peaks either earlier or later (partly due to a late N5 peak) we also tested the time window 500-1000. In this time window, the positivities for both vocalized and all conditions together are significant at a higher level (p < 0.005). A cappella shows both positivities and negativities at various electrodes, hence on average there were no significant effects. Time windows 1350-1450, 1350-1800. In all conditions, the ook-curve shows a late negative shift. In the a-cappella condition, this leads to a significant negativity peaking around 1400. This negativity is significant over the whole scalp between 1350 and 1450 (p < 0.05); and at 8 electrodes between 1350 and 1850 (p < 0.05), see figure 4 upper half.
… 
Content may be subject to copyright.
A preview of the PDF is not available
... The classroom experiment will be reported on in Study 1, and the laboratory experiment in Study 2; it was reported on already in Schotanus, Eekhof and Willems (2018), but within that paper, parts of the data, relevant for the comparison with the classroom experiment, were left unanlyzed. These two settings were chosen in order to combine the ecological validity of the classroom study with the enhanced randomization of the laboratory study, to test sub-hypothesis 1.7, and to test whether songs can helpfully be used in classes in which a song is presented only once (for example History or Literature classes). ...
... An opportunity to investigate whether the results of Study 1 could be replicated in a design less confounded by group effects arose when a laboratory experiment (aiming for EEG and GSR measures) was conducted among 24 adults, using the same stimuli and almost the same questionnaires in a pseudo-randomized order (Schotanus, Eekhof & Willems, 2018). So far, only part of the behavioural data from this experiment has been analysed. ...
... So far, only part of the behavioural data from this experiment has been analysed. Schotanus, Eekhof, and Willems (2018) only reported regressions on the four factors retained after a factor analysis of the ratings for 17 lyric and voice related items, and on the results of the enhanced recall task. The results were largely in line with those of Study 1. ...
Article
Full-text available
A growing body of evidence indicates that music can support the processing of language. Some of its beneficial effects may even occur after one exposure. Accompaniment can also have an impact: in a-cappella singing, silences and out-of-key notes may confuse listeners, while accompaniment avoids silences and elucidates both rhythm and harmony, thereby supporting music-processing and concentration. These hypotheses were tested in two experiments. In a classroom setting, 271 pupils (M = 15.7 years old, SD = 0.9), listened to five out of 24 tracks (four songs in six different conditions) and completed a questionnaire after each one. As expected, the instrumental interludes between sung or spoken phrases in accompanied versions were rated less distracting than the silences that replace them in unaccompanied ones. Furthermore, perceived arousal, emotion, valence, and purity of singing were rated more positively in accompanied versions. Singing, on the other hand, supports the perceived intelligibility and comprehensibility of the lyrics. Finally, the music makes repetitions of words and phrases more meaningful and changes the lyrics' emotional meaning, wereby some aspects of sadness are associated with negative affect while other aspects of sadness are associated with positive affect. These results were by and large replicated in a better randomized laboratory experiment among 24 adults (M = 24.4; SD = 4.8).
... THE two studies presented in the target article are part of a program of research (Schotanus, 2015;Schotanus, Eekhof & Willems, 2018) investigating the possible beneficial effects of singing as a pedagogical mode of presentation, which builds on previous work that has found beneficial effects on verbatim memory in language-learning contexts (Ludke, Ferreira & Overy, 2013;Tegge, 2015, and references therein). The studies focus on three claims derived from the author's general theory (dubbed the Musical Foreground Hypothesis): (i) singing can facilitate the processing of language, with beneficial effects in particular on intelligibility and recall; (ii) these effects can be enhanced by song accompaniments; and (iii) music can add meaning to a song by increasing the emotional impact and meaningfulness of the lyrics. ...
... Lin & Zabrucky, 1998, on reading comprehension;Rozenblit & Keil, 2002, on the more general phenomenon). Asking participants to rate the lyrical texts on the extent to which they are "intelligible" ("goedtebegrijpen") and "comprehensible" ("goedteverstaan") is therefore likely to yield less reliable estimates of their cognitive performance, the outcome of interest, than behavioral tasks such as word identification and text recall (as indeed the author acknowledges), and it is a pity that the only such task that produced usable results was a text recall task in Study 2 (reported in Schotanus et al, 2018). ...
... In Study 1, separate factor analyses were performed over five groups of questions probing distinct domains that were asked in all the same song conditions ("Processing fluency", "Voice", "Lyrics", "Emotion", and "Repetition"). 2. In Study 2, factor analyses were performed over the same two groups of questions probing emotional content and repetition ("Emotion" and "Repetition") as in Study 1, while a single separate factor analysis (reported in more detail in Schotanus et al, 2018) was performed over (most of) the questions asked in the three song conditions where there was text and music (accompanied speech, a cappella, and complete). ...
Article
Full-text available
In this commentary, a number of problematic aspects of the studies presented in the target article are discussed. Suggestions have been made for further analyses of some of the data and for additional experimental investigations.
... They may also try to enrich one affect with another, to deneutralize neutral words, or simply to create prosodic accents signalling the relative importance of specific words or highlighting their ambiguity. A tonic chord, for example, seems to function as a punctuation mark (Schotanus, Koops, & Reed Edworty, 2018), and out-of-key notes appear to cause an N400 response (a brain potential indicating special interest in the meaning of a word, Schotanus, Eekhof, & Willems, 2018). Accordingly, if they are aligned to ambiguous words, out-of-key notes support ironic, metaphoric, or highly emotional interpretations of the sentences in which these words occur (Schotanus, 2018). ...
Article
Full-text available
In the article "Emotion Painting: Lyric, affect, and musical relationships in a large lead-sheet corpus", Sun and Cuthbert (2017) explored the correlations between affect-carrying lyrics and musical features such as beat strength, pitch height, consonance, and mode. Several musical features did indeed turn out to be highly correlated with the affect of the lyrics. However, correlations between other features, particularly mode-related musical features and lyric affect, were either insignificant or even contradicted previous research. In the current commentary, it is argued that the difference between the musical features that show significant correlations and those that do not is that the former have a local musical effect whereas the latter tend to affect the mood of a whole phrase or piece, and that the way Sun and Cuthbert estimate lyric affect for sentences or song may not be appropriate. Furthermore, a few remarks are made about the way Sun and Cuthbert treat multi-syllable words and about some basic assumptions concerning the relation between music and lyrics in a song. Nevertheless, the authors are praised for their innovative and interesting work, while several alternative and additional analyses (for example with scale-degree qualia and syncopations) are proposed.
Conference Paper
Full-text available
The question how music influences the perception of lyrics is rather complicated. There are indications that music enhances the perception, comprehension and memorization of sung language, but also that it obstructs language perception and comprehension by withdrawing attention from the words, or undermining their meaning. The linguistic concept of foregrounding, might be helpful to understand the paradoxical way in which music seems to affect lyric perception. Foregrounding (the use of stylistic features such as metaphors and parallelisms, etcetera) is supposed to obstruct normal understanding, but, by doing so, to draw attention to the language too. Foregrounding, for example, both slows down reading and increases strikingness and affects ratings. The Musical Foregrounding Hypothesis (MFH) states that matching music to words has a similar effect to language perception as linguistic foregrounding. However, music is far more complex than any stylistic feature. Music consists of components such as rhythm, pitch, harmony, song structure, etcetera, all of which might affect lyric perception independently. This leads to several sub-hypotheses. An MFH-based model for lyric perception explains the relations between the MFH and these sub-hypotheses. Support for some of these sub-hypotheses is found in the existing literature, others should be tested. The MFH offers an interdisciplinary approach to song and to the relationship between language and music, that might be beneficial to science, education, music, advertising and literature.
Article
Full-text available
Music perception builds on expectancy in harmony, melody, and rhythm. Neural responses to the violations of such expectations are observed in event-related potentials (ERPs) measured using electroencephalography. Most previous ERP studies demonstrating sensitivity to musical violations used stimuli that were temporally regular and musically structured, with less-frequent deviant events that differed from a specific expectation in some feature such as pitch, harmony, or rhythm. Here, we asked whether expectancies about Western musical scale are strong enough to elicit ERP deviance components. Specifically, we explored whether pitches inconsistent with an established scale context elicit deviant components even though equally rare pitches that fit into the established context do not, and even when their timing is unpredictable. We used Markov chains to create temporally irregular pseudo-random sequences of notes chosen from one of two diatonic scales. The Markov pitch-transition probabilities resulted in sequences that favored notes within the scale, but that lacked clear melodic, harmonic, or rhythmic structure. At the random positions, the sequence contained probe tones that were either within the established scale or were out of key. Our subjects ignored the note sequences, watching a self-selected silent movie with subtitles. Compared to the in-key probes, the out-of-key probes elicited a significantly larger P2 ERP component. Results show that random note sequences establish expectations of the "first-order" statistical property of musical key, even in listeners not actively monitoring the sequences.
Article
Full-text available
Maximum likelihood or restricted maximum likelihood (REML) estimates of the parameters in linear mixed-effects models can be determined using the lmer function in the lme4 package for R. As for most model-fitting functions in R, the model is described in an lmer call by a formula, in this case including both fixed- and random-effects terms. The formula and data together determine a numerical representation of the model from which the profiled deviance or the profiled REML criterion can be evaluated as a function of some of the model parameters. The appropriate criterion is optimized, using one of the constrained optimization functions in R, to provide the parameter estimates. We describe the structure of the model, the steps in evaluating the profiled deviance or REML criterion, and the structure of classes or types that represents such a model. Sufficient detail is included to allow specialization of these structures by users who wish to write functions to fit specialized linear mixed models, such as models incorporating pedigrees or smoothing splines, that are not easily expressible in the formula language used by lmer.
Article
Full-text available
Musical skills and expertise vary greatly in Western societies. Individuals can differ in their repertoire of musical behaviours as well as in the level of skill they display for any single musical behaviour. The types of musical behaviours we refer to here are broad, ranging from performance on an instrument and listening expertise, to the ability to employ music in functional settings or to communicate about music. In this paper, we first describe the concept of 'musical sophistication' which can be used to describe the multi-faceted nature of musical expertise. Next, we develop a novel measurement instrument, the Goldsmiths Musical Sophistication Index (Gold-MSI) to assess self-reported musical skills and behaviours on multiple dimensions in the general population using a large Internet sample (n = 147,636). Thirdly, we report results from several lab studies, demonstrating that the Gold-MSI possesses good psychometric properties, and that self-reported musical sophistication is associated with performance on two listening tasks. Finally, we identify occupation, occupational status, age, gender, and wealth as the main socio-demographic factors associated with musical sophistication. Results are discussed in terms of theoretical accounts of implicit and statistical music learning and with regard to social conditions of sophisticated musical engagement.
Article
Despite general agreement on shared syntactic resources in music and language, the neuroanatomical underpinnings of this overlap remain largely unexplored. While previous studies mainly considered frontal areas as supramodal grammar processors, the domain-general syntactic role of temporal areas has been so far neglected. Here we capitalized on the excellent spatial and temporal resolution of subdural EEG recordings to co-localize low-level syntactic processes in music and language in the temporal lobe in a within-subject design. We used Brain Surface Current Density mapping to localize and compare neural generators of the early negativities evoked by violations of phrase structure grammar in both music and spoken language. The results show that the processing of syntactic violations relies in both domains on bilateral temporo-fronto-parietal neural networks. We found considerable overlap of these networks in the superior temporal lobe, but also differences in the hemispheric timing and relative weighting of their fronto-temporal constituents. While alluding to the dissimilarity in how shared neural resources may be configured depending on the musical or linguistic nature of the perceived stimulus, the combined data lend support for a co-localization of early musical and linguistic syntax processing in the temporal lobe.
Article
Almost all modern rotation of factor loadings is based on optimizing a criterion, for example, the quartimax criterion for quartimax rotation. Recent advancements in numerical methods have led to general orthogonal and oblique algorithms for optimizing essentially any rotation criterion. All that is required for a specific application is a definition of the criterion and its gradient. The authors present the implementations of gradient projection algorithms, both orthogonal and oblique, as well as a catalogue of rotation criteria and corresponding gradients. Software for these is downloadable and free; a specific version is given for each of the computing environments used most by statisticians. Examples of rotation methods are presented by applying them to a loading matrix from Wehmeyer and Palmer.
Article
The notion that stylistic features of literary texts deautomatize perception is central to a tradition of literary theory from Coleridge through Shklovsky and Mukařovský to Van Peer. Stylistic variations, known as foregrounding, hypothetically prompt defamiliarization, evoke feelings, and prolong reading time. These possibilities were tested in four studies in which segment by segment reading times and ratings were collected from readers of a short story. In each study, foregrounded segments of the story were associated with increased reading times, greater strikingness ratings, and greater affect ratings. Response to foregrounding appeared to be independent of literary competence or experience. Reasons for considering readers' response to foregrounding as a distinctive aspect of interaction with literary texts are discussed.