Literature Review

Resource allocation models of auditory working memory

Article· Literature ReviewinBrain research 1640(Pt B) · February 2016with 321 Reads
DOI: 10.1016/j.brainres.2016.01.044

Do you want to read the rest of this article?

Request full-text
Request Full-text Paper PDF
  • Article
    Auditory short‐term memory (STM) is a fundamental ability to make sense of auditory information as it unfolds over time. Whether separate STM systems exist for different types of auditory information (music and speech, in particular) is a matter of debate. The present paper reviews studies that have investigated both musical and verbal STM in healthy individuals and in participants with neurodevelopmental and neurological disorders. Overall, the results are in favor of only partly shared networks for musical and verbal STM. Evidence for a distinction in STM for the two materials stems from (1) behavioral studies in healthy participants, in particular from the comparison between nonmusicians and musicians; (2) behavioral studies in congenital amusia, where a selective pitch STM deficit is observed; and (3) studies in brain‐damaged patients with cases of double dissociation. In this review we highlight the need for future studies comparing STM for the same perceptual dimension (e.g., pitch) in different materials (e.g., music and speech), as well as for studies aiming at a more insightful characterization of shared and distinct mechanisms for speech and music in the different components of STM, namely encoding, retention, and retrieval.
  • ... The upper tempo limit to beat perception (200 ms) may be determined by the time constant for temporal integration ( Loveless et al., 1996), which is comparable to the duration of auditory short term sensory memory, or ''short auditory store" ( Cowan, 1984). Recent work, however, suggests that temporal memory resources may not be fixed for a discrete number of items but flexibly distributed according to the number of intervals to be encoded in a sequence ( Teki and Griffiths, 2014;Joseph et al., 2016). ...
    Article
    Full-text available
    Music is a curious example of a temporally patterned acoustic stimulus, and a compelling pan-cultural phenomenon. This review strives to bring some insights from decades of music psychology and sensorimotor synchronization (SMS) literature into the mainstream auditory domain, arguing that musical rhythm perception is shaped in important ways by temporal processing mechanisms in the brain. The feature that unites these disparate disciplines is an appreciation of the central importance of timing, sequencing, and anticipation. Perception of musical rhythms relies on an ability to form temporal predictions, a general feature of temporal processing that is equally relevant to auditory scene analysis, pattern detection, and speech perception. By bringing together findings from the music and auditory literature, we hope to inspire researchers to look beyond the conventions of their respective fields and consider the cross-disciplinary implications of studying auditory temporal sequence processing. We begin by highlighting music as an interesting sound stimulus that may provide clues to how temporal patterning in sound drives perception. Next, we review the SMS literature and discuss possible neural substrates for the perception of, and synchronization to, musical beat. We then move away from music to explore the perceptual effects of rhythmic timing in pattern detection, auditory scene analysis, and speech perception. Finally, we review the neurophysiology of general timing processes that may underlie aspects of the perception of rhythmic patterns. We conclude with a brief summary and outlook for future research.
  • ... Several experimental studies have demonstrated these novel hypotheses to be consistent with visual[14][15][16][17][18][19][20]and auditory working memory performance[21][22][23]. In light of these recent findings, the current consensus is that working memory might be best viewed as a limited resource that is flexibly allocated across all items to be stored in memory[8,[24][25][26]. ...
  • ... Rhythm may predispose us to employ discrete categorical strategies for representing time, by emphasizing the relation between sequential intervals, and thereby using a more economical code. Non-rhythmic time sequences, on the other hand, may recruit different neural mechanisms [48]. Perceiving rhythm also leads to phase-dependent facilitation for many aspects of auditory perception and cognition [49]. ...
    Article
    Full-text available
    Space and time appear to play key roles in the way that information is organized in short-term memory (STM). Some argue that they are crucial contexts within which other stored features are embedded, allowing binding of information that belongs together within STM. Here we review recent behavioral, neurophysiological and imaging studies that have sought to investigate the nature of spatial, sequential and duration representations in STM, and how these might break down in disease. Findings from these studies point to an important role of the hippocampus and other medial temporal lobe structures in aspects of STM, challenging conventional accounts of involvement of these regions in only long-term memory.
  • Article
    Full-text available
    In recent years, there has been increasing interest in studying listening effort. Research on listening effort intersects with the development of active theories of speech perception and contributes to the broader endeavor of understanding speech perception within the context of neuroscientific theories of perception, attention, and effort. Due to the multidisciplinary nature of the problem, researchers vary widely in their precise conceptualization of the catch-all term listening effort. Very recent consensus work stresses the relationship between listening effort and the allocation of cognitive resources, providing a conceptual link to current cognitive neuropsychological theories associating effort with the allocation of selective attention. By linking listening effort to attentional effort, we enable the application of a taxonomy of external and internal attention to the characterization of effortful listening. More specifically, we use a vectorial model to decompose the demand causing listening effort into its mutually orthogonal external and internal components and map the relationship between demanded and exerted effort by means of a resource-limiting term that can represent the influence of motivation as well as vigilance and arousal. Due to its quantitative nature and easy graphical interpretation, this model can be applied to a broad range of problems dealing with listening effort. As such, we conclude that the model provides a good starting point for further research on effortful listening within a more differentiated neuropsychological framework.
  • ... Such studies do demonstrate limitations in the number of durations that can be remembered, but give no indication of the precision with which each duration is remembered. Rhythm discrimination may in fact predispose subjects to use discrete categorical strategies for representing time, whereas for non-rhythmic time sequences, different neural mechanisms are thought to be recruited (Grahn and Brett, 2007; Grube et al., 2010; Joseph et al., 2016). It is possible that in our task, durations could either be encoded individually, as absolute time intervals, or as relative times approximating a rhythmic structure. ...
    Article
    Full-text available
    When a sequence is held in working memory, different items are retained with differing fidelity. Here we ask whether a sequence of brief time intervals that must be remembered show recency effects, similar to those observed in verbal and visuospatial working memory. It has been suggested that prioritizing some items over others can be accounted for by a “focus of attention,” maintaining some items in a privileged state. We therefore also investigated whether such benefits are vulnerable to disruption by attention or expectation. Participants listened to sequences of one to five tones, of varying durations (200 ms to 2 s). Subsequently, the length of one of the tones in the sequence had to be reproduced by holding a key. The discrepancy between the reproduced and actual durations quantified the fidelity of memory for auditory durations. Recall precision decreased with the number of items that had to be remembered, and was better for the first and last items of sequences, in line with set-size and serial position effects seen in other modalities. To test whether attentional filtering demands might impair performance, an irrelevant variation in pitch was introduced in some blocks of trials. In those blocks, memory precision was worse for sequences that consisted of only one item, i.e., the smallest memory set-size. Thus, when irrelevant information was present, the benefit of having only one item in memory is attenuated. Finally we examined whether expectation could interfere with memory. On half the trials, the number of items in the upcoming sequence was cued. When the number of items was known in advance, performance was paradoxically worse when the sequence consisted of only one item. Thus the benefit of having only one item to remember is stronger when it is unexpectedly the only item. Our results suggest that similar mechanisms are used to hold auditory time durations in working memory, as for visual or verbal stimuli. Further, solitary items were remembered better when more items were expected, but worse when irrelevant features were present. This suggests that the “privileged” state of one item in memory is particularly volatile and susceptible to interference.
  • Article
    The capacity of serially-ordered auditory-verbal short-term memory (AVSTM) is sensitive to the timing of the material to be stored, and both temporal processing and AVSTM capacity are implicated in the development of language. We developed a novel ?rehearsal-probe? task to investigate the relationship between temporal precision and the capacity to remember serial order. Participants listened to a sub-span sequence of spoken digits and silently rehearsed the items and their timing during an unfilled retention interval. After an unpredictable delay, a tone prompted report of the item being rehearsed at that moment. An initial experiment showed cyclic distributions of item responses over time, with peaks preserving serial order and broad, overlapping tails. The spread of the response distributions increased with additional memory load and correlated negatively with participants? auditory digit spans. A second study replicated the negative correlation and demonstrated its specificity to AVSTM by controlling for differences in visuo-spatial STM and nonverbal IQ. The results are consistent with the idea that a common resource underpins both the temporal precision and capacity of AVSTM. The rehearsal-probe task may provide a valuable tool for investigating links between temporal processing and AVSTM capacity in the context of speech and language abilities. PQJE_1239749_supplemental_material.docx
  • Article
    Full-text available
    Perception of auditory time intervals is critical for accurate comprehension of natural sounds like speech and music. However, the neural substrates and mechanisms underlying the representation of time intervals in working memory are poorly understood. In this study, we investigate the brain bases of working memory for time intervals in rhythmic sequences using functional magnetic resonance imaging. We used a novel behavioral paradigm to investigate time-interval representation in working memory as a function of the temporal jitter and memory load of the sequences containing those time intervals. Human participants were presented with a sequence of intervals and required to reproduce the duration of a particular probed interval. We found that perceptual timing areas including the cerebellum and the striatum were more or less active as a function of increasing and decreasing jitter of the intervals held in working memory respectively whilst the activity of the inferior parietal cortex is modulated as a function of memory load. Additionally, we also analyzed structural correlations between gray and white matter density and behavior and found significant correlations in the cerebellum and the striatum, mirroring the functional results. Our data demonstrate neural substrates of working memory for time intervals and suggest that the cerebellum and the striatum represent core areas for representing temporal information in working memory.
Literature Review
  • Article
    Full-text available
    This work considers bases for working memory for non-verbal sounds. Specifically we address whether sounds are represented as integrated objects or individual features in auditory working memory and whether the representational format influences WM capacity. The experiments used sounds in which two different stimulus features, spectral passband and temporal amplitude modulation rate, could be combined to produce different auditory objects. Participants had to memorize sequences of auditory objects of variable length (1-4 items). They either maintained sequences of whole objects or sequences of individual features until recall for one of the items was tested. Memory recall was more accurate when the objects had to be maintained as a whole compared to the individual features alone. This is due to interference between features of the same object. Additionally a feature extraction cost was associated with maintenance and recall of individual features, when extracted from bound object representations. An interpretation of our findings is that, at some stage of processing, sounds might be stored as objects in WM with features bound into coherent wholes. The results have implications for feature-integration theory in the context of WM in the auditory system.
  • Article
    Full-text available
    Abstract Memory for speech sounds is a key component of models of verbal working memory (WM). But how good is verbal WM? Most investigations assess this using binary report measures to derive a fixed number of items that can be stored. However, recent findings in visual WM have challenged such 'quantized' views by employing measures of recall precision with an analogue response scale. WM for speech sounds might rely on both continuous and categorical storage mechanisms. Using a novel speech matching paradigm, we measured WM recall precision for phonemes. Vowel qualities were sampled from a formant space continuum. A probe vowel had to be adjusted to match the vowel quality of a target on a continuous, analogue response scale. Crucially, this provided an index of the variability of a memory representation around its true value, and thus allowed us to estimate how memories were distorted from the original sounds. Memory load affected the quality of speech sound recall in two ways. First, there was a gradual decline in recall precision with increasing number of items, consistent with the view that WM representations of speech sounds become noisier with an increase in the number of items held in memory, just as for vision. Based on multidimensional scaling (MDS), the level of noise appeared to be reflected in distortions of the formant space. Second, as memory load increased, there was evidence of greater clustering of participants' responses around particular vowels. A mixture model captured both continuous and categorical responses, demonstrating a shift from continuous to categorical memory with increasing WM load. This suggests that direct acoustic storage can be used for single items, but when more items must be stored, categorical representations must be used.
  • Article
    Full-text available
    The brain can hold information about multiple objects in working memory. It is not known, however, whether intervals of time can be stored in memory as distinct items. Here, we developed a novel paradigm to examine temporal memory where listeners were required to reproduce the duration of a single probed interval from a sequence of intervals. We demonstrate that memory performance significantly varies as a function of temporal structure (better memory in regular vs. irregular sequences), interval size (better memory for sub- vs. supra-second intervals), and memory load (poor memory for higher load). In contrast memory performance is invariant to attentional cueing. Our data represent the first systematic investigation of temporal memory in sequences that goes beyond previous work based on single intervals. The results support the emerging hypothesis that time intervals are allocated a working memory resource that varies with the amount of other temporal information in a sequence.
  • Article
    Visual working memory stores stimuli from our environment as representations that can be accessed by high-level control processes. This study addresses a longstanding debate in the literature about whether storage limits in visual working memory include a limit to the complexity of discrete items. We examined the issue with a number of change-detection experiments that used complex stimuli that possessed multiple features per stimulus item. We manipulated the number of relevant features of the stimulus objects in order to vary feature load. In all of our experiments, we found that increased feature load led to a reduction in change-detection accuracy. However, we found that feature load alone could not account for the results but that a consideration of the number of relevant objects was also required. This study supports capacity limits for both feature and object storage in visual working memory. (PsycINFO Database Record (c) 2014 APA, all rights reserved).
  • Article
    Full-text available
    Working memory is widely considered to be limited in capacity, holding a fixed, small number of items, such as Miller's 'magical number' seven or Cowan's four. It has recently been proposed that working memory might better be conceptualized as a limited resource that is distributed flexibly among all items to be maintained in memory. According to this view, the quality rather than the quantity of working memory representations determines performance. Here we consider behavioral and emerging neural evidence for this proposal.
  • Article
    Humans share with other animals an ability to measure the passage of physical time and subjectively experience a sense of time passing. Subjective time has hallmark qualities, akin to other senses, which can be accounted for by formal, psychological, and neurobiological models of the internal clock. These include first-order principles, such as changes in clock speed and how temporal memories are stored, and second-order principles, including timescale invariance, multi-sensory integration, rhythmical structure, and attentional time-sharing. Within these principles there are both typical individual differences—influences of emotionality, thought speed, and psychoactive drugs—and atypical differences in individuals affected with certain clinical disorders (e.g., autism, Parkinson’s disease, and schizophrenia). This review summarizes recent behavioral and neurobiological findings and provides a theoretical framework for considering how changes in the properties of the internal clock impact time perception and other psychological domains.
  • Article
    Full-text available
    For decades, researchers have sought to understand the organizing principles of auditory and visual short-term memory (STM). Previous work in audition has suggested that there are independent memory stores for different sound features, but the nature of the representations retained within these stores is currently unclear. Do they retain perceptual features, or do they instead retain representations of the sound's specific acoustic properties? In the present study we addressed this question by measuring listeners' abilities to keep one of three acoustic properties (interaural time difference [ITD], interaural level difference [ILD], or frequency) in memory when the target sound was followed by interfering sounds that varied randomly in one of the same properties. Critically, ITD and ILD evoked the same percept (spatial location), despite being acoustically different and having different physiological correlates, whereas frequency evoked a different percept (pitch). The results showed that listeners found it difficult to remember the percept of spatial location when the interfering tones varied either in ITD or ILD, but not when they varied in frequency. The study demonstrates that percepts are the units of auditory STM, and provides testable predictions for future neuroscientific work on both auditory and visual STM. (PsycINFO Database Record (c) 2013 APA, all rights reserved).
  • Article
    Humans share with other animals an ability to measure the passage of physical time and subjectively experience a sense of time passing. Subjective time has hallmark qualities, akin to other senses, which can be accounted for by formal, psychological, and neurobiological models of the internal clock. These include first-order principles, such as changes in clock speed and how temporal memories are stored, and second-order principles, including timescale invariance, multisensory integration, rhythmical structure, and attentional time-sharing. Within these principles there are both typical individual differences-influences of emotionality, thought speed, and psychoactive drugs-and atypical differences in individuals affected with certain clinical disorders (e.g., autism, Parkinson's disease, and schizophrenia). This review summarizes recent behavioral and neurobiological findings and provides a theoretical framework for considering how changes in the properties of the internal clock impact time perception and other psychological domains. Expected final online publication date for the Annual Review of Psychology Volume 65 is January 03, 2014. Please see http://www.annualreviews.org/catalog/pubdates.aspx for revised estimates.
  • Article
    Full-text available
    Some prominent studies have claimed that the medial temporal lobe is not involved in retention of information over brief intervals of just a few seconds. However, in the last decade several investigations have reported that patients with medial temporal lobe damage exhibit an abnormally large number of errors when required to remember visual information over brief intervals. But the nature of the deficit and the type of error associated with medial temporal lobe lesions remains to be fully established. Voltage-gated potassium channel complex antibody-associated limbic encephalitis has recently been recognized as a form of treatable autoimmune encephalitis, frequently associated with imaging changes in the medial temporal lobe. Here, we tested a group of these patients using two newly developed visual short-term memory tasks with a sensitive, continuous measure of report. These tests enabled us to study the nature of reporting errors, rather than only their frequency. On both paradigms, voltage-gated po