No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Resolving the associative learning paradox by category learning in pigeons
Abstract
A wealth of evidence indicates that humans can engage two types of mechanisms to solve category-learning tasks: declarative mechanisms, which involve forming and testing verbalizable decision rules, and associative mechanisms, which involve gradually linking stimuli to appropriate behavioral responses.1,2,3 In contrast to declarative mechanisms, associative mechanisms have received surprisingly little attention in the broader category-learning literature. Although various forms of associatively driven artificial intelligence (AI) have matched-and even surpassed-humans' performance on several challenging problems,3,4,5,6 associative learning is routinely dismissed as being too simple to power the impressive cognitive achievements of both humans and non-human species.6,7,8,9 Here, we attempt to resolve this paradox by demonstrating that pigeons-which appear to rely solely on associative learning mechanisms in several tasks that promote declarative rule use by humans3,10,11,12-succeed at learning a novel, highly demanding category structure that ought to hinder declarative rule use: the sectioned-rings task. Our findings highlight the power and flexibility that associative mechanisms afford in the realm of category learning.
... In a recent study, Wasserman, Kain, and O'Donoghue (Current Biology, 33(6),1112-1116, 2023) set out to resolve the associative learning paradox by showing that pigeons can solve a complex categorylearning task through associative learning. The present Outlook paperpresents their findings, expands on this paradox, and discusses implications oftheir results. ...
... The link between associative learning and the scientific community can be characterized as a love-andhate relationship, where associative processes are by some considered sufficient as causes for complex behavior, whereas others argue that associative processes are far too simple for generating complex behavior. Wasserman et al. (2023) shine a light on these issues in a study on category learning in pigeons. They contend that in the category-learning literature, associative learning is too often left out, and has been considered too simple to generate complex category learning. ...
... Building from this, they devised a complicated categorization task and tested if associative learning alone was sufficient for category learning of novel complex categorical structures. They concluded that it was (Wasserman et al., 2023). ...
In a recent study, Wasserman, Kain, and O'Donoghue (Current Biology, 33(6), 1112-1116, 2023) set out to resolve the associative learning paradox by showing that pigeons can solve a complex category learning task through associative learning. The present Outlook paper presents their findings, expands on this paradox, and discusses implications of their results.
... Although pigeons can learn to selectively attend to categoryrelevant features, they do not produce rule-like behaviour and, instead, they learn these selective attention weightings gradually, associatively and non-analytically 141,142,147,154 . A series of experiments with pigeons and humans provides further support for this dissociation: they learned a novel and complex category structure based on concentric rings 155,156 (Fig. 3b). Unlike two-dimensional rule-based and information-integration categories, the concentric rings category structure is non-linear and cannot be solved by abstracting a prototype or comparing the similarity to previous exemplars but only through associative learning. ...
... First, research from cognitive neuroscience, cognitive psychology, developmental psychology and comparative psychology offers strong support for the role of multiple systems in category learning. The core behavioural findings supporting COVIS have been replicated numerous times across many laboratories 18,82,85 and the comparative literature offers compelling evidence for COVIS, showing how brains that differ in structure learn categories in different ways 141,156 . ...
... However, as soon as one of them is difficult to verbalize (in the present study the geometric forms), performance in the test phase of the audiovisual test dips significantly. According to the original paper of Ashby et al. (1998) and more recent studies ( Ashby and Ell 2001;Smith et al. 2012;Wasserman et al. 2023), categories may be learned in both humans and animals in a verbal (explicit) or nonverbal (implicit) way. These studies make it clear that the difficulty of the verbalizability of the rule of categorization is intimately related to the verbalizability of stimulus features: verbalizable rules are formed easily if the stimuli are easy to describe verbally. ...
Background
The visually guided Rutgers Acquired Equivalence Test (RAET) and the various visual and audiovisual versions of the test with the same structure involve rule acquisition, retrieval, and generalization and is based on learning stimulus pairs (antecedents and consequents). In an earlier study we have found no difference in the acquisition learning and only slight enhancement in retrieval and generalization in the audiovisual learning compared to the visual one if complex readily verbalizable visual stimuli (cartoon faces and color fish) were used. In this study, we sought to examine whether similar phenomena can be observed with feature‐restricted, less verbalizable visual stimuli (geometric shapes).
Methods
A total of 119 healthy adult volunteers completed two computer‐based test paradigms: Polygon (PO) and SoundPolygon (SP). PO is a visual test where the antecedents are shaded circles, and the consequents are geometric shapes. SP is an audiovisual test where the antecedents are sounds and the consequents are the same geometric shapes as in PO.
Results
There were no significant differences in the performances and the reaction times in the acquisition phase between the PO (visual) and SP (audiovisual) tests. However, the performances in retrieval and generalization were significantly poorer in the audiovisual test and the reaction times were also longer.
Conclusion
The acquisition phase seems to be independent from the stimulus modality if the simple geometric shapes were visual stimuli. However, feature‐restricted, less verbalizable visual stimuli make more difficult to retrieve and generalize the already acquired audiovisual information.
... Non-human animals communicate in ways that defy, or at least cast serious doubt, on purely mechanistic explanations (e.g., audience effects: Doutrelant et al., 2001;Dzieweczynski et al., 2005;Evans & Marler, 1994;Le Roux et al., 2008, Townsend & Zuberbühler, 2009content attribution: Ostojic et al., 2013;Price & Fischer 2014; concept formation: Hare & Atkins, 2001;Thompson & Oden, 2000;Wasserman, 1995;Wasserman et al. 2023;Zuberbühler et al., 1999;and controlled signal production: Miller et al., 2009;Snowdon, 1998). Exchangeability provides a simple mechanism via which individuals can process information about a situation, and relate that to similar situations recovered from memories stored in the context of their past participation as a signaler or receiver in a communicative context (Bruni, 2008;Francescoli, 2021;Prather et al., 2008Prather et al., , 2009Reznikova, 2007). ...
Signals that inform prospective receivers of potential contingencies associated with the signaler or its environment may be innate, or may rely upon repeated association between signal production and a context relevant to the receiver, or theory of mind, such that signalers and/or receivers infer the state or contextual situation communicative partners find themselves in. While theoretical discourse on the information content of signals has focused on the coevolution of signalers and receivers as distinct entities, the fact that individuals often act both as receivers and signalers in similar communicative contexts over the course of their lives presents the possibility that signals emitted in a given context may reflect the signaler's own experience as a receiver in the same or similar contexts. This process, which we term "Exchangeability," would allow signalers to encode appropriate information in signals and receivers to extract meaningful information without conditioning or recourse to theory of mind. We propose that Exchangeability readily accounts for the expression of true communication, where both signalers and receivers benefit, eavesdropping, where receivers alone benefit, and manipulation, where signalers benefit at the expense of receivers, thus providing a previously overlooked mechanism via which information can be encoded and decoded in animal communicative exchanges.
... Associative learning develops cognitive functions that are more evolved: it is based on the consequences of the existence of relationships between separate stimuli resulting in a particular behaviour. The stimuli may range from concrete objects and events to abstract concepts, such as time, location, context or even categories [see references in a thorough analysis of category learning in pigeons, with the comparison with a variety of AI approaches (Wasserman et al., 2023)]. All these techniques are obviously promising for the prediction of epidemics as they are able to associate a non-limited variety of datasets, including knowledge of the pathogen and of the human population structure and behaviour. ...
The emergence of new techniques in both microbial biotechnology and artificial intelligence (AI) is opening up a completely new field for monitoring and sometimes even controlling the evolution of pathogens. However, the now famous generative AI extracts and reorganizes prior knowledge from large datasets, making it poorly suited to making predictions in an unreliable future. In contrast, an unfamiliar perspective can help us identify key issues related to the emergence of new technologies, such as those arising from synthetic biology, whilst revisiting old views of AI or including generative AI as a generator of abduction as a resource. This could enable us to identify dangerous situations that are bound to emerge in the not‐too‐distant future, and prepare ourselves to anticipate when and where they will occur. Here, we emphasize the fact that amongst the many causes of pathogen outbreaks, often driven by the explosion of the human population, laboratory accidents are a major cause of epidemics. This review, limited to animal pathogens, concludes with a discussion of potential epidemic origins based on unusual organisms or associations of organisms that have rarely been highlighted or studied.
... Associative learning is a fundamental mechanism that holds strong explanatory power for general learning in both humans and other animals (Pavlov 1949;Skinner 1965;Mackintosh 1983;Rescorla and Wagner 1972;Heyes 2018;Wasserman, Kain, and O'Donoghue 2023;Lind 2018;Heyes 2012bHeyes , 2012aEnquist, Lind, and Ghirlanda 2016;Bouton 2016;Haselgrove 2016;Enquist, Ghirlanda, and Lind 2023). However, the inability of non-human animals to match the language capacities of humans calls for identifying unique properties that are present in humans, and not in other animals. ...
This study explores the cognitive mechanisms underlying human language acquisition through grammar induction by a minimal cognitive architecture, with a short and flexible sequence memory as its most central feature. We use reinforcement learning for the task of identifying sentences in a stream of words from artificial languages. Results demonstrate the model’s ability to identify frequent and informative multi-word chunks, reproducing characteristics of natural language acquisition. The model successfully navigates varying degrees of linguistic complexity, exposing efficient adaptation to combinatorial challenges through the reuse of sequential patterns. The emergence of parsimonious tree structures suggests an optimization for the sentence identification task, balancing economy and information. The cognitive architecture reflects aspects of human memory systems and decision-making processes, enhancing its cognitive plausibility. While the model exhibits limitations in generalization and semantic representation, its minimalist nature offers insights into some fundamental mechanisms of language learning. Our study demonstrates the power of this simple architecture and stresses the importance of sequence memory in language learning. Since other animals do not seem to have faithful sequence memory, this may be a key to understanding why only humans have developed complex languages.
... The second task we studied involved stimuli which were similar to the concentric-rings task, but which cut each training ring into 4 sections. 12 Figures 3E and G depict the complete stimulus distributions possible in the "+ cut" and the "× cut" variants of the "sectioned-rings" task, respectively, where Category A is shown in red and Category B is shown in blue. Like the concentric-rings task, this task thwarts the efforts of a learner to use a simple, unidimensional rule. ...
Over the past 30 years, behavioral, computational, and neuroscientific investigations have yielded fresh insights into how pigeons adapt to the diverse complexities of their visual world. A prime area of interest has been how pigeons categorize the innumerable individual stimuli they encounter. Most studies involve either photorealistic representations of actual objects thus affording the virtue of being naturalistic, or highly artificial stimuli thus affording the virtue of being experimentally manipulable. Together those studies have revealed the pigeon to be a prodigious classifier of both naturalistic and artificial visual stimuli. In each case, new computational models suggest that elementary associative learning lies at the root of the pigeon’s category learning and generalization. In addition, ongoing computational and neuroscientific investigations suggest how naturalistic and artificial stimuli may be processed along the pigeon’s visual pathway. Given the pigeon’s availability and affordability, there are compelling reasons for this animal model to gain increasing prominence in contemporary neuroscientific research.
... Turing himself proposed: "Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child's?" (1950: 456) But also, in media reports dealing with scientific developments in AI, we regularly come across headlines in the manner of "AI had IQ of four-year-old child" (BBC 2015). For the case of animals, a good example would be the recently published study by Wasserman, Kain and O'Donoghue (2023), which deals with the learning mechanisms of pigeons that are said to bear significant similarities with the type of learning of machine learning algorithms, particularly reinforcement learning. The authors point to BF Skinner's planned usage of pigeons as 'brains' for his experimental guidance system for directing ballistic missiles to possible WWII military targets. ...
... These studies show that birds and mammals operate with highly similar mental processes when facing a cognitive task, but this does not imply that all mammal species or all bird species are identical with respect to the same levels of cognitive performance. Although both pigeons [27] Trends Trends in in Cognitive Cognitive Sciences Sciences [13]. Picture credit by Barbara Klump. ...
Many cognitive neuroscientists believe that both a large brain and an isocortex are crucial for complex cognition. Yet corvids and parrots possess non-cortical brains of just 1–25 g, and these birds exhibit cognitive abilities comparable with those of great apes such as chimpanzees, which have brains of about 400 g. This opinion explores how this cognitive equivalence is possible. We propose four features that may be required for complex cognition: a large number of associative pallial neurons, a prefrontal cortex (PFC)-like area, a dense dopaminergic innervation of association areas, and dynamic neurophysiological fundaments for working memory. These four neural features have convergently evolved and may therefore represent ‘hard to replace’ mechanisms enabling complex cognition.
... Turing himself proposed: "Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child's?" (1950: 456) But also, in media reports dealing with scientific developments in AI, we regularly come across headlines in the manner of "AI had IQ of four-year-old child" (BBC 2015). For the case of animals, a good example would be the recently published study by Wasserman, Kain and O'Donoghue (2023), which deals with the learning mechanisms of pigeons that are said to bear significant similarities with the type of learning of machine learning algorithms, particularly reinforcement learning. The authors point to BF Skinner's planned usage of pigeons as 'brains' for his experimental guidance system for directing ballistic missiles to possible WWII military targets. ...
How do artificial neural networks and other forms of artificial intelligence interfere with methods and practices in the sciences? Which interdisciplinary epistemological challenges arise when we think about the use of AI beyond its dependency on big data? Not only the natural sciences, but also the social sciences and the humanities seem to be increasingly affected by current approaches of subsymbolic AI, which master problems of quality (fuzziness, uncertainty) in a hitherto unknown way. But what are the conditions, implications, and effects of these (potential) epistemic transformations and how must research on AI be configured to address them adequately?
... The final task we studied used stimuli similar to the concentric-rings task, but which subdivided each ring into four segments, as reported in Wasserman et al. 27 Figures 9A and 9C depict the stimulus distributions used in the ''+ cut'' and ''3 cut'' conditions, respectively, where Category A is shown in red and Category B is shown in blue. Like the concentric-rings task, the sectioned-rings task thwarts any effort of the learner to use a simple, unidimensional rule; but the sectioned-rings task eliminates the possibility of a complex rule that would allow a learner to divide the stimulus space into an interior category and an ''other'' category. ...
Never known for its smarts, the pigeon has proven to be a prodigious classifier of complex visual stimuli. What explains its surprising success? Does it possess elaborate executive functions akin to those deployed by humans? Or does it effectively deploy an unheralded, but powerful associative learning mechanism? In a series of experiments, we first confirm that pigeons can learn a variety of category structures – some devised to foil the use of advanced cognitive processes. We then contrive a simple associative learning model to see how effectively the model learns the same tasks given to pigeons. The close fit of the associative model to pigeons’ categorization behavior provides unprecedented support for associative learning as a viable mechanism for mastering complex category structures and for the pigeon’s using this mechanism to adapt to a rich visual world. This model will help guide future neuroscientific research into the biological substrates of visual cognition.
... Although once thought to be a uniquely human quality, the formation and application of abstract concepts has now been assessed across multiple taxa 13 . Abstract concept learning is considered to be a 'higher' level of cognitive functioning 14 , but recent work by Wasserman et al. 15 shows that associative learning is capable of performing complicated discrimination feats. ...
concept formation is a cognitive skill that nonhuman animals have been shown to possess. Most often, this ability has been shown in laboratory tasks; a new study sheds light on what role abstract concept formation may play in the wild.
Category learning gives rise to category formation, which is a crucial ability in human cognition. Category learning is also one of the required learning abilities in language development. Understanding the evolution of category learning thus can shed light on the evolution of human cognition and language. The current paper emphasizes its foundational role in language evolution by reviewing behavioral and neurological studies on category learning across species. In doing so, we first review studies on the critical role of category learning in learning sounds, words, and grammatical patterns of language. Next, from a comparative perspective, we review studies on category learning conducted on different species of nonhuman animals, including invertebrates and vertebrates, suggesting that category learning displays evolutionary continuity. Then, from a neurological perspective, we focus on the prefrontal cortex and the basal ganglia. Reviewing the involvement of these structures in vertebrates and the proposed homologous brain structure to the basal ganglia in invertebrates in category learning, as well as in language processing in humans, suggests that the neural basis of category learning likely has an ancient origin dating back to invertebrates. With evidence from both behavioral and neurological levels in both nonhuman animals and humans, we conclude that category learning lays a crucial cognitive foundation for language evolution.
Color vision is an important perceptual ability in most species and a crucial capacity underlying any cognitive task working with color stimuli. Birds are known for their outstanding vision and tetrachromacy. Two jackdaws were trained to indicate whether they perceive two colors as same or different. The dominant wavelengths of the experimental colors were assessed to relate the birds’ performance to the physical qualities of the stimuli. The results indicate that the differences or similarities in dominant wavelengths of the colors had a strong influence on the behavioral data. Colors related to a reduced discriminatory performance were colors of particularly close wavelengths, whereas differences in saturation or brightness were less relevant. Overall, jackdaws mostly relied on hue to discriminate color pairs, and their behavior strongly reflected the physical composition of the color set. These findings show that when working with color stimuli, not only the perceptual abilities of the particular species, but also the technical aspects concerning the color presentation have to be considered carefully.
Pigeons and doves (family Columbidae) are one of the most diverse extant avian lineages, and many species have served as key models for evolutionary genomics, developmental biology, physiology, and behavioral studies. Building genomic resources for colubids is essential to further many of these studies. Here, we present high-quality genome assemblies and annotations for two columbid species, Columba livia and C. guinea. We simultaneously assembled C. livia and C. guinea genomes from long-read sequencing of a single F1 hybrid individual. The new C. livia genome assembly (Cliv_3) shows improved completeness and contiguity relative to Cliv_2.1, with an annotation incorporating long-read IsoSeq data for more accurate gene models. Intensive selective breeding of C. livia has given rise to hundreds of breeds with diverse morphological and behavioral characteristics, and Cliv_3 offers improved tools for mapping the genomic architecture of interesting traits. The C. guinea genome assembly is the first for this species and is a new resource for avian comparative genomics. Together, these assemblies and annotations provide improved resources for functional studies of columbids and avian comparative genomics in general.
Pigeons and doves (family Columbidae) are one of the most diverse extant avian lineages, and many species have served as key models for evolutionary genomics, developmental biology, physiology, and behavioral studies. Building genomic resources for colubids is essential to further many of these studies. Here, we present high-quality genome assemblies and annotations for two columbid species, Columba livia and C. guinea . We simultaneously assembled C. livia and C. guinea genomes from long-read sequencing of a single F 1 hybrid individual. The new C. livia genome assembly (Cliv_3) shows improved completeness and contiguity relative to Cliv_2.1, with an annotation incorporating long-read IsoSeq data for more accurate gene models. Intensive selective breeding of C. livia has given rise to hundreds of breeds with diverse morphological and behavioral characteristics, and Cliv_3 offers improved tools for mapping the genomic architecture of interesting traits. The C. guinea genome assembly is the first for this species and is a new resource for avian comparative genomics. Together, these assemblies and annotations provide improved resources for functional studies of columbids and avian comparative genomics in general.
ARTICLE SUMMARY
Pigeons and doves are important models for evolutionary genomics, developmental biology, physiology, and behavioral studies. Here, we present high-quality reference genome assemblies and annotations for two pigeon species, the domestic rock pigeon ( Columba livia ) and the African speckled pigeon ( C. guinea ). These assemblies and annotations provide improved resources for both comparative genomics and functional studies.
Humans can learn tasks explicitly, as they can often describe the rules they have used to learn the task.1,2,3 Animals, however, are thought to learn tasks implicitly (i.e., purely associatively).2,3 That is, they gradually learn the correlation or association between the stimulus (or response) and the outcome. Both humans and pigeons can learn matching, where a sample stimulus indicates which one of two stimuli matches the sample. The 1-back reinforcement task is a difficult version of matching in which a correct response on trial N is rewarded only following a response on trial N + 1 (independent of the response on trial N + 1),4 and the correct response on trial N + 1 indicates whether a reward will occur on trial N + 2, and so forth. Humans do not appear to be able to learn the 1-back rule.5 Pigeons, however, do show 1-back reinforcement learning,6,7 and they appear to do so implicitly by gradually learning the correlation between their response on one trial and the outcome on the next trial (because all other relations are uncorrelated with the outcome). They learn the task slowly and to a level below what would be expected had they learned it explicitly. The present results, together with research with humans,7 suggest that there are times when human explicit learning may interfere with the ability of humans to learn. Pigeons, however, are not "distracted" by attempts at explicit learning, and thus they are able to learn this and other similar tasks.6,7,8.
A neuropsychological theory is proposed that assumes category learning is a competition between separate verbal and implicit (i.e., procedural-learning-based) categorization systems. The theory assumes that the caudate nucleus is an important component of the implicit system and that the anterior cingulate and prefrontal cortices are critical to the verbal system. In addition to making predictions for normal human adults, the theory makes specific predictions for children, elderly people, and patients suffering from Parkinson's disease, Huntington's disease, major depression, amnesia, or lesions of the prefrontal cortex. Two separate formal descriptions of the theory are also provided. One describes trial-by-trial learning, and the other describes global dynamics. The theory is tested on published neuropsychological data and on category learning data with normal adults.
Non-human animals tend to solve behavioral tasks using local information. Pigeons are particularly biased toward using the local features of stimuli to guide behavior in small-scale environments. When behavioral tasks are performed in large-scale environments, pigeons are much better global processors of information. The local and global strategies are mediated by two different fovea in the pigeon retina that are associated with the tectofugal and thalamofugal pathways. We discuss the neural mechanisms of pigeons’ bias for local information within the tectofugal pathway, which terminates at an intermediate stage of extracting shape complexity. We also review the evidence suggesting that the thalamofugal pathway participates in global processing in pigeons and is primarily engaged in constructing a spatial representation of the environment in conjunction with the hippocampus.
The associative learning theory of Robert Rescorla and Allan Wagner has been duly celebrated for its 50-year reign as the predominant model in learning science. One special recognition is warranted: its close correspondence with David Hume’s associative theory of causality judgment. Hume’s rules by which causes come to suggest effects are not only embraced by the Rescorla-Wagner model, but their mechanistic account makes precise quantitative predictions that can be assessed by empirical evidence rather than by speculation and argumentation. Framed in this way, the Rescorla-Wagner model truly represents the scientific culmination of Hume’s philosophical theory of causation.
Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block opponents while operating their vehicles at their traction limits¹. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the non-linear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world’s best e-sports drivers. We combine state-of-the-art, model-free, deep reinforcement learning algorithms with mixed-scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing’s important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world’s best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.
Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.
A prominent model of categorization (Ashby, Alfonso-Reese, Turken, & Waldron, 1998) posits that 2 separate mechanisms-one declarative, one associative-can be recruited in category learning. These 2 systems can effectively be distinguished by 2 task structures: rule-based (RB) tasks are unidimensional and encourage analytic processing, whereas information-integration (II) tasks are bidimensional and encourage nonanalytic associative learning. Humans and nonhuman primates have been reported to learn RB tasks more quickly than II tasks; however, pigeons and rats have shown no learning speed differences are thus believed to lack the declarative system. In the present trio of experiments, we further explored pigeons' dimensional category learning. We replicated the finding that pigeons learn RB and II tasks at equal speeds. Further, we found that stimulus generalization performance was equivalent on both tasks. We also explored the effect of switching from one task to another. Task switches between phases of training as well as within individual training sessions posed little difficulty for pigeons; they quickly and flexibly switched their categorization responses with no cost in choice speed or accuracy. Together, our data indicate that, although pigeons may lack the capacity to form explicit dimensional rules, their associative learning system is both powerful and flexible. Further exploration of this associative system would help us better appreciate possible contributions of the declarative system. (PsycINFO Database Record (c) 2020 APA, all rights reserved).
Most accounts of behavior in nonhuman animals assume that they make choices to maximize expected reward value. However, model-free reinforcement learning based on reward associations cannot account for choice behavior in transitive inference paradigms. We manipulated the amount of reward associated with each item of an ordered list, so that maximizing expected reward value was always in conflict with decision rules based on the implicit list order. Under such a schedule, model-free reinforcement algorithms cannot achieve high levels of accuracy, even after extensive training. Monkeys nevertheless learned to make correct rule-based choices. These results show that monkeys’ performance in transitive inference paradigms is not driven solely by expected reward and that appropriate inferences are made despite discordant reward incentives. We show that their choices can be explained by an abstract, model-based representation of list order, and we provide a method for inferring the contents of such representations from observed data.
In a seminal study, Shepard, Hovland, and Jenkins (1961; henceforth SHJ) assessed potential mechanisms involved in categorization learning. To do so, they sequentially trained human participants with 6 different visual categorization tasks that varied in structural complexity. Humans' exceptionally strong performance on 1 of these tasks (Type 2, organized around exclusive-or relations) could not be solely explained by structural complexity, and has since been considered the hallmark of rule-use in these tasks. In the present project, we concurrently trained pigeons on all 6 SHJ tasks. Our results revealed that the structural complexity of the tasks was highly correlated with group-level performance. Nevertheless, we observed notable individual differences in performance. Two extensions of a prominent categorization model, ALCOVE (Kruschke, 1992), suggested that disparities in the discriminability of the dimensions used to construct the experimental stimuli could account for these differences. Overall, our pigeons' generally weak performance on the Type 2 task provides no evidence of rule-use on the SHJ tasks. Pigeons thus join monkeys in the contingent of species that solve these categorization tasks solely on the basis of the physical properties of the training stimuli. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
There is a new associative learning paradox. The power of associative learning for producing flexible behaviour in non-human animals is downplayed or ignored by researchers in animal cognition, whereas artificial intelligence research shows that associative learning models can beat humans in chess. One phenomenon in which associative learning often is ruled out as an explanation for animal behaviour is flexible planning. However, planning studies have been criticized and questions have been raised regarding both methodological validity and interpretations of results. Due to the power of associative learning and the uncertainty of what causes planning behaviour in non-human animals, I explored what associative learning can do for planning. A previously published sequence learning model which combines Pavlovian and instrumental conditioning was used to simulate two planning studies, namely Mulcahy & Call 2006 'Apes save tools for future use.' Science 312, 1038-1040 and Kabadayi & Osvath 2017 'Ravens parallel great apes in flexible planning for tool-use and bartering.' Science 357, 202-204. Simulations show that behaviour matching current definitions of flexible planning can emerge through associative learning. Through conditioned reinforcement, the learning model gives rise to planning behaviour by learning that a behaviour towards a current stimulus will produce high value food at a later stage; it can make decisions about future states not within current sensory scope. The simulations tracked key patterns both between and within studies. It is concluded that one cannot rule out that these studies of flexible planning in apes and corvids can be completely accounted for by associative learning. Future empirical studies of flexible planning in non-human animals can benefit from theoretical developments within artificial intelligence and animal learning.
Exemplar theory assumes that people categorize a novel object by comparing its similarity to the memory representations of all previous exemplars from each relevant category. Exemplar theory has been the most prominent cognitive theory of categorization for more than 30 years. Despite its considerable success in providing good quantitative fits to a wide variety of accuracy data, it has never had a detailed neurobiological interpretation. This article proposes a neural interpretation of exemplar theory in which category learning is mediated by synaptic plasticity at cortical-striatal synapses. In this model, categorization training does not create new memory representations, rather it alters connectivity between striatal neurons and neurons in sensory association cortex. The new model makes identical quantitative predictions as exemplar theory, yet it can account for many empirical phenomena that are either incompatible with or outside the scope of the cognitive version of exemplar theory.
Retrospective revaluation refers to an increase (or decrease) in responding to conditioned stimulus (CS X) as a result of decreasing (or increasing) the associative strength of another CS (A) with respect to the unconditioned stimulus (i.e., A-US) that was previously trained in compound with the target CS (e.g., AX-US or just AX). We discuss the conditions under which retrospective revaluation phenomena are most apt to be observed and their implications for various models of learning that are able to account for retrospective revaluation (e.g., Dickinson and Burke, 1996; Miller and Matzel, 1988; Van Hamme and Wasserman, 1994). Although retroactive revaluation is relatively parameter specific, it is seen to be a reliable phenomenon observed across many tasks and species. As it is not anticipated by many conventional models of learning (e.g., Rescorla and Wagner, 1972), it serves as a critical benchmark for evaluating traditional and newer models.
Copyright © 2015. Published by Elsevier B.V.
Humans can spontaneously create rules that allow them to efficiently generalize what they have learned to novel situations. An enduring question is whether rule-based generalization is uniquely human or whether other animals can also abstract rules and apply them to novel situations. In recent years, there have been a number of high-profile claims that animals such as rats can learn rules. Most of those claims are quite weak because it is possible to demonstrate that simple associative systems (which do not learn rules) can account for the behavior in those tasks. Using a procedure that allows us to clearly distinguish feature-based from rule-based generalization (the Shanks–Darby procedure), we demonstrate that adult humans show rule-based generalization in this task, while generalization in rats and pigeons was based on featural overlap between stimuli. In brief, when learning that a stimulus made of two components (“AB”) predicts a different outcome than its elements (“A” and “B”), people spontaneously abstract an opposites rule and apply it to new stimuli (e.g., knowing that “C” and “D” predict one outcome, they will predict that “CD” predicts the opposite outcome). Rats and pigeons show the reverse behavior—they generalize what they have learned, but on the basis of similarity (e.g., “CD” is similar to “C” and “D”, so the same outcome is predicted for the compound stimulus as for the components). Genuinely rule-based behavior is observed in humans, but not in rats and pigeons, in the current procedure.
Electronic supplementary material
The online version of this article (doi:10.1007/s10071-015-0895-8) contains supplementary material, which is available to authorized users.
The study of the mechanism that detects the contingency between events. in both humans and non-human animals, is a matter of considerable research activity. Two broad categories of explanations of the acquisition of contingency information have received extensive evaluation: rule-based models and associative models. This article assesses the two categories of models for human contingency judgments. The data reveal systematic departures in contingency judgments from the predictions of rule-based models. Recent studies indicate that a contiguity model of Pavlovian conditioning is a useful heuristic for conceptualizing human contingency judgments.
Classic approaches to word learning emphasize referential ambiguity: In naming situations, a novel word could refer to many possible objects, properties, actions, and so forth. To solve this, researchers have posited constraints, and inference strategies, but assume that determining the referent of a novel word is isomorphic to learning. We present an alternative in which referent selection is an online process and independent of long-term learning. We illustrate this theoretical approach with a dynamic associative model in which referent selection emerges from real-time competition between referents and learning is associative (Hebbian). This model accounts for a range of findings including the differences in expressive and receptive vocabulary, cross-situational learning under high degrees of ambiguity, accelerating (vocabulary explosion) and decelerating (power law) learning, fast mapping by mutual exclusivity (and differences in bilinguals), improvements in familiar word recognition with development, and correlations between speed of processing and learning. Together it suggests that (a) association learning buttressed by dynamic competition can account for much of the literature; (b) familiar word recognition is subserved by the same processes that identify the referents of novel words (fast mapping); (c) online competition may allow the children to leverage information available in the task to augment performance despite slow learning; (d) in complex systems, associative learning is highly multifaceted; and (e) learning and referent selection, though logically distinct, can be subtly related. It suggests more sophisticated ways of describing the interaction between situation- and developmental-time processes and points to the need for considering such interactions as a primary determinant of development. (PsycINFO Database Record (c) 2012 APA, all rights reserved).
"This is the history of a crackpot idea, born on the wrong side of the tracks intellectually speaking, but eventually vindicated in a sort of middle class respectability. It is the story of a proposal to use living organisms to guide missiles—of a research program during World War II called 'Project Pigeon' and a peace-time continuation at the Naval Research Laboratory called 'ORCON,' from the words 'organic Control.' " Major sections are: Project Pelican, Orcon, and The Crackpot Idea. Wide-ranging speculation about human affairs, "supported by studies of compensating rigor, will make a substantial contribution toward that world of the future in which… there will be no need for guided missiles." (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Cognitive ethologists are in need of a good theoretical framework for attributing intentional states. Heyes and Dickinson (1990) present criteria that they claim are necessary for an intentional explanation of behavior to be justified. They suggest that questions of intentionality can only be investigated under controlled laboratory conditions and they apply their criteria to laboratory experiments to argue that the common behavior of approaching food is not intentional in most animals. We dispute the details of their argument and interpretation of the laboratory experiments. While criteria such as those suggested have a role to play in comparative studies of cognition, both laboratory and field studies are important for assessing the applicability of intentional explanations across different taxa.
A wealth of empirical evidence has now accumulated concerning animals' categorizing photographs of real-world objects. Although these complex stimuli have the advantage of fostering rapid category learning, they are difficult to manipulate experimentally and to represent in formal models of behavior. We present a solution to the representation problem in modeling natural categorization by adopting a common-elements approach. A common-elements stimulus representation, in conjunction with an error-driven learning rule, can explain a wide range of experimental outcomes in animals' categorization of naturalistic images. The model also generates novel predictions that can be empirically tested. We report 2 experiments that show how entirely hypothetical representational elements can nevertheless be subject to experimental manipulation. The results represent the first evidence of error-driven learning in natural image categorization, and they support the idea that basic associative processes underlie this important form of animal cognition.
A unified quantitative approach to modeling subjects' identification and categorization of multidimensional perceptual stimuli is proposed and tested. Two subjects identified and categorized the same set of perceptually confusable stimuli varying on separable dimensions. The identification data were modeled using Shepard's (1957) multidimensional scaling-choice framework. This framework was then extended to model the subjects' categorization performance. The categorization model, which generalizes the context theory of classification developed by Medin and Schaffer (1978), assumes that subjects store category exemplars in memory. Classification decisions are based on the similarity of stimuli to the stored exemplars. It is assumed that the same multidimensional perceptual representation underlies performance in both the identification and categorization paradigms. However, because of the influence of selective attention, similarity relationships change systematically across the two paradigms. Some support was gained for the hypothesis that subjects distribute attention among component dimensions so as to optimize categorization performance. Evidence was also obtained that subjects may have augmented their category representations with inferred exemplars. Implications of the results for theories of multidimensional scaling and categorization are discussed.
Planning for future needs, not just current ones, is one of the most formidable human cognitive achievements. Whether this skill is a uniquely human adaptation is a controversial issue. In a study we conducted, bonobos and orangutans selected, transported, and saved appropriate tools above baseline levels to use them 1 hour later (experiment 1). Experiment 2 extended these results to a 14-hour delay between collecting and using the tools. Experiment 3 showed that seeing the apparatus during tool selection was not necessary to succeed. These findings suggest that the precursor skills for planning for the future evolved in great apes before 14 million years ago, when all extant great ape species shared a common ancestor.
COVIS (COmpetition between Verbal and Implicit Systems; Ashby, Alfonso-Reese, & Waldron, 1998) is a prominent model of categorization which hypothesizes that humans have two independent categorization systems – one declarative, one associative – that can be recruited to solve category learning tasks. To date, most COVIS-related research has focused on just two experimental tasks: linear rule-based (RB) tasks, which purportedly encourage declarative rule use, and linear information-integration (II) tasks, which purportedly require associative learning mechanisms. We introduce and investigate a novel alternative: the concentric-rings task, a nonlinear category structure that both humans and pigeons can successfully learn and transfer to untrained exemplars. Yet, despite their broad behavioral similarities, humans and pigeons achieve their successful learning through decidedly different means. As predicted by COVIS, pigeons appear to rely solely on associative learning mechanisms, whereas humans appear to initially test but subsequently reject unidimensional rules. We discuss how variants of our concentric-rings task might yield further insights into which category-learning mechanisms are shared across species, as well as how categorization strategies might change throughout training.
In the last century, learning theory has been dominated by an approach assuming that associations between hypothetical representational nodes can support the acquisition of knowledge about the environment. The similarities between this approach and connectionism did not go unnoticed to learning theorists, with many of them explicitly adopting a neural network approach in the modeling of learning phenomena. Skinner famously criticized such use of hypothetical neural structures for the explanation of behavior (the "Conceptual Nervous System"), and one aspect of his criticism has proven to be correct: theory underdetermination is a pervasive problem in cognitive modeling in general, and in associationist and connectionist models in particular. That is, models implementing two very different cognitive processes often make the exact same behavioral predictions, meaning that important theoretical questions posed by contrasting the two models remain unanswered. We show through several examples that theory underdetermination is common in the learning theory literature, affecting the solvability of some of the most important theoretical problems that have been posed in the last decades. Computational cognitive neuroscience (CCN) offers a solution to this problem, by including neurobiological constraints in computational models of behavior and cognition. Rather than simply being inspired by neural computation, CCN models are built to reflect as much as possible about the actual neural structures thought to underlie a particular behavior. They go beyond the "Conceptual Nervous System" and offer a true integration of behavioral and neural levels of analysis.
This article argues that the dual-process position can be a useful first approximation when studying human mental life, but it cannot be the whole truth. Instead, we argue that cognition is built on association, in that associative processes provide the fundamental building blocks that enable propositional thought. One consequence of this position is to suggest that humans are able to learn associatively in a similar fashion to a rat or a pigeon, but another is that we must typically suppress the expression of basic associative learning in favour of rule-based computation. This stance conceptualises us as capable of symbolic computation but acknowledges that, given certain circumstances, we will learn associatively and, more importantly, be seen to do so. We present three types of evidence that support this position: The first is data on human Pavlovian conditioning that directly support this view. The second is data taken from task-switching experiments that provide convergent evidence for at least two modes of processing, one of which is automatic and carried out "in the background." And the last suggests that when the output of propositional processes is uncertain, the influence of associative processes on behaviour can manifest.
Word learning is a notoriously difficult induction problem because meaning is underdetermined by positive examples. How do children solve this problem? Some have argued that word learning is achieved by means of inference: young word learners rely on a number of assumptions that reduce the overall hypothesis space by favoring some meanings over others. However, these approaches have difficulty explaining how words are learned from conversations or text, without pointing or explicit instruction. In this research, we propose an associative mechanism that can account for such learning. In a series of experiments, 4-year-olds and adults were presented with sets of words that included a single nonsense word (e.g. dax). Some lists were taxonomic (i.,e., all items were members of a given category), some were associative (i.e., all items were associates of a given category, but not members), and some were mixed. Participants were asked to indicate whether the nonsense word was an animal or an artifact. Adults exhibited evidence of learning when lists consisted of either associatively or taxonomically related items. In contrast, children exhibited evidence of word learning only when lists consisted of associatively related items. These results present challenges to several extant models of word learning, and a new model based on the distinction between syntagmatic and paradigmatic associations is proposed.
Significance
A novel theory suggests that orthographic processing is the product of neuronal recycling, with visual circuits that evolved to code visual objects now co-opted to code words. Here, we provide a litmus test of this theory by assessing whether pigeons, an organism with a visual system organizationally distinct from that of primates, code words orthographically. Pigeons not only correctly identified novel words but also display the hallmarks of orthographic processing, in that they are sensitive to the bigram frequencies of words, the orthographic similarity between words and nonwords, and the transposition of letters. These findings demonstrate that visual systems neither genetically nor organizationally similar to humans can be recycled to represent the orthographic code that defines words.
A fundamental task of language acquisition is to extract abstract algebraic rules. Three experiments show that 7-month-old
infants attend longer to sentences with unfamiliar structures than to sentences with familiar structures. The design of the
artificial language task used in these experiments ensured that this discrimination could not be performed by counting, by
a system that is sensitive only to transitional probabilities, or by a popular class of simple neural network models. Instead,
these results suggest that infants can represent, extract, and generalize abstract algebraic rules.
Transitive inference (TI) is a form of deductive reasoning that allows one to derive a relation between
items that have not been explicitly compared before. In a general form, TI is the ability to deduce that
if Item B is related to Item C and Item C is related to Item D, then Item B must be related to Item D.
This chapter begins with discussions of the methodology of nonverbal TI research, followed by
problems in interpreting the results of TI research. It reviews cognitive and reinforcement-based
models of TI and considers research on neurobiological mechanisms of nonverbal inferences. It shows
that with the single exception of honey bees, every species tested so far has been found to exhibit
transitive-like behavior in an n -term series task. Whether this overwhelming incidence of success
means that animals truly make inferences, however, has been a matter of a considerable debate that
continues to this day. The author concludes that current experimental evidence does not strongly
support either cognitive models or reinforcement-based models of TI.
Previous comparative work has suggested that the mechanisms of object categorization differ importantly for birds and primates. However, behavioral and neurobiological differences do not preclude the possibility that at least some of those mechanisms are shared across these evolutionarily distant groups. The present study integrates behavioral, neurobiological, and computational evidence concerning the "general processes" that are involved in object recognition in vertebrates. We start by reviewing work implicating error-driven learning in object categorization by birds and primates, and also consider neurobiological evidence suggesting that the basal ganglia might implement this process. We then turn to work with a computational model showing that principles of visual processing discovered in the primate brain can account for key behavioral findings in object recognition by pigeons, including cases in which pigeons' behavior differs from that of people. These results provide a proof of concept that the basic principles of visual shape processing are similar across distantly related vertebrate species, thereby offering important insights into the evolution of visual cognition.
The scientific study of associative learning began nearly 100 years ago with the pioneering studies of Thorndike and Pavlov, and it continues today as an active area of research and theory. Associative learning should be the foundation for our understanding of other forms of behavior and cognition in human and nonhuman animals. The laws of associative learning are complex, and many modern theorists posit the involvement of attention, memory, and information processing in such basic conditioning phenomena as overshadowing and blocking, and the effects of stimulus preexposure on later conditioning. An unresolved problem for learning theory is distinguishing the formation of associations from their behavioral expression. This and other problems will occupy future generations of behavioral scientists interested in the experimental investigation of associative learning. Neuroscientists and cognitive scientists will both contribute to and benefit from that effort in the next 100 years of inquiry.
Transitive inference has long been considered one of the hallmarks of human deductive reasoning. Recent reports of transitive-like behaviors in non-human animals have prompted a flourishing empirical and theoretical search for the mechanism(s) that may mediate this ability in non-humans. In this paper, I begin by describing the transitive inference tasks customarily used with non-human animals and then review the empirical findings. Transitive inference has been demonstrated in a wide variety of species, and the signature effects that usually accompany transitive inference in humans (the serial position effect and the symbolic distance effect) have also been found in non-humans. I then critically analyze the most prominent models of this ability in non-human animals. Some models are cognitive, proposing for instance that animals use the rules of formal logic or form mental representations of the premises to solve the task, others are based on associative mechanisms such as value transfer and reinforcement and non-reinforcement. Overall, I argue that the reinforcement-based models are in a much better empirical and theoretical position. Hence, transitive inference in non-human animals should be considered a property of reinforcement history rather than of inferential processes. I finalize by shedding some light on some promising lines of research.