Striatal activity underlies novelty-based choice in humans.

Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London WC1N3BG, UK.
Neuron (Impact Factor: 15.77). 07/2008; 58(6):967-73. DOI: 10.1016/j.neuron.2008.04.027
Source: PubMed

ABSTRACT The desire to seek new and unfamiliar experiences is a fundamental behavioral tendency in humans and other species. In economic decision making, novelty seeking is often rational, insofar as uncertain options may prove valuable and advantageous in the long run. Here, we show that, even when the degree of perceptual familiarity of an option is unrelated to choice outcome, novelty nevertheless drives choice behavior. Using functional magnetic resonance imaging (fMRI), we show that this behavior is specifically associated with striatal activity, in a manner consistent with computational accounts of decision making under uncertainty. Furthermore, this activity predicts interindividual differences in susceptibility to novelty. These data indicate that the brain uses perceptual novelty to approximate choice uncertainty in decision making, which in certain contexts gives rise to a newly identified and quantifiable source of human irrationality.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Humans and other biological agents are able to autonomously learn and cache different skills in the absence of any biological pressure or any assigned task. In this respect, Intrinsic Motivations (i.e. motivations not connected to reward-related stimuli) play a cardinal role in animal learning, and can be considered as a fundamental tool for developing more autonomous and more adaptive artificial agents. In this work, we provide an exhaustive analysis of a scarcely investigated problem: which kind of IM reinforcement signal is the most suitable for driving the acquisition of multiple skills in the shortest time? To this purpose we implemented an artificial agent with a hierarchical architecture that allows to learn and cache different skills. We tested the system in a setup with continuous states and actions, in particular, with a cinematic robotic arm that has to learn different reaching tasks. We compare the results of different versions of the system driven by several different intrinsic motivation signals. The results show a) that intrinsic reinforcements purely based on the knowledge of the system are not appropriate to guide the acquisition of multiple skills, and b) that the stronger the link between the IM signal and the competence of the system, the better the performance.
    Frontiers in Neurorobotics 11/2013; 7(22).
  • [Show abstract] [Hide abstract]
    ABSTRACT: Impulsive-compulsive behaviors (ICBs) in Parkinson's disease (PD) are a common and devastating side effect of dopamine replacement therapy. In this review we describe the phenomenology, prevalence, and risk factors of patients with PD. Results of behavioral studies assessing the neuropsychological profile of patients withPDemphasize that the ICBs, which are behavioral addictions, are not hedonically motivated. Rather, other factors such as the inability to cope with uncertainty may be triggering ICBs. New insights from functional imaging studies, strengthening the incentive salience hypothesis, are discussed, and therapeutic guidelines for the management of ICBs in PD are given. Expected final online publication date for the Annual Review of Clinical Psychology Volume 10 is March 20, 2014. Please see for revised estimates.
    Annual Review of Clinical Psychology 12/2013; · 12.42 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: How do we use our memories of the past to guide decisions we've never had to make before? Although extensive work describes how the brain learns to repeat rewarded actions, decisions can also be influenced by associations between stimuli or events not directly involving reward - such as when planning routes using a cognitive map or chess moves using predicted countermoves - and these sorts of associations are critical when deciding among novel options. This process is known as model-based decision making. While the learning of environmental relations that might support model-based decisions is well studied, and separately this sort of information has been inferred to impact decisions, there is little evidence concerning the full cycle by which such associations are acquired and drive choices. Of particular interest is whether decisions are directly supported by the same mnemonic systems characterized for relational learning more generally, or instead rely on other, specialized representations. Here, building on our previous work, which isolated dual representations underlying sequential predictive learning, we directly demonstrate that one such representation, encoded by the hippocampal memory system and adjacent cortical structures, supports goal-directed decisions. Using interleaved learning and decision tasks, we monitor predictive learning directly and also trace its influence on decisions for reward. We quantitatively compare the learning processes underlying multiple behavioral and fMRI observables using computational model fits. Across both tasks, a quantitatively consistent learning process explains reaction times, choices, and both expectation- and surprise-related neural activity. The same hippocampal and ventral stream regions engaged in anticipating stimuli during learning are also engaged in proportion to the difficulty of decisions. These results support a role for predictive associations learned by the hippocampal memory system to be recalled during choice formation.
    PLoS Computational Biology 12/2013; 9(12):e1003387. · 4.87 Impact Factor

Full-text (2 Sources)

Available from
May 22, 2014