John P. O’Doherty's research while affiliated with California Institute of Technology and other places

... Consequently, the time limit can hinder goal-directed control system and force participants to rely more on habitual control system. These findings suggest that the imbalance between control systems in individuals who with overweight/obesity may not be as severe as it is in individuals who have binge eating disorder (Waltmann et al., 2021) or other disorders with compulsive behaviors (Seok et al., 2022). But those impairment can still affect reward-seeking behaviors among these individuals when the goal-directed control system was hindered by time pressure (Hardwick et al., 2019), negative effect, or depletion in cognitive resources (Ognibene et al., 2019). ...

... However, with extended training, those actions become increasingly more automatic (i. e., habitual; see de Wit and Dickinson, 2009, but see de Pool et al., 2022 for recent discussions on how overtraining affects the formation of habit-like behaviors). Thus, certain stimuli trigger automatic responses regardless of the value of that stimulus (Balleine and O'Doherty, 2010). ...

... In the present studies, we also investigated the effect of behavioral repetition on test performance. Despite several attempts, experimental investigations of habit formation in humans have failed to provide convincing evidence for the development of behavioral autonomy as a function of behavioral repetition during training Pool et al., 2021;Tricomi et al., 2009). Although there are recent suggestions that participants are slower to make novel responses in the presence of stimuli that have been overtrained with a different response (Hardwick et al., 2019;Luque et al., 2020), participants rarely make more overt 'slips of action' (commission errors) for long-trained relative to short-trained devalued outcomes, in classical outcome devaluation tasks . ...

... Particularly in recent years, researchers have discovered correspondences between the internal representations of deep learning models and those of the brain across various sensory and cognitive modalities (Yamins et al., 2014;Kell et al., 2018). These findings have advanced our understanding of brain functions through (a) the development of encoding models mediated by the representations (Güçlü & van Gerven, 2015), (b) interpretations of the representations based on their correspondence with brain functions (Cross et al., 2021;Takagi & Nishimoto, 2022), and (c) reconstruction of experienced content (such as visual images) from brain activity (Shen et al., 2019;Takagi & Nishimoto, 2023). ...

... We, thus, hypothesized that a potentially "latent " surprise signal would be present in the brain, performing "latent learning " ( Gläscher et al., 2010;Tolman, 1948 ). Correlated activity with the SPE or S BF would indicate that the relevant information is available in the brain, even if it is not currently used in guiding behavior ( Daw et al., 2005;Lee et al., 2014;O'Doherty et al., 2020 ). In order to study the correlation of brain activity with a potentially latent surprise signal, we used the linear model GLM 4 (see Methods). ...

... Place aversion, which is a form of Pavlovian conditioning, is also possible in virtual ecologies, where avoidance of particular areas of the environment demonstrates aversion (e.g., a predator was encountered in the location). Another approach taken from the field of behavioral ecology is the concept of margin of safety, or when prey adopt choices that prevent deadly outcomes from occurring by keeping close proximity to a safety refuge and increasing the success of escape (Cooper and Blumstein, 2015;Qi et al., 2020). Finally, potential threats lead to vigilance behaviors, including orienting toward and attending to threat, both in non-human species and in humans (Mobbs and Kim, 2015;Wise et al., 2019). ...

... While we cannot pinpoint the precise source of this reactivation using MEG alone, the regions within this cluster include those proposed to represent cognitive maps across multiple task domains (41,42). These regions are implicated also in counterfactual thinking (43)(44)(45) and avoidance (35,46,47). Our results extend upon this prior work and would be consistent with a proposal that these regions support a form of counterfactual updating based on a learned model of the environment. ...

... We find that the ramp develops and increases in slope over learning and also has a consistently greater slope during trials where the animal is actively engaged in reporting the reward location as opposed to when they are more disengaged. These characterizations are similarly applied to phasic RPEs Parker et al., 2016;Schultz et al., 1997;Tanaka et al., 2019). ...

... The power of formal computational models to uncover patterns, principles, and dynamics in social perception and behavior [100][101][102][103] has made them an increasingly popular tool in economics, psychology, and neuroscience [104][105][106][107][108][109][110] . Our results contribute to this movement by showing how computational models of social perception can provide novel insights into the different computations (i.e., bias and sensitivity) underlying impression formation and its effect on behavior. ...

... A functional magnetic resonance imaging study of social facilitation by the observer effect reported enhanced functional connectivity between cortical regions related to reward computation and behavioral motivation due to the presence of an observer 18 . Thus, if other people sharing a place were made aware of each other, their neural activity would be coordinated even if this is not directly related to their behavior. ...