Ida Momennejad

Ida Momennejad
Microsoft research NYC

PhD
I don't check this website often. If you need a manuscript, my website incudes PDFs for all papers: www.momen-nejad.org

About

49
Publications
7,995
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
933
Citations
Additional affiliations
July 2013 - present
Princeton University
Position
  • PostDoc Position

Publications

Publications (49)
Article
Full-text available
Successful realization of planned actions requires the brain to encode intentions over delays. Previous research has indicated that several regions in the rostral or anterior prefrontal cortex (PFC) encode delayed intentions. However, different processes may encode the same future task depending on task load during the delay. This difference may de...
Article
The development of shared memories, beliefs, and norms is a fundamental characteristic of human communities. These emergent outcomes are thought to occur owing to a dynamic system of information sharing and memory updating, which fundamentally depends on communication. Here we report results on the formation of collective memories in laboratory-cre...
Article
Full-text available
Theories of reward learning in neuroscience have focused on two families of algorithms thought to capture deliberative versus habitual choice. ‘Model-based’ algorithms compute the value of candidate actions from scratch, whereas ‘model-free’ algorithms make choice more efficient but less flexible by storing pre-computed action values. We examine an...
Preprint
Full-text available
The successor representation (SR) is a candidate principle for generalization in reinforcement learning, computational accounts of memory, and the structure of neural representations in the hippocampus. Given a sequence of states, the SR learns a predictive representation for every given state that encodes how often, on average, each upcoming state...
Preprint
Consider the problem setting of Interaction-Grounded Learning (IGL), in which a learner's goal is to optimally interact with the environment with no explicit reward to ground its policies. The agent observes a context vector, takes an action, and receives a feedback vector, using this information to effectively optimize a policy with respect to a l...
Preprint
Full-text available
The human cultural repertoire relies on innovation: our ability to continuously and hierarchically explore how existing elements can be combined to create new ones. Innovation is not solitary, it relies on collective accumulation and merging of previous solutions. Machine learning approaches commonly assume that fully connected multi-agent networks...
Preprint
In this work we propose Neuro-Nav, an open-source library for neurally plausible reinforcement learning (RL). RL is among the most common modeling frameworks for studying decision making, learning, and navigation in biological organisms. In utilizing RL, cognitive scientists often handcraft environments and agents to meet the needs of their particu...
Article
Our understanding of the world is shaped by inferences about underlying structure. For example, at the gym, you might notice that the same people tend to arrive around the same time and infer that they are friends that work out together. Consistent with this idea, after participants are presented with a temporal sequence of objects that follows an...
Preprint
In recent years, a growing number of deep model-based reinforcement learning (RL) methods have been introduced. The interest in deep model-based RL is not surprising, given its many potential benefits, such as higher sample efficiency and the potential for fast adaption to changes in the environment. However, we demonstrate, using an improved versi...
Preprint
Humans have the capability, aided by the expressive compositionality of their language, to learn quickly by demonstration. They are able to describe unseen task-performing procedures and generalize their execution to other contexts. In this work, we introduce DescribeWorld, an environment designed to test this sort of generalization skill in ground...
Article
Human cognition is not solitary, it is shaped by collective learning and memory. Unlike swarms or herds, human social networks have diverse topologies, serving diverse modes of collective cognition and behaviour. Here, we review research that combines network structure with psychological and neural experiments and modelling to understand how the to...
Article
Full-text available
As we navigate the world, we use learned representations of relational structures to explore and to reach goals. Studies of how relational knowledge enables inference and planning are typically conducted in controlled small-scale settings. It remains unclear, however, how people use stored knowledge in continuously unfolding navigation, e.g., walki...
Preprint
Full-text available
Our understanding of the world is shaped by inferences about underlying structure. For example, at the gym, you might notice that the same people tend to arrive around the same time and infer that they are friends that work out together. Consistent with this idea, after participants are presented with a temporal sequence of objects that follows an...
Article
The Learning Salon is an online weekly forum for discussing points of contention and common ground in biological and artificial learning. Hosting neuroscientists, computer scientists, AI researchers, and philosophers, the Salon promotes short talks and long discussions, committed to an ethos of participation, horizontality, and inclusion.
Preprint
Full-text available
Evaluating choices in multi-step tasks is thought to involve mentally simulating trajectories. Recent theories propose that the brain simplifies these laborious computations using temporal abstraction: storing actions' consequences, collapsed over multiple timesteps (the Successor Representation; SR). Although predictive neural representations and,...
Preprint
Artificial intelligence (AI) research plays an increasingly important role in society, impacting key aspects of human life. From face recognition algorithms aiding national security in airports, to software that advises judges in criminal cases, and medical staff in healthcare, AI research is shaping critical facets of our experience in the world....
Preprint
Consider a prosthetic arm, learning to adapt to its user's control signals. We propose Interaction-Grounded Learning for this novel setting, in which a learner's goal is to interact with the environment with no grounding or explicit reward to optimize its policies. Such a problem evades common RL solutions which require an explicit reward. The lear...
Preprint
Full-text available
A key challenge on the path to developing agents that learn complex human-like behavior is the need to quickly and accurately quantify human-likeness. While human assessments of such behavior can be highly accurate, speed and scalability are limited. We address these limitations through a novel automated Navigation Turing Test (ANTT) that learns to...
Article
Humans often simultaneously pursue multiple plans at different time scales, a capacity known as prospective memory (PM). The successful realization of non-immediate plans (e.g., post package after work) requires keeping track of a future plan while accomplishing other intermediate tasks (e.g., write a paper). Prospective memory capacity requires th...
Article
Memory and planning rely on learning the structure of relationships among experiences. Compact representations of these structures guide flexible behavior in humans and animals. A century after ‘latent learning’ experiments summarized by Tolman, the larger puzzle of cognitive maps remains elusive: how does the brain learn and generalize relational...
Preprint
Memory and planning rely on learning the structure of relationships among experiences. Compact representations of these structures guide flexible behavior in humans and animals. A century after ‘latent learning’ experiments summarized by Tolman, the larger puzzle of cognitive maps remains elusive: how does the brain learn and generalize relational...
Article
Full-text available
Anxiety disorders are characterized by a range of aberrations in the processing of and response to threat, but there is little clarity what core pathogenesis might underlie these symptoms. Here we propose that a particular set of unrealistically pessimistic assumptions can distort an agent’s behavior and underlie a host of seemingly disparate anxie...
Preprint
Full-text available
As we navigate the world we learn about associations among events, extract relational structures, and store them in memory. This relational knowledge, in turn, enables generalization, inference, and hierarchical planning. Here we investigated relational knowledge during spatial navigation as multiscale predictive representations in the brain. We hy...
Preprint
Full-text available
Gender inequality has been documented across a variety of high-prestige professions. Both structural bias (e.g., lack of proportionate representation) and interpersonal bias (e.g., sexism, discrimination) generate costs to underrepresented minorities. How can we estimate these costs and what interventions are most effective for reducing them? We us...
Preprint
Full-text available
We present the Spreading Activation and Memory PLasticity Model (SAMPL), a computational model of how memory retrieval changes memories. SAMPL restructures memory networks as a function of spreading activation and plasticity. Memory networks are represented as graphs of items in which edge weights capture the strength of association between items....
Preprint
Full-text available
Anxiety disorders are characterized by a range of aberrations in the processing of and response to threat, but there is little clarity what core pathogenesis might underlie these symptoms. Here we propose that a particular set of unrealistically pessimistic assumptions can distort an agent's behavior and underlie a host of seemingly disparate anxie...
Research
How can communication across networks of people be optimized to share information, while at the same time lessening the likelihood of information bubbles and echo chambers? In Episode 54, we're joined by Ida Momennejad and Ajua Duker from Columbia University and Yale University, respectively, to discuss their open access article “Bridge ties bind c...
Article
Full-text available
From families to nations, what binds individuals in social groups is, to a large degree, their shared beliefs, norms, and memories. These emergent outcomes are thought to occur because communication among individuals results in community-wide synchronization. Here, we use experimental manipulations in lab-created networks to investigate how the tem...
Preprint
Full-text available
A bstract Humans often simultaneously pursue multiple plans at different time scales. The successful realization of non-immediate plans (e.g., post package after work) requires keeping track of a future plan while accomplishing other intermediate tasks (e.g., write a paper), a capacity known as prospective memory . This capacity requires the integr...
Article
Full-text available
Making decisions in sequentially structured tasks requires integrating distally acquired information. The extensive computational cost of such integration challenges planning methods that integrate online, at decision time. Furthermore, it remains unclear whether 'offline' integration during replay supports planning, and if so which memories should...
Preprint
Full-text available
Making decisions in sequentially structured tasks requires integrating distally acquired information. The extensive computational cost of such integration challenges planning methods that integrate online, at decision time. Furthermore, it remains unclear whether “offline” integration during replay supports planning, and if so which memories should...
Article
Full-text available
Making decisions in sequentially structured tasks requires integrating distally acquired information. The extensive computational cost of such integration challenges planning methods that integrate online, at decision time. Furthermore, it remains unclear whether offline integration during replay supports planning, and if so which memories should b...
Data
Preventing replay slows acquisition for both SR-Dyna and Dyna-Q. Both algorithms under the two sampling settings were simulated on the task displayed in S1 Fig. a) Results of simulations with SR-Dyna. b) Results of simulations with Dyna-Q. Both a) and b) show number of steps on each trial for agent permitted to replay 20 samples between each decisi...
Data
Robustness of simulation results to varying parameters. Here, we display the results of simulating each task, using each algorithm under a wide variety of parameter settings. Each table below corresponds to a particular algorithm simulating a particular task. For a given parameter setting, the algorithm was simulated 500 times. A check indicates th...
Data
Advantage of TD learning over direct reward learning of weights. a) Task environment. On each trial, the agent was placed in state S. Trials ended when the agent reached state R, which contained a reward value of 10. Unlike the latent learning task in the main text, this task did not contain an exploratory period enabling the agent to learn the suc...
Preprint
Humans and animals are capable of evaluating actions by considering their long-run future rewards through a process described using model-based reinforcement learning (RL) algorithms. The mechanisms by which neural circuits perform the computations prescribed by model-based RL remain largely unknown; however, multiple lines of evidence suggest that...
Preprint
Full-text available
Theories of reward learning in neuroscience have focused on two families of algorithms, thought to capture deliberative vs. habitual choice. “Model-based” algorithms compute the value of candidate actions from scratch, whereas “model-free” algorithms make choice more efficient but less flexible by storing pre-computed action values. We examine an i...
Article
Full-text available
From families to nations, what binds individuals in social groups is the degree to which they share beliefs, norms, and memories. While local clusters of communicating individuals can sustain shared memories and norms, communities characterized by isolated cliques are susceptible to information fragmentation and polarization dynamics. We employ exp...
Article
Full-text available
From families to nations, what binds individuals in social groups is, to a large degree, their shared beliefs, norms, and memories. It is assumed that these emergent outcomes occur because communication among individuals results in community-wide synchronization. We employ experimental manipulations in lab-created networks to investigate how the te...
Article
Full-text available
Humans and animals are capable of evaluating actions by considering their long-run future rewards through a process described using model-based reinforcement learning (RL) algorithms. The mechanisms by which neural circuits perform the computations prescribed by model-based RL remain largely unknown; however, multiple lines of evidence suggest that...
Poster
Full-text available
The successful realization of future plans, prospective memory or PM, requires the agent to maintain and retrieve a goal for execution at a future time. PM poses a memory problem for periods during which the agent is occupied with other ongoing tasks (OG) while being responsive to target events that demand goal execution. We suggest a mechanistic a...
Article
Full-text available
Rewards obtained from specific behaviors can and do change across time. To adapt to such conditions, humans need to represent and update associations between behaviors and their outcomes. Much previous work focused on how rewards affect the processing of specific tasks. However, abstract associations between multiple potential behaviors and multipl...
Conference Paper
Full-text available
In recent years multivariate decoding has allowed to test where and how mental representations can be decoded from neuroimaging signals, which sheds light on how these representations are encoded in the brain. In one line of experiments, we investigated how intentions are encoded in fMRI signals, thus revealing information in medial and lateral pre...

Network

Cited By

Projects

Projects (4)
Project
The goal of this project is to describe and explain how motivation affects the neural coding of intentional action.