Chris R. Sims’s research while affiliated with Rensselaer Polytechnic Institute and other places

The efficient representation of visual information is essential for learning and decision making due to the complexity and uncertainty of the world, as well as inherent constraints on the capacity of cognitive systems. We hypothesize that biological agents learn to efficiently represent visual information in a manner that balances performance across multiple potentially competing objectives. In this article, we examine two such objectives: storing information in a manner that supports accurate recollection (maximizing veridicality) and in a manner that facilitates utility-based decision making (maximizing behavioral utility). That these two objectives may be in conflict is not immediately obvious. Our hypothesis suggests that neither behavior nor representation formation can be fully understood by studying either in isolation, with information processing constraints exerting an overarching influence. Alongside this hypothesis we develop a computational model of representation formation and behavior motivated by recent methods in machine learning and neuroscience. The resulting model explains both the beneficial aspects of human visual learning, such as fast acquisition and high generalization, as well as the biases that result from information constraints. To test this model, we developed two experimental paradigms, in decision making and learning, to evaluate how well the model’s predictions match human behavior. A key feature of the proposed model is that it predicts the occurrence of commonly found biases in human decision making, resulting from the desire to form efficient representations of visual information that are useful for behavioral goals in learning and decision making and optimized under an information processing constraint.

Evidence accumulation models have enabled strong advances in our understanding of decision-making, yet their application to examining learning has not been common. Using data from participants completing a dynamic random dot-motion direction discrimination task across four days, we characterized alterations in two components of perceptual decision-making (Drift Diffusion Model drift rate and response boundary). Continuous-time learning models were applied to characterize trajectories of performance change, with different models allowing for varying dynamics. The best-fitting model included drift rate changing as a continuous, exponential function of cumulative trial number. In contrast, response boundary changed within each daily session, but in an independent manner across daily sessions. Our results highlight two different processes underlying the pattern of behavior observed across the entire learning trajectory, one involving a continuous tuning of perceptual sensitivity, and another more variable process describing participants' threshold of when enough evidence is present to act.

Humans learn quickly even in tasks that contain complex visual information. This is due in part to the efficient formation of compressed representations of visual information, allowing for better generalization and robustness. However, compressed representations alone are insufficient for explaining the high speed of human learning. Reinforcement learning (RL) models that seek to replicate this impressive efficiency may do so through the use of factored representations of tasks. These informationally simplistic representations of tasks are similarly motivated as the use of compressed representations of visual information. Recent studies have connected biological visual perception to disentangled and compressed representations. This raises the question of how humans learn to efficiently represent visual information in a manner useful for learning tasks. In this paper we present a model of human factored representation learning based on an altered form of a $\beta$-Variational Auto-encoder used in a visual learning task. Modelling results demonstrate a trade-off in the informational complexity of model latent dimension spaces, between the speed of learning and the accuracy of reconstructions.

The human brain is capable of forming informationally constrained representations of complex visual stimuli in order to achieve its behavioural goals, such as utility-based learning. Recently, methods borrowed from machine learning have demonstrated a close connection between the mechanisms of visual representation formation in primate brains with the latent representations formed by Beta-Variational Auto-Encoders (Beta-VAEs). While auto-encoder models capture some aspects of visual representations, they fail to explain how visual representations are adapted in a task-directed manner. We developed a model of visual representation formation in learning environments based on a modified Beta-VAE model that simultaneously learns the task-specific utility of visual information. We hypothesized that humans update their visual representations as they learn which visual features are associated with utility in learning tasks. To test this hypothesis, we applied the proposed model onto the data from a visual contextual bandit learning task [Niv et al. 2015; J. Neuroscience]. The experiment involved humans (N=22) learning the utility associated with 9 possible visual features (3 colors, shapes or textures). Critically, our model takes in as input the same visual information that is presented to participants, instead of the hand-crafted features typically used to model human learning. A comparison of predictive accuracy between our proposed model and models using hand-crafted features demonstrated a similar correlation to human learning. These results show that representations formed by our Beta-VAE based model can predict human learning from complex visual information. Additionally, our proposed model makes predictions of how visual representations adapt during human learning in a utility-based task. Further, we performed a comparison of our proposed model across a range of parameters such as information-constraint, utility-weight, and number of training steps between predictions. Results from this comparison give insight into how the human brain adjusts its visual representation formation during learning.

Humans learn quickly even in tasks that contain complex visual information. This is due in part to the efficient formation of compressed representations of visual information, allowing for better generalization and robustness. However, compressed representations alone are insufficient for explaining the high speed of human learning. Reinforcement learning (RL) models that seek to replicate this impressive efficiency may do so through the use of factored representations of tasks. These informationally simplistic representations of tasks are similarly motivated as the use of compressed representations of visual information. Recent studies have connected biological visual perception to disentangled and compressed representations. This raises the question of how humans learn to efficiently represent visual information in a manner useful for learning tasks. In this paper we present a model of human factored representation learning based on an altered form of a β-Variational Auto-encoder used in a visual learning task. Modelling results demonstrate a trade-off in the informational complexity of model latent dimension spaces, between the speed of learning and the accuracy of reconstructions.

Drift Diffusion Models have enabled strong advances in our understanding of decision-making, yet their application to examining learning has not been common. Using data from participants completing a dynamic random dot-motion direction discrimination task across four days, we characterized alterations in two components of perceptual decision-making (drift rate and response boundary). Continuous-time learning DDMs were applied to characterize trajectories of performance change, with different models allowing for varying dynamics. The best-fitting model included drift rate changing as a continuous, exponential function of cumulative trial number. In contrast, response boundary changed within each daily session, but in an independent manner across daily sessions. Our results highlight two different processes underlying the pattern of behavior observed across the entire learning trajectory, one involving a continuous tuning of perceptual sensitivity, and another more variable process describing participants’ threshold of when enough evidence is present to act.

Humans are able to learn about the visual world with a remarkable degree of generality and robustness, in part due to attention mechanisms which focus limited resources onto relevant features. Deep learning models that seek to replicate this feature of human learning can do so by optimizing a so-called "disentanglement objective", which encourages representations that factorize stimuli into separable feature dimensions [4]. This objective is achieved by methods such as the β-Variational Autoencoder (β-VAE), which has demonstrated a strong correspondence to neural activity in biological visual representation formation [5]. However, in the β-VAE method, learned visual representations are not influenced by the utility of information, but are solely learned in an unsupervised fashion. In contrast to this, humans exhibit generalization of learning through acquired equivalence of visual stimuli associated with similar outcomes [7]. The question of how humans combine utility-based and unsupervised learning in the formation of visual representations is therefore unanswered. The current paper seeks to address this question by developing a modified β-VAE model which integrates both unsupervised learning and reinforcement learning. This model is trained to produce both psychological representations of visual information as well as predictions of utility based on these representations. The result is a model that predicts the impact of changing utility on visual representations. Our model demonstrates a high degree of predictive accuracy of human visual learning in a contextual multi-armed bandit learning task [8]. Importantly, our model takes as input the same complex visual information presented to participants, instead of relying on hand-crafted features. These results provide further support for disentanglement as a plausible learning objective for visual representation formation by demonstrating their usefulness in learning tasks that rely on attention mechanisms.

Human visual working memory (VWM) is a memory store people use to maintain the visual features of objects and scenes. Although it is obvious that bottom-up information influences VWM, the extent to which top-down conceptual information influences VWM is largely unknown. We report an experiment in which groups of participants were trained in one of two different categories of geologic faults (left/right lateral, or normal/reverse faults), or received no category training. Following training, participants performed a visual change detection task in which category knowledge was irrelevant to the task. Participants were more likely to detect a change in geologic scenes when the changes crossed a trained categorical distinction (e.g., the left/right lateral fault boundary), compared to within-category changes. In addition, participants trained to distinguish left/right lateral faults were more likely to detect changes when the scenes were mirror images along the left/right dimension. Similarly, participants trained to distinguish normal/reverse faults were more likely to detect changes when scenes were mirror images along the normal/reverse dimension. Our results provide direct empirical evidence that conceptual knowledge influences VWM performance for complex visual information. An implication of our results is that cognitive scientists may need to reconceptualize VWM so that it is closer to “conceptual short-term memory”.

This paper explores information-theoretic constraints on methods for multi-agent reinforcement learning (MARL) in mixed cooperative and competitive games. Within this domain, decentralized training has been employed to increase learning sample efficiency. However, these approaches do not explicitly discourage complex policies, which can lead to over-fitting. To address this, we apply an information theoretic constraint onto agents' policies that discourages overly complex behaviour when it is not associated with a significant increase in reward. A second challenge in MARL is the non-stationarity of the environment introduced by other agents' changing policies. Previous methods in MARL have sought to reduce the impact of non-stationarity by inferring other agents' policies, but this can lead to over-fitting to previously observed behaviour. To avoid this, a similar information-theoretic constraint is applied onto the inference of other agents' policies, resulting in a more robust estimate. We evaluate the effects of these information-theoretic constraints on a test suite of multi-agent games, and report an overall improvement in performance, with greater improvements found in competitive domains compared to cooperative games.