No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Aesthetic preference for art emerges from a weighted integration over hierarchically structured visual features in the brain
Abstract
It is an open question whether preferences for visual art can be lawfully predicted from the basic constituent elements of a visual image. Moreover, little is known about how such preferences are actually constructed in the brain. Here we developed and tested a computational framework to gain an understanding of how the human brain constructs aesthetic value. We show that it is possible to explain human preferences for a piece of art based on an analysis of features present in the image. This was achieved by analyzing the visual properties of drawings and photographs by multiple means, ranging from image statistics extracted by computer vision tools, subjective human ratings about attributes, to a deep convolutional neural network. Crucially, it is possible to predict subjective value ratings not only within but also across individuals, speaking to the possibility that much of the variance in human visual preference is shared across individuals. Neuroimaging data revealed that preference computations occur in the brain by means of a graded hierarchical representation of lower and higher level features in the visual system. These features are in turn integrated to compute an overall subjective preference in the parietal and prefrontal cortex. Our findings suggest that rather than being idiosyncratic, human preferences for art can be explained at least in part as a product of a systematic neural integration over underlying visual features of an image. This work not only advances our understanding of the brain-wide computations underlying value construction but also brings new mechanistic insights to the study of visual aesthetics and art appreciation.
... Consequently, due to its multifaceted nature, researchers have employed various approaches, both objective and subjective, to measure creativity. On the one hand, studies have focused on objective formal-perceptual and design-oriented features, such as the Golden Ratio and luminance, comparing them to subjective preference ratings using classical and statistical learning methods [18][19][20][21] . Computational aesthetics has also evaluated aesthetic measures and quantification to generate designs (e.g. ...
... With the prediction of creativity judgements ratings as a target of art-attributes, we introduce a comprehensive method and a newly established initial model for art judgment analysis. The introduction of machine learning into art research is a consequent development of various methods to identify predictors of aesthetic appreciation and art evaluations, beginning with pure correlation or regression analyses [27][28][29] and more recent approaches using machine learning with objective values 8,[19][20][21] . ...
Creativity is a compelling yet elusive phenomenon, especially when manifested in visual art, where its evaluation is often a subjective and complex process. Understanding how individuals judge creativity in visual art is a particularly intriguing question. Conventional linear approaches often fail to capture the intricate nature of human behavior underlying such judgments. Therefore, in this study, we employed interpretable machine learning to probe complex associations between 17 subjective art-attributes and creativity judgments across a diverse range of artworks. A cohort of 78 non-art expert participants assessed 54 artworks varying in styles and motifs. The applied Random Forests regressor models accounted for 30% of the variability in creativity judgments given our set of art-attributes. Our analyses revealed symbolism, emotionality, and imaginativeness as the primary attributes influencing creativity judgments. Abstractness, valence, and complexity also had an impact, albeit to a lesser degree. Notably, we observed non-linearity in the relationship between art-attribute scores and creativity judgments, indicating that changes in art-attributes did not consistently correspond to changes in creativity judgments. Employing statistical learning, this investigation presents the first attribute-integrating quantitative model of factors that contribute to creativity judgments in visual art among novice raters. Our research represents a significant stride forward building the groundwork for first causal models for future investigations in art and creativity research and offering implications for diverse practical applications. Beyond enhancing comprehension of the intricate interplay and specificity of attributes used in evaluating creativity, this work introduces machine learning as an innovative approach in the field of subjective judgment.
... (2) Readability. e font should convey clear information, and the content is brief and easy to read. is is in line with the speed of modern information, which has the instant effect of visual communication [21]. (3) Modeling. ...
... In order to verify the effectiveness of the visual analysis method proposed in this paper, this paper defines the technique proposed in the paper as A and then compares with other methods, including method B: multimedia design based on dynamic information model [13]; method C: based on e-learning continuity image analysis model [14]; method D: analysis method based on discriminative visual elements [16]; method E: analysis method based on reactive calculation as a model [20]; method F: based on subjective VoIP quality evaluation model [21]; method G: analysis model based on collision design optimization [23]. Except for method A, the other methods are commonly used visual analysis methods. ...
With the rapid development of Internet technology, the dissemination of information is undergoing rapid changes. The Internet has become an indispensable part of people’s study, work, and life. In the context of the Internet, this paper analyzes the application of multimedia elements in visual communication art design, elaborates the basic concepts, visual features, and design principles of the visual communication art design, and analyzes the multifaceted aspects of multimedia elements. This paper discusses the basic methods and general rules of the application of multimedia elements in visual communication art design under the background of the Internet, as well as the research on the emotional factors and interactions of multimedia elements on people. Brain-like computing is the meaning of imitation brain. First is the structure, then the function, then from the structure design and analysis of the brain, and finally use the brain's thinking ability to solve problems. It summarizes the text, graphic images, and colors in the Internet background. The innovative application of multimedia elements such as page layout and animation in visual communication art design explains the influence of multimedia elements on the overall visual effect in visual communication art design. It explores how visual information can be conveyed more reasonably and effectively. In the context of the Internet, the application of multimedia elements has brought a new possibility for developing visual communication art design, providing a new platform for the traditional visual communication art design.
... Literature [22] rated the aesthetic stimulation of 14 painting elements on people, and found that people's overall feelings about painting should take priority over individual visual elements. Literature [8] found that human visual preference differences can be shared through the computational framework, and this difference is the human feeling of the whole painting. Literature [13] attempts to study human's objective feelings about abstract art and attempts to digitize this feeling. ...
An abstract painting’s hanging orientation directly affects how audiences judge its artistic value. Choosing the optimal hanging orientation can preserve the artist’s primary intention, preserving the original aesthetic value to a greater extent. Aesthetic value is frequently influenced by human subjective consciousness. Previous approaches improved direction recognition accuracy only by improving the feature extraction method and deep learning network. For this paper, the key factors that can influence recognition accuracy (such as painting content, image features and learning models) were investigated in conjunction with painting skills to find an experimental setting method that can enhance recognition accuracy. Experiment results show that the content of the painting has the greatest impact on classification accuracy. Furthermore, the average accuracy can be increased to more than 90% by reducing the number of painting categories in a dataset and the number of directions to be classified. While the outcome is superior to the state of the art, it is one-sided to rely solely on the information in the abstract painting. A combination of eye tracker data and questionnaires will be used in the future to examine the effect of audience subjective feelings on orientation classification.
... The regularization coefficient for the regression was tuned by doing a grid search, and the best-performing coefficient for each layer and feature was chosen based on the scores from the tenfold cross-validation. We tested for a total of 19 features, including all 18 features that we used for our functional magnetic resonance imaging (fMRI) analysis 55 , as well as the simplest feature that was not included into our fMRI analysis (as a result of our group-level feature selection) but that was also of interest here: the average hue value. In a supplementary analysis, we also explored whether adding style matrices of hidden layers 56 to the PCA-transformed hidden-layer activations can improve the decoding accuracy; however, we found that style matrices do not improve the decoding accuracy. ...
It is an open question whether preferences for visual art can be lawfully predicted from the basic constituent elements of a visual image. Here, we developed and tested a computational framework to investigate how aesthetic values are formed. We show that it is possible to explain human preferences for a visual art piece based on a mixture of low- and high-level features of the image. Subjective value ratings could be predicted not only within but also across individuals, using a regression model with a common set of interpretable features. We also show that the features predicting aesthetic preference can emerge hierarchically within a deep convolutional neural network trained only for object recognition. Our findings suggest that human preferences for art can be explained at least in part as a systematic integration over the underlying visual features of an image.
... These features can then be used to construct more abstract, high-level features about the painting, such as how dynamic or still the painting is. These higher level features can then in turn be integrated to compute an overall value judgement [12]. ...
Here we argue that the assignment of subjective value to potential outcomes at the time of decision-making is an active process, in which individual features of a potential outcome of varying degrees of abstraction are represented hierarchically and integrated in a weighted fashion to produce an overall value judgment. We implicate the lateral orbital and medial prefrontal cortex in this function, situating these areas more broadly within a hierarchical integration process that takes place throughout the cortex for the ultimate purpose of valuing options to guide decisions.
Based on the recent findings in human decision sciences, we propose a neurocomputational mechanism for the construction of a value signal for a food reward. In dietary choice, an individual computes the subjective value of a food by integrating its nutritive attributes (e.g. carbohydrate, fat, and protein content) that are inferred from low-level visual information (e.g. colour). This process is implemented in a hierarchically organised brain structure, including the early visual, parietal, and prefrontal cortices. We also discuss the effects of other factors such as perceived tastiness and healthiness and how the value evolves through experience. The proposed architecture for food valuation would provide a stimulating hypothesis for future studies in diverse research fields.
Empirical aesthetics has found its way into mainstream cognitive science. Until now, most research has focused either on identifying the internal processes that underlie a perceiver’s aesthetic experience or on identifying the stimulus features that lead to a specific type of aesthetic experience. To progress, empirical aesthetics must integrate these approaches into a unified paradigm that encourages researchers to think in terms of temporal dynamics and interactions between: (i) the stimulus and the perceiver; (ii) different systems within the perceiver; and (iii) different layers of the stimulus. At this critical moment, empirical aesthetics must also clearly identify and define its key concepts, sketch out its agenda, and specify its approach to grow into a coherent and distinct discipline.
Humans possess an exceptional aptitude to efficiently make decisions from high-dimensional sensory observations. However, it is unknown how the brain compactly represents the current state of the environment to guide this process. The deep Q-network (DQN) achieves this by capturing highly nonlinear mappings from multivariate inputs to the values of potential actions. We deployed DQN as a model of brain activity and behavior in participants playing three Atari video games during fMRI. Hidden layers of DQN exhibited a striking resemblance to voxel activity in a distributed sensorimotor network, extending throughout the dorsal visual pathway into posterior parietal cortex. Neural state-space representations emerged from nonlinear transformations of the pixel space bridging perception to action and reward. These transformations reshape axes to reflect relevant high-level features and strip away information about task-irrelevant sensory features. Our findings shed light on the neural encoding of task representations for decision-making in real-world situations.
In making decisions, we often choose from among options with multiple value-relevant attributes. Neuroeconomic models propose that the value associated with each attribute is integrated in a global value for each option. However, some evidence from patients with ventromedial frontal lobe (VMF) damage argues against a very general role for this region in value integration, suggesting instead that it contributes critically to a specific value inference or comparison process. Here,wetested value-based decision-making involving artificial multiattribute objects in humans with focal damage to the VMF (N=12) compared with a healthy group matched for age and education (N = 24) and with a group with frontal lobe damage sparing the VMF (N = 12). In a “configural” condition, overall object value was predicted by the conjunction of two attributes, while in an “elemental” condition, object value could be assessed by combining the independent values of individual attributes. Patients with VMF damage were impaired in making choices when value was uniquely predicted by the configuration of attributes, but intact when choosing based on elemental attribute values. This is evidence that the VMF is critical for inferring the value of whole objects in a multiattribute choice. These findings have implications for models of value-based choice and add to emerging views of how this region may interact with medial temporal lobe systems involved in configural object processing and relational memory.
The dominant paradigm for inference in psychology is a null-hypothesis significance testing one. Recently, the foundations of this paradigm have been shaken by several notable replication failures. One recommendation to remedy the replication crisis is to collect larger samples of participants. We argue that this recommendation misses a critical point, which is that increasing sample size will not remedy psychology’s lack of strong measurement, lack of strong theories and models, and lack of effective experimental control over error variance. In contrast, there is a long history of research in psychology employing small-N designs that treats the individual participant as the replication unit, which addresses each of these failings, and which produces results that are robust and readily replicated. We illustrate the properties of small-N and large-N designs using a simulated paradigm investigating the stage structure of response times. Our simulations highlight the high power and inferential validity of the small-N design, in contrast to the lower power and inferential indeterminacy of the large-N design. We argue that, if psychology is to be a mature quantitative science, then its primary theoretical aim should be to investigate systematic, functional relationships as they are manifested at the individual participant level and that, wherever possible, it should use methods that are optimized to identify relationships of this kind.
The concept of subjective value is central to current neurobiological views of economic decision-making. Much of this work has focused on signals in the ventromedial frontal lobe (VMF) that correlate with the subjective value of a variety of stimuli (e.g., food, monetary gambles), and are thought to support decision-making. However, the neural processes involved in assessing and integrating value information from the attributes of such complex options remain to be defined. Here, we tested the necessary role of VMF in weighting attributes of naturalistic stimuli during value judgments. We asked how distinct attributes of visual artworks influenced the subjective value ratings of subjects with VMF damage, compared to healthy participants and a frontal lobe damaged control group. Subjects with VMF damage were less influenced by the energy (emotion, complexity) and color radiance (warmth, saturation) of the artwork, while they were similar to control groups in considering saliency, balance and concreteness. These dissociations argue that VMF is critical for allowing certain affective content to influence subjective value, while sparing the influence of perceptual or representational information. These distinctions are important for better defining the often-underspecified concept of subjective value and developing more detailed models of the brain mechanisms underlying decision behavior.
The valuation of food is a fundamental component of our decision-making. Yet little is known about how value signals for food and other rewards are constructed by the brain. Using a food-based decision task in human participants, we found that subjective values can be predicted from beliefs about constituent nutritive attributes of food: protein, fat, carbohydrates and vitamin content. Multivariate analyses of functional MRI data demonstrated that, while food value is represented in patterns of neural activity in both medial and lateral parts of the orbitofrontal cortex (OFC), only the lateral OFC represents the elemental nutritive attributes. Effective connectivity analyses further indicate that information about the nutritive attributes represented in the lateral OFC is integrated within the medial OFC to compute an overall value. These findings provide a mechanistic account for the construction of food value from its constituent nutrients.
Converging evidence suggests that the primate ventral visual pathway encodes increasingly complex stimulus features in downstream areas. We quantitatively show that there indeed exists an explicit gradient for feature complexity in the ventral pathway of the human brain. This was achieved by mapping thousands of stimulus features of increasing complexity across the cortical sheet using a deep neural network. Our approach also revealed a fine-grained functional specialization of downstream areas of the ventral stream. Furthermore, it allowed decoding of representations from human brain activity at an unsurpassed degree of accuracy, confirming the quality of the developed approach. Stimulus features that successfully explained neural responses indicate that population receptive fields were explicitly tuned for object categorization. This provides strong support for the hypothesis that object categorization is a guiding principle in the functional organization of the primate ventral stream.
The human visual system contains an array of topographically organized regions. Identifying these regions in individual subjects is a powerful approach to group-level statistical analysis, but this is not always feasible. We addressed this limitation by generating probabilistic maps of visual topographic areas in 2 standardized spaces suitable for use with adult human brains. Using standard fMRI paradigms, we identified 25 topographic maps in a large population of individual subjects (N = 53) and transformed them into either a surface- or volume-based standardized space. Here, we provide a quantitative characterization of the inter-subject variability within and across visual regions, including the likelihood that a given point would be classified as a part of any region (full probability map) and the most probable region for any given point (maximum probability map). By evaluating the topographic organization across the whole of visual cortex, we provide new information about the organization of individual visual field maps and large-scale biases in visual field coverage. Finally, we validate each atlas for use with independent subjects. Overall, the probabilistic atlases quantify the variability of topographic representations in human cortex and provide a useful reference for comparing data across studies that can be transformed into these standard spaces.
© The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
Inferior temporal (IT) cortex in human and nonhuman primates serves visual object recognition. Computational object-vision models, although continually improving, do not yet reach human performance. It is unclear to what extent the internal representations of computational models can explain the IT representation. Here we investigate a wide range of computational model representations (37 in total), testing their categorization performance and their ability to account for the IT representational geometry. The models include well-known neuroscientific object-recognition models (e.g. HMAX, VisNet) along with several models from computer vision (e.g. SIFT, GIST, self-similarity features, and a deep convolutional neural network). We compared the representational dissimilarity matrices (RDMs) of the model representations with the RDMs obtained from human IT (measured with fMRI) and monkey IT (measured with cell recording) for the same set of stimuli (not used in training the models). Better performing models were more similar to IT in that they showed greater clustering of representational patterns by category. In addition, better performing models also more strongly resembled IT in terms of their within-category representational dissimilarities. Representational geometries were significantly correlated between IT and many of the models. However, the categorical clustering observed in IT was largely unexplained by the unsupervised models. The deep convolutional network, which was trained by supervision with over a million category-labeled images, reached the highest categorization performance and also best explained IT, although it did not fully explain the IT data. Combining the features of this model with appropriate weights and adding linear combinations that maximize the margin between animate and inanimate objects and between faces and other objects yielded a representation that fully explained our IT data. Overall, our results suggest that explaining IT requires computational features trained through supervised learning to emphasize the behaviorally important categorical divisions prominently reflected in IT.
The primate visual system achieves remarkable visual object recognition
performance even in brief presentations and under changes to object exemplar,
geometric transformations, and background variation (a.k.a. core visual object
recognition). This remarkable performance is mediated by the representation
formed in inferior temporal (IT) cortex. In parallel, recent advances in
machine learning have led to ever higher performing models of object
recognition using artificial deep neural networks (DNNs). It remains unclear,
however, whether the representational performance of DNNs rivals that of the
brain. To accurately produce such a comparison, a major difficulty has been a
unifying metric that accounts for experimental limitations such as the amount
of noise, the number of neural recording sites, and the number trials, and
computational limitations such as the complexity of the decoding classifier and
the number of classifier training examples. In this work we perform a direct
comparison that corrects for these experimental limitations and computational
considerations. As part of our methodology, we propose an extension of "kernel
analysis" that measures the generalization accuracy as a function of
representational complexity. Our evaluations show that, unlike previous
bio-inspired models, the latest DNNs rival the representational performance of
IT cortex on this visual object recognition task. Furthermore, we show that
models that perform well on measures of representational performance also
perform well on measures of representational similarity to IT and on measures
of predicting individual IT multi-unit responses. Whether these DNNs rely on
computational mechanisms similar to the primate visual system is yet to be
determined, but, unlike all previous bio-inspired models, that possibility
cannot be ruled out merely on representational performance grounds.
Prefrontal cortex is thought to have a fundamental role in flexible, context-dependent behaviour, but the exact nature of the computations underlying this role remains largely unknown. In particular, individual prefrontal neurons often generate remarkably complex responses that defy deep understanding of their contribution to behaviour. Here we study prefrontal cortex activity in macaque monkeys trained to flexibly select and integrate noisy sensory inputs towards a choice. We find that the observed complexity and functional roles of single neurons are readily understood in the framework of a dynamical process unfolding at the level of the population. The population dynamics can be reproduced by a trained recurrent neural network, which suggests a previously unknown mechanism for selection and integration of task-relevant inputs. This mechanism indicates that selection and integration are two aspects of a single dynamical process unfolding within the same prefrontal circuits, and potentially provides a novel, general framework for understanding context-dependent computations.
Prior experience is critical for decision-making. It enables explicit representation of potential outcomes and provides training to valuation mechanisms. However, we can also make choices in the absence of prior experience by merely imagining the consequences of a new experience. Using functional magnetic resonance imaging repetition suppression in humans, we examined how neuronal representations of novel rewards can be constructed and evaluated. A likely novel experience was constructed by invoking multiple independent memories in hippocampus and medial prefrontal cortex. This construction persisted for only a short time period, during which new associations were observed between the memories for component items. Together, these findings suggest that, in the absence of direct experience, coactivation of multiple relevant memories can provide a training signal to the valuation system that allows the consequences of new experiences to be imagined and acted on.
Single-neuron activity in the prefrontal cortex (PFC) is tuned to mixtures of multiple task-related aspects. Such mixed selectivity is highly heterogeneous, seemingly disordered and therefore difficult to interpret. We analysed the neural activity recorded in monkeys during an object sequence memory task to identify a role of mixed selectivity in subserving the cognitive functions ascribed to the PFC. We show that mixed selectivity neurons encode distributed information about all task-relevant aspects. Each aspect can be decoded from the population of neurons even when single-cell selectivity to that aspect is eliminated. Moreover, mixed selectivity offers a significant computational advantage over specialized responses in terms of the repertoire of input-output functions implementable by readout neurons. This advantage originates from the highly diverse nonlinear selectivity to mixtures of task-relevant variables, a signature of high-dimensional neural representations. Crucially, this dimensionality is predictive of animal behaviour as it collapses in error trials. Our findings recommend a shift of focus for future studies from neurons that have easily interpretable response tuning to the widely observed, but rarely analysed, mixed selectivity neurons.
We often have to make choices among multiattribute stimuli (e.g., a food that differs on its taste and health). Behavioral data suggest that choices are made by computing the value of the different attributes and then integrating them into an overall stimulus value signal. However, it is not known whether this theory describes the way the brain computes the stimulus value signals, or how the underlying computations might be implemented. We investigated these questions using a human fMRI task in which individuals had to evaluate T-shirts that varied in their visual esthetic (e.g., color) and semantic (e.g., meaning of logo printed in T-shirt) components. We found that activity in the fusiform gyrus, an area associated with the processing of visual features, correlated with the value of the visual esthetic attributes, but not with the value of the semantic attributes. In contrast, activity in posterior superior temporal gyrus, an area associated with the processing of semantic meaning, exhibited the opposite pattern. Furthermore, both areas exhibited functional connectivity with an area of ventromedial prefrontal cortex that reflects the computation of overall stimulus values at the time of decision. The results provide supporting evidence for the hypothesis that some attribute values are computed in cortical areas specialized in the processing of such features, and that those attribute-specific values are then passed to the vmPFC to be integrated into an overall stimulus value signal to guide the decision.
We present a theory of human artistic experience and the neural mechanisms that mediate it. Any theory of art (or, indeed, any aspect of human nature) has to ideally have three components. (a) The logic of art: whether there are universal rules or principles; (b) The evolutionary rationale: why did these rules evolve and why do they have the form that they do; (c) What is the brain circuitry involved? Our paper begins with a quest for artistic universals and proposes a list of Eight laws of artistic experience -- a set of heuristics that artists either consciously or unconsciously deploy to optimally titillate the visual areas of the brain. One of these principles is a psychological phenomenon called the peak shift effect: If a rat is rewarded for discriminating a rectangle from a square, it will respond even more vigorously to a rectangle that is longer and skinnier that the prototype. We suggest that this principle explains not only caricatures, but many other aspects of art. Example: An evocative sketch of a female nude may be one which selectively accentuates those feminine form-attributes that allow one to discriminate it from a male figure; a Boucher, a Van Gogh, or a Monet may be a caricature in colour space rather than form space. Even abstract art may employ supernormal stimuli to excite form areas in the brain more strongly than natural stimuli. Second, we suggest that grouping is a very basic principle. The different extrastriate visual areas may have evolved specifically to extract correlations in different domains (e.g. form, depth, colour), and discovering and linking multiple features (grouping) into unitary clusters -- objects -- is facilitated and reinforced by direct connections from these areas to limbic structures. In general, when object-like entities are partially discerned at any stage in the visual hierarchy, messages are sent back to earlier stages to alert them to certain locations or features in order to look for additional evidence for the object (and these processes may be facilitated by direct limbic activation). Finally, given constraints on allocation of attentional resources, art is most appealing if it produces heightened activity in a single dimension (e.g. through the peak shift principle or through grouping) rather than redundant activation of multiple modules. This idea may help explain the effectiveness of outline drawings and sketches, the savant syndrome in autists, and the sudden emergence of artistic talent in fronto-temporal dementia. In addition to these three basic principles we propose five others, constituting a total of eight laws of aesthetic experience(analogous to the Buddha's eightfold path to wisdom).
Aesthetic responses to visual art comprise multiple types of experiences, from sensation and perception to emotion and self-reflection. Moreover, aesthetic experience is highly individual, with observers varying significantly in their responses to the same artwork. Combining fMRI and behavioral analysis of individual differences in aesthetic response, we identify two distinct patterns of neural activity exhibited by different sub-networks. Activity increased linearly with observers' ratings (4-level scale) in sensory (occipito-temporal) regions. Activity in the striatum (STR) also varied linearly with ratings, with below-baseline activations for low-rated artworks. In contrast, a network of frontal regions showed a step-like increase only for the most moving artworks ("4" ratings) and non-differential activity for all others. This included several regions belonging to the "default mode network" (DMN) previously associated with self-referential mentation. Our results suggest that aesthetic experience involves the integration of sensory and emotional reactions in a manner linked with their personal relevance.
The purpose of this study is to investigate multiregion graph cut image partitioning via kernel mapping of the image data. The image data is transformed implicitly by a kernel function so that the piecewise constant model of the graph cut formulation becomes applicable. The objective function contains an original data term to evaluate the deviation of the transformed data, within each segmentation region, from the piecewise constant model, and a smoothness, boundary preserving regularization term. The method affords an effective alternative to complex modeling of the original image data while taking advantage of the computational benefits of graph cuts. Using a common kernel function, energy minimization typically consists of iterating image partitioning by graph cut iterations and evaluations of region parameters via fixed point computation. A quantitative and comparative performance assessment is carried out over a large number of experiments using synthetic grey level data as well as natural images from the Berkeley database. The effectiveness of the method is also demonstrated through a set of experiments with real images of a variety of types such as medical, synthetic aperture radar, and motion maps.
The explosion of image data on the Internet has the potential to foster more sophisticated and robust models and algorithms to index, retrieve, organize and interact with images and multimedia data. But exactly how such data can be harnessed and organized remains a critical problem. We introduce here a new database called ldquoImageNetrdquo, a large-scale ontology of images built upon the backbone of the WordNet structure. ImageNet aims to populate the majority of the 80,000 synsets of WordNet with an average of 500-1000 clean and full resolution images. This will result in tens of millions of annotated images organized by the semantic hierarchy of WordNet. This paper offers a detailed analysis of ImageNet in its current state: 12 subtrees with 5247 synsets and 3.2 million images in total. We show that ImageNet is much larger in scale and diversity and much more accurate than the current image datasets. Constructing such a large-scale database is a challenging task. We describe the data collection scheme with Amazon Mechanical Turk. Lastly, we illustrate the usefulness of ImageNet through three simple applications in object recognition, image classification and automatic object clustering. We hope that the scale, accuracy, diversity and hierarchical structure of ImageNet can offer unparalleled opportunities to researchers in the computer vision community and beyond.
The problem of efficient, interactive foreground/background segmentation in still images is of great practical importance in image editing. Classical image segmentation tools use either texture (colour) information, e.g. Magic Wand, or edge (contrast) information, e.g. Intelligent Scissors. Recently, an approach based on optimization by graph-cut has been developed which successfully combines both types of information. In this paper we extend the graph-cut approach in three respects. First, we have developed a more powerful, iterative version of the optimisation. Secondly, the power of the iterative algorithm is used to simplify substantially the user interaction needed for a given quality of result. Thirdly, a robust algorithm for "border matting" has been developed to estimate simultaneously the alpha-matte around an object boundary and the colours of foreground pixels. We show that for moderately difficult examples the proposed method outperforms competitive tools.
Uncertainty about the function of orbitofrontal cortex (OFC) in guiding decision-making may be a result of its medial (mOFC) and lateral (lOFC) divisions having distinct functions. Here we test the hypothesis that the mOFC is more concerned with reward-guided decision making, in contrast with the lOFC's role in reward-guided learning. Macaques performed three-armed bandit tasks and the effects of selective mOFC lesions were contrasted against lOFC lesions. First, we present analyses that make it possible to measure reward-credit assignment--a crucial component of reward-value learning--independently of the decisions animals make. The mOFC lesions do not lead to impairments in reward-credit assignment that are seen after lOFC lesions. Second, we examined how the reward values of choice options were compared. We present three analyses, one of which examines reward-guided decision making independently of reward-value learning. Lesions of the mOFC, but not the lOFC, disrupted reward-guided decision making. Impairments after mOFC lesions were a function of the multiple option contexts in which decisions were made. Contrary to axiomatic assumptions of decision theory, the mOFC-lesioned animals' value comparisons were no longer independent of irrelevant alternatives.
Dieter's Dilemma
The ability to exercise self-control is central to human success and well-being. However, little is known about the neurobiological underpinnings of self-control and how or why these neural mechanisms might differ between successful and unsuccessful decision-makers. Hare et al. (p. 646 ) used brain imaging in a dieting population undergoing real-life decisions between a healthy or a tempting, yet nutritionally inferior, choice of food. Activity in the ventromedial prefrontal cortex correlated with the value of the stimulus, termed goal value. Importantly, this activity integrated both health and taste values in individuals who were able to exert self-control in their choices, while reflecting only taste in those unable to exert self-control.
Neuroimaging studies of decision-making have generally related neural activity to objective measures (such as reward magnitude, probability or delay), despite choice preferences being subjective. However, economic theories posit that decision-makers behave as though different options have different subjective values. Here we use functional magnetic resonance imaging to show that neural activity in several brain regions—particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex—tracks the revealed subjective value of delayed monetary rewards. This similarity provides unambiguous evidence that the subjective value of potential rewards is explicitly represented in the human brain.
In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively.
Deep learning algorithms seek to exploit the unknown structure in the input distribution in order to discover good representations, often at multiple levels, with higher-level learned features defined in terms of lower-level features. The objective is to make these higher- level representations more abstract, with their individual features more invariant to most of the variations that are typically present in the training distribution, while collectively preserving as much as possible of the information in the input. Ideally, we would like these representations to disentangle the unknown factors of variation that underlie the training distribution. Such unsupervised learning of representations can be exploited usefully under the hypothesis that the input distribution P(x) is structurally related to some task of interest, say predicting P(y|x). This paper focusses on why unsupervised pre-training of representations can be useful, and how it can be exploited in the transfer learning scenario, where we care about predictions on examples that are not from the same distribution as the training distribution
Image aesthetics has become an important criterion for visual content curation on social media sites and media content repositories. Previous work on aesthetic prediction models in the computer vision community has focused on aesthetic score prediction or binary image labeling. However, raw aesthetic annotations are in the form of score histograms and provide richer and more precise information than binary labels or mean scores. Consequently, in this work we focus on the rarely-studied problem of predicting aesthetic score distributions and propose a novel architecture and training procedure for our model. Our model achieves state-of-the-art results on the standard AVA large-scale benchmark dataset for three tasks: (i) aesthetic quality classification; (ii) aesthetic score regression; and (iii) aesthetic score distribution prediction, all while using one model trained only for the distribution prediction task. We also introduce a method to modify an image such that its predicted aesthetics changes, and use this modification to gain insight into our model.
To clarify the organization of motor representations in posterior parietal cortex, we test how three motor variables (body side, body part, cognitive strategy) are coded in the human anterior intraparietal cortex. All tested movements were encoded, arguing against strict anatomical segregation of effectors. Single units coded for diverse conjunctions of variables, with different dimensions anatomically overlapping. Consistent with recent studies, neurons encoding body parts exhibited mixed selectivity. This mixed selectivity resulted in largely orthogonal coding of body parts, which “functionally segregate” the effector responses despite the high degree of anatomical overlap. Body side and strategy were not coded in a mixed manner as effector determined their organization. Mixed coding of some variables over others, what we term “partially mixed coding,” argues that the type of functional encoding depends on the compared dimensions. This structure is advantageous for neuroprosthetics, allowing a single array to decode movements of a large extent of the body.
People can judge whether they will enjoy dishes like waffles with horseradish cream sauce or broccoli ice cream even if they have never tried them. What representations and computations support reasoning in such situations? We develop a theory of decision making in combinatorial domains. Its central claim is that utility functions can be compositionally structured: The utility of a combination is a function of its constituents’ utilities and the rules for combining them. Utilities are induced from experience by probabilistic reasoning over the structured space of utility functions. In a series of experiments, we show how this theory can capture human evaluations of novel food combinations. We first show that the theory quantitatively predicts evaluations of novel food combinations. We then report more strongly controlled experiments (using unfamiliar foods) that rule out several alternative theories. Taken together, these experiments demonstrate how compositionally structured representations of utility can support decision making in combinatorial domains.
Extensive research has revealed that the ventral visual stream hierarchically builds a robust representation for supporting visual object categorization tasks. We systematically explored the ability of multiple ventral visual areas to support a variety of 'category-orthogonal' object properties such as position, size and pose. For complex naturalistic stimuli, we found that the inferior temporal (IT) population encodes all measured category-orthogonal object properties, including those properties often considered to be low-level features (for example, position), more explicitly than earlier ventral stream areas. We also found that the IT population better predicts human performance patterns across properties. A hierarchical neural network model based on simple computational principles generates these same cross-area patterns of information. Taken together, our empirical results support the hypothesis that all behaviorally relevant object properties are extracted in concert up the ventral visual hierarchy, and our computational model explains how that hierarchy might be built.
In recent years, many new cortical areas have been identified in the macaque monkey. The number of identified connections between areas has increased even more dramatically. We report here on (1) a summary of the layout of cortical areas associated with vision and with other modalities, (2) a computerized database for storing and representing large amounts of information on connectivity patterns, and (3) the application of these data to the analysis of hierarchical organization of the cerebral cortex. Our analysis concentrates on the visual system, which includes 25 neocortical areas that are predominantly or exclusively visual in function, plus an additional 7 areas that we regard as visual-association areas on the basis of their extensive visual inputs. A total of 305 connections among these 32 visual and visual-association areas have been reported. This represents 31% of the possible number of pathways it each area were connected with all others. The actual degree of connectivity is likely to be closer to 40%. The great majority of pathways involve reciprocal connections between areas. There are also extensive connections with cortical areas outside the visual system proper, including the somatosensory cortex, as well as neocortical, transitional, and archicortical regions in the temporal and frontal lobes. In the somatosensory/motor system, there are 62 identified pathways linking 13 cortical areas, suggesting an overall connectivity of about 40%. Based on the laminar patterns of connections between areas, we propose a hierarchy of visual areas and of somato sensory/motor areas that is more comprehensive than those suggested in other recent studies. The current version of the visual hierarchy includes 10 levels of cortical processing. Altogether, it contains 14 levels if one includes the retina and lateral geniculate nucleus at the bottom as well as the entorhinal cortex and hippocampus at the top. Within this hierarchy, there are multiple, intertwined processing streams, which, at a low level, are related to the compartmental organization of areas V1 and V2 and, at a high level, are related to the distinction between processing centers in the temporal and parietal lobes. However, there are some pathways and relationships (about 10% of the total) whose descriptions do not fit cleanly into this hierarchical scheme for one reason or another. In most instances, though, it is unclear whether these represent genuine exceptions to a strict hierarchy rather than inaccuracies or uncertainties in the reported assignment.
Availability: Full Text Available
The neural basis for visual aesthetics is largely unknown. Yet, murmurings within cognitive neuroscience suggest this will soon change. In this review, I suggest several ways in which cognitive neuroscience might contribute to studies in aesthetics. First, I present a framework, adapted from the cognitive neuroscience of vision, from which hypotheses about neuroaesthetics might be tested. Following that, I outline several ideas advanced by prominent neuroscientists that are provocative, but in need of experimental testing. Then I point to the effects of brain damage on artists, as contributing to our understanding of the neural bases of artistic production. Finally, I mention recent functional neuroimaging studies that are relevant to aesthetic concerns. These studies examine the neural response to beautiful faces and its relationship to affective systems within the brain. While it is too early to be sure, programmatic research in the cognitive neuroscience of aesthetics promises rich rewards by bringing new ways to approach empirical aesthetics. However, much work remains to be done. (PsycEXTRA Database Record (c) 2013 APA, all rights reserved)
Most real-world odors are complex mixtures of distinct molecular components. Olfactory systems can adopt different strategies to contend with this stimulus complexity. In elemental processing, odor perception is derived from the sum of its parts; in configural processing, the parts are integrated into unique perceptual wholes. Here we used gas-chromatography/mass-spectrometry techniques to deconstruct a complex natural food smell and assess whether olfactory salience is confined to the whole odor or is also embodied in its parts. By implementing an fMRI sensory-specific satiety paradigm, we identified reward-based changes in orbitofrontal cortex (OFC) for the whole odor and for a small subset of components. Moreover, component-specific changes in OFC-amygdala connectivity correlated with perceived value. Our findings imply that the human brain has direct access to the elemental content of a natural food odor, and highlight the dynamic capacity of the olfactory system to engage both object-level and component-level mechanisms to subserve behavior.
Copyright © 2014 Elsevier Inc. All rights reserved.
Art museum attendance is rising steadily, unchallenged by online alternatives. However, the psychological value of the real museum experience remains unclear because the experience of art in the museum and other contexts has not been compared. Here we examined the appreciation and memory of an art exhibition when viewed in a museum or as a computer simulated version in the laboratory. In line with the postulates of situated cognition, we show that the experience of art relies on organizing resources present in the environment. Specifically, artworks were found more arousing, positive, interesting and liked more in the museum than in the laboratory. Moreover, participants who saw the exhibition in the museum later recalled more artworks and used spatial layout cues for retrieval. Thus, encountering real art in the museum enhances cognitive and affective processes involved in the appreciation of art and enriches information encoded in long-term memory.
What do we do when we view a work of art? What does it mean to have an "aesthetic" experience? Are such experiences purely in the eye (and brain) of the beholder? Such questions have entertained philosophers for millennia and psychologists for over a century. More recently, with the advent of functional neuroimaging methods, a handful of ambitious brain scientists have begun to explore the neural correlates of such experiences. The notion of aesthetics is generally linked to the way art evokes an hedonic response-we like it or we don't. Of course, a multitude of factors can influence such judgments, such as personal interest, past experience, prior knowledge, and cultural biases. In this book, philosophers, psychologists, and neuroscientists were asked to address the nature of aesthetic experiences from their own discipline's perspective. In particular, the scholars were asked to consider whether a multidisciplinary approach, an aesthetic science, could help connect mind, brain, and aesthetics. As such, this book offers an introduction to the way art is perceived, interpreted, and felt and approaches these mindful events from a multidisciplinary perspective.
With the ever-expanding volume of visual content available, the ability to organize and navigate such content by aesthetic preference is becoming increasingly important. While still in its nascent stage, research into computational models of aesthetic preference already shows great potential. However, to advance research, realistic, diverse and challenging databases are needed. To this end, we introduce a new large-scale database for conducting Aesthetic Visual Analysis: AVA. It contains over 250,000 images along with a rich variety of meta-data including a large number of aesthetic scores for each image, semantic labels for over 60 categories as well as labels related to photographic style. We show the advantages of AVA with respect to existing databases in terms of scale, diversity, and heterogeneity of annotations. We then describe several key insights into aesthetic preference afforded by AVA. Finally, we demonstrate, through three applications, how the large scale of AVA can be leveraged to improve performance on existing preference tasks.
Neuropsychological investigations of art production and perception have the potential to offer critical insight into the biology of visual aesthetics. Thus far, however, investigations of art production in patients have been limited to anecdotal observations and investigations of art perception are non-existent. Progress in the field is hampered by the lack of an adequate instrument to provide basic quantification of artwork attributes. Motivated by the need to move neuropsychology of art beyond the fascinating anecdote, we present the Assessment of Art Attributes (AAA). The AAA is an instrument designed to assess six formal-perceptual and six conceptual-representational attributes using 24 paintings from the Western canon. Both artistically naïve and experienced participants were given the AAA. We found high degrees of agreement in the assessment of these attributes in both groups and few differences between the groups. We expect that the AAA's componential and quantitative approach will be useful in advancing neuropsychological studies as well as any investigations in which style and content of art works need to be quantified and compared.
We recorded brain activity when 21 subjects judged the beauty (aesthetic or affective judgment) and brightness (perceptual or cognitive judgment) of simultaneously presented paintings. Aesthetic judgments engaged medial and lateral subdivisions of the orbitofrontal cortex as well as subcortical stations associated with affective motor planning (globus pallidus, putamen-claustrum, amygdala, and cerebellar vermis), whereas the motor, premotor and supplementary motor areas, as well as the anterior insula and the dorsolateral prefrontal cortex, were engaged by both kinds of judgment. The results lead us to conclude: (i) that there is a functional specialization for judgment, with aesthetic judgments engaging distinct systems, in addition to those that they share with perceptual judgments; (ii) that the systems engaged by affective judgments are those in which activity correlates with polar experiences (e.g. love-hate, beauty-ugliness, and attraction-repulsion); and (iii) that there is also a functional specialization in the motor pathways, with aesthetic judgments engaging motor systems not engaged by perceptual judgments, in addition to those engaged by both kinds of judgment.
The Right Choice?
So-called irrational decisions made by humans are popular fodder for “believe it or not” stories. But what's actually happening when we make choices that do not seem to be justifiable on purely economic or logical grounds? Presumably, we are not simply making errors; instead, our choices may reflect an internal bias that we are not aware of. Wimmer and Shohamy (p. 270 ) show how the hippocampus can instill an unconscious bias in valuations, whereby an object that is not highly valued on its own, increases in value when it becomes implicitly associated with a truly high-value object. As a consequence, we then end up preferring the associated object over a neutral object of equal objective value while not really knowing why.
Our memories define who we are and what we do. Aside from a few preferences hardwired by evolution, they also define what we like and how we choose. In this chapter, we argue that our view of preference changes if conceptualized explicitly as the product of memory representations and memory processes. We draw on insights about the functions and operations of memory provided by cognitive psychology and social cognition to show that memory plays a crucial role in preference and choice. We examine memory processes in preference and choice at a more "micro" and process-oriented level than previous investigations into the role of memory processes, but at a level that is cognitive and functional, rather than computational. We suggest that a consideration of properties of memory representation and retrieval may provide a unifying explanatory framework for some seemingly disparate preference phenomena.
This paper aims to evaluate the aesthetic visual quality of a special type of visual media: digital images of paintings. Assessing the aesthetic visual quality of paintings can be considered a highly subjective task. However, to some extent, certain paintings are believed, by consensus, to have higher aesthetic quality than others. In this paper, we treat this challenge as a machine learning problem, in order to evaluate the aesthetic quality of paintings based on their visual content. We design a group of methods to extract features to represent both the global characteristics and local characteristics of a painting. Inspiration for these features comes from our prior knowledge in art and a questionnaire survey we conducted to study factors that affect human's judgments. We collect painting images and ask human subjects to score them. These paintings are then used for both training and testing in our experiments. Experimental results show that the proposed work can classify high-quality and low-quality paintings with performance comparable to humans. This work provides a machine learning scheme for the research of exploring the relationship between aesthetic perceptions of human and the computational visual features extracted from paintings.
A variant of the popular nonparametric nonuniform intensity normalization (N3) algorithm is proposed for bias field correction. Given the superb performance of N3 and its public availability, it has been the subject of several evaluation studies. These studies have demonstrated the importance of certain parameters associated with the B -spline least-squares fitting. We propose the substitution of a recently developed fast and robust B-spline approximation routine and a modified hierarchical optimization scheme for improved bias field correction over the original N3 algorithm. Similar to the N3 algorithm, we also make the source code, testing, and technical documentation of our contribution, which we denote as ??N4ITK,?? available to the public through the Insight Toolkit of the National Institutes of Health. Performance assessment is demonstrated using simulated data from the publicly available Brainweb database, hyperpolarized <sup>3</sup>He lung image data, and 9.4T postmortem hippocampus data.
Several researchers characterized the activation function under which multilayer feedforward networks can act as universal approximators. We show that most of all the characterizations that were reported thus far in the literature are special cases of the following general result: A standard multilayer feedforward network with a locally bounded piecewise continuous activation function can approximate any continuous function to any degree of accuracy if and only if the network's activation function is not a polynomial. We also emphasize the important role of the threshold, asserting that without it the last theorem does not hold.
In the macaque monkey, a dozen distinct visual areas have been identified in the cerebral cortex. These areas can be arranged in a well-defined hierarchy on the basis of their pattern of interconnections. Physiological recordings suggest that there are at least two major functional streams in this hierarchy, one related to the analysis of motion and the other to the analysis of form and color.
This paper contains a new approach toward a theory of robust estimation; it treats in detail the asymptotic theory of estimating a location parameter for contaminated normal distributions, and exhibits estimators--intermediaries between sample mean and sample median--that are asymptotically most robust (in a sense to be specified) among all translation invariant estimators.
Rapid advances have recently been made in understanding how value-based decision-making processes are implemented in the brain. We integrate neuroeconomic and computational approaches with evidence on the neural correlates of value and experienced pleasure to describe how systems for valuation and decision-making are organized in the prefrontal cortex of humans and other primates. We show that the orbitofrontal and ventromedial prefrontal (VMPFC) cortices compute expected value, reward outcome and experienced pleasure for different stimuli on a common value scale. Attractor networks in VMPFC area 10 then implement categorical decision processes that transform value signals into a choice between the values, thereby guiding action. This synthesis of findings across fields provides a unifying perspective for the study of decision-making processes in the brain.
Many problems in machine learning and statistics can be formulated as (generalized) eigenproblems. In terms of the associated optimization problem, computing linear eigenvectors amounts to finding critical points of a quadratic function subject to quadratic constraints. In this paper we show that a certain class of constrained optimization problems with nonquadratic objective and constraints can be understood as nonlinear eigenproblems. We derive a generalization of the inverse power method which is guaranteed to converge to a nonlinear eigenvector. We apply the inverse power method to 1-spectral clustering and sparse PCA which can naturally be formulated as nonlinear eigenproblems. In both applications we achieve state-of-the-art results in terms of solution quality and runtime. Moving beyond the standard eigenproblem should be useful also in many other applications and our inverse power method can be easily adapted to new problems. Comment: Long version of paper accepted at NIPS 2010
Neuroaesthetics is gaining momentum. At this early juncture, it is worth taking stock of where the field is and what lies ahead. Here, I review writings that fall under the rubric of neuroaesthetics. These writings include discussions of the parallel organizational principles of the brain and the intent and practices of artists, the description of informative anecdotes, and the emergence of experimental neuroaesthetics. I then suggest a few areas within neuroaesthetics that might be pursued profitably. Finally, I raise some challenges for the field. These challenges are not unique to neuroaesthetics. As neuroaesthetics comes of age, it might take advantage of the lessons learned from more mature domains of inquiry within cognitive neuroscience.