No file available
To read the file of this research,
you can request a copy directly from the authors.
Aesthetics Without Semantics
Preprints and early-stage research may not have been peer reviewed yet.
Abstract
While it is easy for human observers to judge an image as beautiful or ugly, aesthetic decisions result from a combination of entangled perceptual and cognitive (semantic) factors, making the understanding of aesthetic judgements particularly challenging from a scientific point of view. Furthermore, our research shows a prevailing bias in current databases, which include mostly beautiful images, further complicating the study and prediction of aesthetic responses. We address these limitations by creating a database of images with minimal semantic content and devising, and next exploiting, a method to generate images on the ugly side of aesthetic valuations. The resulting Minimum Semantic Content (MSC) database consists of a large and balanced collection of 10,426 images, each evaluated by 100 observers. We next use established image metrics to demonstrate how augmenting an image set biased towards beautiful images with ugly images can modify, or even invert, an observed relationship between image features and aesthetics valuation. Taken together, our study reveals that works in empirical aesthetics attempting to link image content and aesthetic judgements may magnify, underestimate, or simply miss interesting effects due to a limitation of the range of aesthetic values they consider.
ResearchGate has not been able to resolve any citations for this publication.
Aesthetic judgments were investigated using a combined nomothetic and idiographic approach. Participants judged novel graphic patterns with respect to their own personal definitions of “beauty” Judgment analysis was employed to derive individual case models of judgment strategies as well as a group model. As predicted, symmetry had the highest correlations with aesthetic judgments of beauty. Stimulus complexity was the second-highest correlate of a positive evaluation. Thus, there was agreement at the group level. The judgment analyses, however, indicated substantial individual differences. These included use of symmetry or complexity cues that were contrary to the main group use, e.g., a few participants considered nonsymmetric patterns more beautiful. These findings suggest that exclusive consideration of the group model would have leveled the individual differences and been misleading. The group model is significant; however, the individual judgment analyses represent individual patterns of judgment in a notedly more accurate way.
To what extent do aesthetic experiences arise from the human ability to perceive and extract meaning from visual features? Ordinary scenes, such as a beach sunset, can elicit a sense of beauty in most observers. Although it appears that aesthetic responses can be shared among humans, little is known about the cognitive mechanisms that underlie this phenomenon. We developed a contour model of aesthetics that assigns values to visual properties in scenes, allowing us to predict aesthetic responses in adults from around the world. Through a series of experiments, we manipulate contours to increase or decrease aesthetic value while preserving scene semantic identity. Contour manipulations directly shift subjective aesthetic judgments. This provides the first experimental evidence for a causal relationship between contour properties and aesthetic valuation. Our findings support the notion that visual regularities underlie the human capacity to derive pleasure from visual information.
Much of the neural machinery of the early visual cortex, from the extraction of local orientations to contextual modulations through lateral interactions, is thought to have developed to provide a sparse encoding of contour in natural scenes, allowing the brain to process efficiently most of the visual scenes we are exposed to. Certain visual stimuli, however, cause visual stress, a set of adverse effects ranging from simple discomfort to migraine attacks, and epileptic seizures in the extreme, all phenomena linked with an excessive metabolic demand. The theory of efficient coding suggests a link between excessive metabolic demand and images that deviate from natural statistics. Yet, the mechanisms linking energy demand and image spatial content in discomfort remain elusive. Here, we used theories of visual coding that link image spatial structure and brain activation to characterize the response to images observers reported as uncomfortable in a biologically based neurodynamic model of the early visual cortex that included excitatory and inhibitory layers to implement contextual influences. We found three clear markers of aversive images: a larger overall activation in the model, a less sparse response, and a more unbalanced distribution of activity across spatial orientations. When the ratio of excitation over inhibition was increased in the model, a phenomenon hypothesised to underlie interindividual differences in susceptibility to visual discomfort, the three markers of discomfort progressively shifted toward values typical of the response to uncomfortable stimuli. Overall, these findings propose a unifying mechanistic explanation for why there are differences between images and between observers, suggesting how visual input and idiosyncratic hyperexcitability give rise to abnormal brain responses that result in visual stress.
Preference for curvature, the curvature effect, seems to transcend cultures, species, and stimulus kinds. However, its nature and psychological mechanisms remain obscure because studies often overlook the complexity of contour characterization and disregard personal and contextual factors. To investigate the curvature effect, we propose a continuous and multidimensional manipulation and contrasting experimental conditions examined at the group and individual levels that unveil a complex picture, not reducible to monotonous relationships: Perceptual and hedonic evaluations relied on multiple geometric features defining contour and shape. These features were specifically weighted to characterize each construct, depending on the individual and contingent on whether evaluating perceptually or hedonically. Crucially, the curvature effect was not robust to preference with respect to the median and continuous manipulations of contour for varying shapes. As curved contours are more easily perceived and processed than polygons, we hypothesized that perceived contour might explain liking for a figure beyond the effect of geometric features, finding that this association was subordinated to shape categorizations. Finally, domain-specific, personality, and cognitive-preference traits moderated how people used each geometric feature in their perceptual and hedonic evaluations. We conclude that research on perception and appreciation of contour and shape should factor in their complexity and defining features. Additionally, embracing individual sensitivities opens potential avenues to advance the understanding of psychological phenomena. In summary, our approach unpacks a complex picture of contour preference that prompts critical reflections on past research and advice for future research, and it is applicable to other psychological constructs.
Art is embedded in its historical, social, political, and cultural context, and rarely evaluated in isolation. The semantic context created by providing text-based information about an artwork influences how an artwork will be evaluated. In the current study, we investigated how contextual information influences the aesthetic appreciation of artworks. Experiment 1 explored whether contextual information such as artist or technique information influenced aesthetic judgments of abstract artworks by Jackson Pollock. The combination of artist and technique information increased liking and interest for the artworks. Experiment 2 investigated whether contextual information about the artist, technique, or content of representational artworks by Indian and European/American artists influenced aesthetic responses of Northern American participants. We found that artist, content, and technique information compared to no information influenced the aesthetic experience of representational artworks. For both experiments, the effect of contextual information was stronger in participants with little art experience, and those more open to experience, and for artworks from another culture compared to one’s own. In sum, along-with theories of empirical and neuro-aesthetics, the current findings also have implications for aesthetics education and museum curation. It seems crucial to consider the type of artwork, the type of contextual information, its potential to enhance aesthetic experience, and the curatorial background of the museum or exhibition, as well as individual differences of viewers. Artworks that are unfamiliar to its viewers might require more contextual information to have an impact on the viewers, and may lower viewers’ prejudices against artworks/artists originating from an out-group.
Little is known about how the brain computes the perceived aesthetic value of complex stimuli such as visual art. Here, we used computational methods in combination with functional neuroimaging to provide evidence that the aesthetic value of a visual stimulus is computed in a hierarchical manner via a weighted integration over both low and high level stimulus features contained in early and late visual cortex, extending into parietal and lateral prefrontal cortices. Feature representations in parietal and lateral prefrontal cortex may in turn be utilized to produce an overall aesthetic value in the medial prefrontal cortex. Such brain-wide computations are not only consistent with a feature-based mechanism for value construction, but also resemble computations performed by a deep convolutional neural network. Our findings thus shed light on the existence of a general neurocomputational mechanism for rapidly and flexibly producing value judgements across an array of complex novel stimuli and situations.
It is an open question whether preferences for visual art can be lawfully predicted from the basic constituent elements of a visual image. Here, we developed and tested a computational framework to investigate how aesthetic values are formed. We show that it is possible to explain human preferences for a visual art piece based on a mixture of low- and high-level features of the image. Subjective value ratings could be predicted not only within but also across individuals, using a regression model with a common set of interpretable features. We also show that the features predicting aesthetic preference can emerge hierarchically within a deep convolutional neural network trained only for object recognition. Our findings suggest that human preferences for art can be explained at least in part as a systematic integration over the underlying visual features of an image.
Estimating the value of alternative options is a key process in decision-making. Human functional magnetic resonance imaging and monkey electrophysiology studies have identified brain regions, such as the ventromedial prefrontal cortex (vmPFC) and lateral orbitofrontal cortex (lOFC), composing a value system. In the present study, in an effort to bridge across species and techniques, we investigated the neural representation of value ratings in 36 people with epilepsy, using intracranial electroencephalography. We found that subjective value was positively reflected in both vmPFC and lOFC high-frequency activity, plus several other brain regions, including the hippocampus. We then demonstrated that subjective value could be decoded (1) in pre-stimulus activity, (2) for various categories of items, (3) even during a distractive task and (4) as both linear and quadratic signals (encoding both value and confidence). Thus, our findings specify key functional properties of neural value signals (anticipation, generality, automaticity, quadraticity), which might provide insights into human irrational choice behaviors.
Identifying relative idiosyncratic and shared contributions to judgments is a fundamental challenge to the study of human behavior, yet there is no established method for estimating these contributions. Using edge cases of stimuli varying in intrarater reliability and interrater agreement-faces (high on both), objects (high on the former, low on the latter), and complex patterns (low on both)-we showed that variance component analyses (VCAs) accurately captured the psychometric properties of the data (Study 1). Simulations showed that the VCA generalizes to any arbitrary continuous rating and that both sample and stimulus set size affect estimate precision (Study 2). Generally, a minimum of 60 raters and 30 stimuli provided reasonable estimates within our simulations. Furthermore, VCA estimates stabilized given more than two repeated measures, consistent with the finding that both intrarater reliability and interrater agreement increased nonlinearly with repeated measures (Study 3). The VCA provides a rigorous examination of where variance lies in data, can be implemented using mixed models with crossed random effects, and is general enough to be useful in any judgment domain in which agreement and disagreement are important to quantify and in which multiple raters independently rate multiple stimuli.
Gamut mapping is the problem of transforming the colors of image or video content so as to fully exploit the color palette of the display device where the content will be shown, while preserving the artistic intent of the original content's creator. In particular in the cinema industry, the rapid advancement in display technologies has created a pressing need to develop automatic and fast gamut mapping algorithms. In this paper we propose a novel framework that is based on vision science models, performs both gamut reduction and gamut extension, is of low computational complexity, produces results that are free from artifacts and outperforms state-of-the-art methods according to psychophysical tests. Our experiments also highlight the limitations of existing objective metrics for the gamut mapping problem.
A growing body of empirical evidence is revealing the value of nature experience for mental health. With rapid urbanization and declines in human contact with nature globally, crucial decisions must be made about how to preserve and enhance opportunities for nature experience. Here, we first provide points of consensus across the natural, social, and health sciences on the impacts of nature experience on cognitive functioning, emotional well-being, and other dimensions of mental health. We then show how ecosystem service assessments can be expanded to include mental health, and provide a heuristic, conceptual model for doing so.
Most people like symmetry, and symmetry has been extensively used in visual art and architecture. In this study, we compared preference for images of abstract and familiar objects in the original format or when containing perfect bilateral symmetry. We created pairs of images for different categories: male faces, female faces, polygons, smoothed version of the polygons, flowers, and landscapes. This design allows us to compare symmetry preference in different domains. Each observer saw all categories randomly interleaved but saw only one of the two images in a pair. After recording preference, we recorded a rating of how salient the symmetry was for each image, and measured how quickly observers could decide which of the two images in a pair was symmetrical. Results reveal a general preference for symmetry in the case of shapes and faces. For landscapes, natural (no perfect symmetry) images were preferred. Correlations with judgments of saliency were present but generally low, and for landscapes the salience of symmetry was negatively related to preference. However, even within the category where symmetry was not liked (landscapes), the separate analysis of original and modified stimuli showed an interesting pattern: Salience of symmetry was correlated positively (artificial) or negatively (original) with preference, suggesting different effects of symmetry within the same class of stimuli based on context and categorization.
Symmetry is effortlessly perceived by humans across changes in viewing geometry. Here, we re‐examined the network subserving symmetry processing in the context of up‐to‐date retinotopic definitions of visual areas. Responses in object selective cortex, as defined by functional localizers, were also examined. We further examined responses to both frontoparallel and slanted symmetry while manipulating attention both toward and away from symmetry. Symmetry‐specific responses first emerge in V3 and continue across all downstream areas examined. Of the retinotopic areas, ventral occipital VO1 showed the strongest symmetry response, which was similar in magnitude to the responses observed in object selective cortex. Neural responses were found to increase with both the coherence and folds of symmetry. Compared to passive viewing, drawing attention to symmetry generally increased neural responses and the correspondence of these neural responses with psychophysical performance. Examining symmetry on the slanted plane found responses to again emerge in V3, continue through downstream visual cortex, and be strongest in VO1 and LOB. Both slanted and frontoparallel symmetry evoked similar activity when participants performed a symmetry‐related task. However, when a symmetry‐unrelated task was performed, fMRI responses to slanted symmetry were reduced relative to their frontoparallel counterparts. These task‐related changes provide a neural signature that suggests slant has to be computed ahead of symmetry being appropriately extracted, known as the “normalization” account of symmetry processing. Specifically, our results suggest that normalization occurs naturally when attention is directed toward symmetry and orientation, but becomes interrupted when attention is directed away from these features.
Scientific disciplines as diverse as biology, physics, and psychological aesthetics regard symmetry as one of the most important principles in nature and one of the most powerful determinants of beauty. However, symmetry has a low standing in the arts and humanities. This difference in the valuation of symmetry is a remarkable illustration of the gap between the two cultures. To close this gap, we conducted an interdisciplinary, empirical study to directly demonstrate the effects of art expertise on symmetry appreciation. Two groups of art experts—artists and art historians—and a group of non-experts provided spontaneous beauty ratings of visual stimuli that varied in symmetry and complexity. In complete contrast to responses typically found in non-art experts, art experts found asymmetrical and simple stimuli as most beautiful. This is evidence of the effects of specific education and training on aesthetic appreciation and a direct challenge to the universality of symmetry.
Recently, there is a rising interest in perceiving image aesthetics. The existing works deal with image aesthetics as a classification or regression problem. To extend the cognition from rating to reasoning, a deeper understanding of aesthetics should be based on revealing why a high- or low-aesthetic score should be assigned to an image. From such a point of view, we propose a model referred to as Neural Aesthetic Image Reviewer, which can not only give an aesthetic score for an image, but also generate a textual description explaining why the image leads to a plausible rating score. Specifically, we propose two multi-task architectures based on shared aesthetically semantic layers and task-specific embedding layers at a high level for performance improvement on different tasks. To facilitate researches on this problem, we collect the AVA-Reviews dataset, which contains 52,118 images and 312,708 comments in total. Through multi-task learning, the proposed models can rate aesthetic images as well as produce comments in an end-to-end manner. It is confirmed that the proposed models outperform the baselines according to the performance evaluation on the AVA-Reviews dataset. Moreover, we demonstrate experimentally that our model can generate textual reviews related to aesthetics, which are consistent with human perception.
Aesthetics has been the subject of long-standing debates by philosophers and psychologists alike. In psychology, it is generally agreed that aesthetic experience results from an interaction between perception, cognition, and emotion. By experimental means, this triad has been studied in the field of experimental aesthetics, which aims to gain a better understanding of how aesthetic experience relates to fundamental principles of human visual perception and brain processes. Recently, researchers in computer vision have also gained interest in the topic, giving rise to the field of computational aesthetics. With computing hardware and methodology developing at a high pace, the modeling of perceptually relevant aspect of aesthetic stimuli has a huge potential. In this review, we present an overview of recent developments in computational aesthetics and how they relate to experimental studies. In the first part, we cover topics such as the prediction of ratings, style and artist identification as well as computational methods in art history, such as the detection of influences among artists or forgeries. We also describe currently used computational algorithms, such as classifiers and deep neural networks. In the second part, we summarize results from the field of experimental aesthetics and cover several isolated image properties that are believed to have a effect on the aesthetic appeal of visual stimuli. Their relation to each other and to findings from computational aesthetics are discussed. Moreover, we compare the strategies in the two fields of research and suggest that both fields would greatly profit from a joined research effort. We hope to encourage researchers from both disciplines to work more closely together in order to understand visual aesthetics from an integrated point of view.
Emerging display technologies are able to produce images with a much wider color gamut than those of conventional distribution gamuts for cinema and TV, creating an opportunity for the development of gamut extension algorithms (GEAs) that exploit the full color potential of these new systems. In this paper we present a novel GEA, implemented as a PDE-based optimization procedure related to visual perception models, that performs gamut extension by taking into account the analysis of distortions in hue, chroma and saturation. User studies performed using a digital cinema projector under cinematic (low ambient light, large screen) conditions show that the proposed algorithm outperforms the state of the art, producing gamut extended images that are perceptually more faithful to the widegamut ground truth, as well as free of color artifacts and hue shifts. We also show how currently available image quality metrics, when applied to the gamut extension problem, provide results that do not correlate with users’ choices.
We propose a method for measuring symmetry in images by using filter responses from Convolutional Neural Networks (CNNs). The aim of the method is to model human perception of left/right symmetry as closely as possible. Using the Convolutional Neural Network (CNN) approach has two main advantages: First, CNN filter responses closely match the responses of neurons in the human visual system; they take information on color, edges and texture into account simultaneously. Second, we can measure higher-order symmetry, which relies not only on color, edges and texture, but also on the shapes and objects that are depicted in images. We validated our algorithm on a dataset of 300 music album covers, which were rated according to their symmetry by 20 human observers, and compared results with those from a previously proposed method. With our method, human perception of symmetry can be predicted with high accuracy. Moreover, we demonstrate that the inclusion of features from higher CNN layers, which encode more abstract image content, increases the performance further. In conclusion, we introduce a model of left/right symmetry that closely models human perception of symmetry in CD album covers.
This study evaluated whether music-induced aesthetic “chill” responses, which typically correspond to peak emotional experiences, can be objectively monitored by degree of pupillary dilation. Participants listened to self-chosen songs versus control songs chosen by other participants. The experiment included an active condition where participants made key presses to indicate when experiencing chills and a passive condition (without key presses). Chills were reported more frequently for self-selected songs than control songs. Pupil diameter was concurrently measured by an eye-tracker while participants listened to each of the songs. Pupil size was larger within specific time-windows around the chill events, as monitored by key responses, than in comparison to pupil size observed during ‘passive’ song listening. In addition, there was a clear relationship between pupil diameter within the chills-related time-windows during both active and passive conditions, thus ruling out the possibility that chills-related pupil dilations were an artifact of making a manual response. These findings strongly suggest that music chills can be visible in the moment-to-moment changes in the size of pupillary responses and that a neuromodulatory role of the central norepinephrine system is thereby implicated in this phenomenon.
The aesthetic quality of a picture is automatically inferred using visual content as a machine learning problem using, for example, a peer-rated, on-line photo sharing Website as data source. Certain visual features of images are extracted based on the intuition that they can discriminate between aesthetically pleasing and displeasing images. A one-dimensional support vector machine is used to identify features that have noticeable correlation with the community-based aesthetics ratings. Automated classifiers are constructed using the support vector machines and classification trees, with a simple feature selection heuristic being applied to eliminate irrelevant features. Linear regression on polynomial terms of the features is also applied to infer numerical aesthetics ratings.
In this work, I propose a model of visual aesthetic experience that combines formalist and contextual aspects of aesthetics. The model distinguishes between two modes of processing. First, perceptual processing is based on the intrinsic form of an artwork, which may or may not be beautiful. If it is beautiful, a beauty-responsive mechanism is activated in the brain. This bottom–up mechanism is universal amongst humans; it is widespread in the visual brain and responsive across visual modalities. Second, cognitive processing is based on contextual information, such as the depicted content, the intentions of the artist or the circumstances of the presentation of the artwork. Cognitive processing is partially top–down and varies between individuals according to their cultural experience. Processing in the two channels is parallel and largely independent. In the general case, an aesthetic experience is induced if processing in both channels is favorable, i.e., if there is resonance in the perceptual processing channel (“aesthetics of perception”), and successful mastering in the cognitive processing channel (“aesthetics of cognition”). I speculate that this combinatorial mechanism has evolved to mediate social bonding between members of a (cultural) group of people. Primary emotions can be elicited via both channels and modulate the degree of the aesthetic experience. Two special cases are discussed. First, in a subset of (post-)modern art, beauty no longer plays a prominent role. Second, in some forms of abstract art, beautiful form can be enjoyed with minimal cognitive processing. The model is applied to examples of Western art. Finally, implications of the model are discussed. In summary, the proposed model resolves the seeming contradiction between formalist perceptual approaches to aesthetic experience, which are based on the intrinsic beauty of artworks, and contextual approaches, which account for highly individual and culturally dependent aspects of aesthetics.
Humans have a preference for curved over angular shapes, an effect noted by artists as well as scientists. It may be that people like smooth curves or that people dislike angles, or both. We investigated this phenomenon in four experiments. Using abstract shapes differing in type of contour (angular vs. curved) and complexity, Experiment 1 confirmed a preference for curvature not linked to perceived complexity. Experiment 2 tested whether the effect was modulated by distance. If angular shapes are associated with a threat, the effect may be stronger when they are presented within peripersonal space. This hypothesis was not supported. Experiment 3 tested whether preference for curves occurs when curved lines are compared to straight lines without angles. Sets of coloured lines (angular vs. curved vs. straight) were seen through a circular or square aperture. Curved lines were liked more than either angular or straight lines. Therefore, angles are not necessary to generate a preference for curved shapes. Finally, Experiment 4 used an implicit measure of preference, the manikin task, to measure approach/avoidance behaviour. Results did not confirm a pattern of avoidance for angularity but only a pattern of approach for curvature. Our experiments suggest that the threat association hypothesis cannot fully explain the curvature effect and that curved shapes are, per se, visually pleasant.
© 2015 The Authors. British Journal of Psychology published by John Wiley & Sons Ltd on behalf of the British Psychological Society.
Quite independently of what they represent, some images provoke discomfort, and even headaches and seizures in susceptible individuals. The visual system has adapted to efficiently process the images it typically experiences, and in nature these images are usually scale-invariant. In this work, we sought to characterize the images responsible for discomfort in terms of their adherence to low-level statistical properties typically seen in natural scenes. It has been conventional to measure scale invariance in terms of the one-dimensional Fourier amplitude spectrum, by averaging amplitude over orientations in the Fourier domain. However, this loses information on the evenness with which information at various orientations is represented. We therefore fitted a two-dimensional surface (regular circular cone 1/f in logarithmic coordinates) to the two-dimensional amplitude spectrum. The extent to which the cone fitted the spectrum explained an average of 18% of the variance in judgments of discomfort from images including rural and urban scenes, works of non-representational art, images of buildings and animals, and images generated from randomly disposed discs of varying contrast and size. Weighting the spectrum prior to fitting the surface to allow for the spatial frequency tuning of contrast sensitivity explained an average of 27% of the variance. Adjusting the shape of the cone to take account of the generally greater energy in horizontal and vertical orientations improved the fit, but only slightly. Taken together, our findings show that a simple measure based on first principles of efficient coding and human visual sensitivity explained more variance than previously published algorithms. The algorithm has a low computational cost and we show that it can identify the images involved in cases that have reached the media because of complaints. We offer the algorithm as a tool for designers rather than as a simulation of the biological processes involved.
In this study, we combined the behavioral and objective approach in the field of empirical aesthetics. First, we studied the perception of beauty by investigating shifts in evaluation on perceived beauty of abstract artworks (Experiment 1). Because the participants showed heterogeneous individual preferences for the paintings, we divided them into seven clusters for the test. The experiment revealed a clear pattern of perceptual contrast. The perceived beauty of abstract paintings increased after exposure to paintings that were rated as less beautiful, and it decreased after exposure to paintings that were rated as more beautiful. Next, we searched for correlations of beauty ratings and perceptual contrast with statistical properties of abstract artworks (Experiment 2). The participants showed significant preferences for particular image properties. These preferences differed between the clusters of participants. Strikingly, next to color measures like hue, saturation, value and lightness, the recently described Pyramid of Histograms of Orientation Gradients (PHOG) self-similarity value seems to be a predictor for aesthetic appreciation of abstract artworks. We speculate that the shift in evaluation in Experiment 1 was, at least in part, based on low-level adaptation to some of the statistical image properties analyzed in Experiment 2. In conclusion, our findings demonstrate that the perception of beauty in abstract artworks is altered after exposure to beautiful or non-beautiful images and correlates with particular image properties, especially color measures and self-similarity.
In a task of rating images of artworks in an fMRI scanner, regions in the medial prefrontal cortex that are known to be part of the default mode network (DMN) were positively activated on the highest-rated trials. This is surprising given the DMN's original characterization as the set of brain regions that show greater fMRI activity during rest periods than during performance of tasks requiring focus on external stimuli. But further research showed that DMN regions could be positively activated also in structured tasks, if those tasks involved self-referential thought or self-relevant information. How may our findings be understood in this context? Although our task had no explicit self-referential aspect and the stimuli had no a priori self-relevance to the observers, the experimental design we employed emphasized the personal aspects of aesthetic experience. Observers were told that we were interested in their individual tastes, and asked to base their ratings on how much each artwork “moved” them. Moreover, we used little-known artworks that covered a wide range of styles, which led to high individual variability: each artwork was rated highly by some observers and poorly by others. This means that rating-specific neural responses cannot be attributed to the features of any particular artworks, but rather to the aesthetic experience itself. The DMN activity therefore suggests that certain artworks, albeit unfamiliar, may be so well-matched to an individual's unique makeup that they obtain access to the neural substrates concerned with the self—access which other external stimuli normally do not get. This mediates a sense of being “moved,” or “touched from within.” This account is consistent with the modern notion that individuals' taste in art is linked with their sense of identity, and suggests that DMN activity may serve to signal “self-relevance” in a broader sense than has been thought so far.
The visual system processes natural scenes in a split second. Part of this process is the extraction of "gist," a global first impression. It is unclear, however, how the human visual system computes this information. Here, we show that, when human observers categorize global information in real-world scenes, the brain exhibits strong sensitivity to low-level summary statistics. Subjects rated a specific instance of a global scene property, naturalness, for a large set of natural scenes while EEG was recorded. For each individual scene, we derived two physiologically plausible summary statistics by spatially pooling local contrast filter outputs: contrast energy (CE), indexing contrast strength, and spatial coherence (SC), indexing scene fragmentation. We show that behavioral performance is directly related to these statistics, with naturalness rating being influenced in particular by SC. At the neural level, both statistics parametrically modulated single-trial event-related potential amplitudes during an early, transient window (100-150 ms), but SC continued to influence activity levels later in time (up to 250 ms). In addition, the magnitude of neural activity that discriminated between man-made versus natural ratings of individual trials was related to SC, but not CE. These results suggest that global scene information may be computed by spatial pooling of responses from early visual areas (e.g., LGN or V1). The increased sensitivity over time to SC in particular, which reflects scene fragmentation, suggests that this statistic is actively exploited to estimate scene naturalness.
A low-cost, portable, video-camera system built by University of Bristol for the UK-DRA, RARDE Fort Halstead, permits in-field acquisition of terrestrial hyper-spectral image sets. Each set is captured as a sequence of thirty-one images through a set of different interference filters which span the visible spectrum, at 10 nm intervals: effectively providing a spectrogram of 256 by 256 pixels. The system is customized from off-the-shelf components. A database of twenty-nine hyper-spectral images sets was acquired and analyzed as a sample of natural environment. We report the manifest information capacity with respect to spatial and optical frequency drawing implications for management of hyper-spectral data and visual processing.
In order to transmit information in images efficiently, the visual system should be tuned to the statistical structure of the ensemble of images that it sees. Several authors have suggested that the ensemble of natural images exhibits fractal behavior and, therefore, has a power spectrum that drops off proportionally to 1/fβ(2 < β < 4). In this paper we investigate the question of which value of the exponent β describes the power spectrum of the ensemble of images to which the visual system is optimally tuned. An experiment in which subjects were asked to discriminate randomly generated noise textures based on their spectral drop-off was used. Whereas the discrimination-threshold function of an ideal observer was flat for different spectral drop-offs, human observers showed a broad peak in sensitivity for 2.8 < β < 3.6. The results are consistent with, but do not provide direct evidence for, the theory that the visual system is tuned to an ensemble of images with Markov statistics.
Image Aesthetic Quality Assessment (IAQA) spans applications such as the fashion industry, AI-generated content, product design, and e-commerce. Recent deep learning advancements have been employed to evaluate image aesthetic quality. A few surveys have been conducted on IAQA models; however, details of recent deep learning models and challenges have not been fully mentioned. This paper aims to fill these gaps by providing a review of deep learning IAQA over the past decade, based on input, process, and output phases. Methodologies for deep learning-based IAQA can be categorized into general and task-specific approaches, depending on the type and diversity of input images. The processing phase involves considerations related to network architecture, learning structures, and feature extraction methods. The output phase generates results such as scoring, distribution, attributes, and description. Despite achieving a maximum accuracy of 91.5%, further improvements in deep learning models are still required. Our study highlights several challenges, including adapting models for task-specific methodology, accounting for environmental factors influencing aesthetics, the lack of substantial datasets with appropriate labels, imbalanced data, preserving image aspect ratio and integrity in network architecture design, and the need for explainable AI to understand the causative factors behind aesthetic judgments.
As one of the earliest domains of aesthetic experience to be studied systematically, there now exists a sizeable body of work on the behavioral basis and neural correlates of visual aesthetic appeal.
Synthesizing work across approaches that seek to explain aesthetic preferences based on objectively measurable stimulus features or that emphasize individual differences and the importance of subjective factors, we adopt the view that aesthetic appeal arises from the interaction of a specific stimulus with the characteristics of a specific observer.
Neuroscience research across a number of visual aesthetic domains, including faces, landscapes, architecture, artwork and dance reveals that several large-scale brain systems contribute to visual aesthetic appreciation. In the visual system, judgments of visual appeal are determined more by higher-level visual processing than low-level processing. Subcortically, both ventral and dorsal striatum, as well as the amygdala have been implicated in judgments of aesthetic appeal. A large number of studies using a variety of methods have found correlates of aesthetic appeal in prefrontal cortex, most prominently in medial prefrontal and orbital cortex. Other regions, including inferior frontal gyrus and the insula may also play a role for at least some domains, and there is evidence that the default-mode network, a brain system thought to support aspects of internally directed mentation, is engaged for highly moving artworks.
While the question of ‘why’ particular individuals like particular images remains largely unanswered, we argue for a perspective that deemphasizes the role of specific visual features and focuses instead on the potential for aesthetic appeal to signal the presence of learnable information, linking visual aesthetic appeal to the intrinsic motivation to make sense of our visual world.
The Routledge International Handbook of Neuroaesthetics is an authoritative reference work that provides the reader with a wide-ranging introduction to this exciting new scientific discipline. The book brings together leading international academics to offer a well-balanced overview of this burgeoning field while addressing two questions central to the field: how the brain computes aesthetic appreciation for sensory objects and how art is created and experienced.
The editors, Martin Skov and Marcos Nadal, have compiled a neuroscientific, physiological, and psychological overview of the systems underlying the evaluation of sensory objects and aesthetic appreciation. Covering a variety of art forms mediated by vision, audition, movement, and language, the handbook puts forward a critical review of the current research to explain how and why perceptual and emotional processes are essential for art production. The work also unravels the interaction of art with expectations, experience and knowledge and the modulation of artistic appreciation through social and contextual settings, eventually bringing to light the potential of art to influence mental states, health, and well-being. The concepts are presented through research on the neural processes enabling artistic creativity, artistic expertise, and the evolution of symbolic cognition.
This handbook is a compelling read for anyone interested in making a first venture into this exciting new area of study and is best suited for students and researchers in the fields of neuroaesthetics, perceptual learning, and cognitive psychology.
At present, most of the research on image aesthetics focuses on scoring pictures. We propose Aesthetic Assessment of Images, which means the dense aesthetic captioning. The image captioning model uses many photos for www.flickr.com as a training set, including lots of captions about every photo meantime. Then we use LDA for aesthetic topics and active learning to screen sentence. The remaining sentence dataset, we called FAE-Captions, is a professional dataset on the topic of image aesthetics. Put the dataset into a CNN-LSTM model for predicting and generating answers. In order to let neural network connected with aesthetics, we propose an aesthetic loss function for many aesthetics topics additionally.
We addressed the question of the extent to which external information is capable of modifying aesthetic ratings given to two different categories of stimuli—images of faces (which belong to the biological category) and those of abstract paintings with no recognizable objects (which sit in the artifactual category). A total of 51 participants of different national origins rated the beauty of both sets of stimuli, indicating the certainty of their rating; they then re‐rated them after being exposed to the opinions of others on their aesthetic status. Of these 51 participants, 42 who met our criteria were selected to complete the experiment. The results showed that individuals were less prone to modifying their ratings of stimuli belonging to the biological category compared to those falling into the artifactual category. We discuss this finding in light of our theoretical Bayesian–Laplacian model and on the evidence given by previous empirical research.
Visual aesthetic evaluations, which impact decision-making and well-being, recruit the ventral visual pathway, subcortical reward circuitry, and parts of the medial prefrontal cortex overlapping with the default-mode network (DMN). However, it is unknown whether these networks represent aesthetic appeal in a domain-general fashion, independent of domain-specific representations of stimulus content (artworks versus architecture or natural landscapes). Using a classification approach, we tested whether the DMN or ventral occipitotemporal cortex (VOT) contains a domain-general representation of aesthetic appeal. Classifiers were trained on multivoxel functional MRI response patterns collected while observers made aesthetic judgments about images from one aesthetic domain. Classifier performance (high vs. low aesthetic appeal) was then tested on response patterns from held-out trials from the same domain to derive a measure of domain-specific coding, or from a different domain to derive a measure of domain-general coding. Activity patterns in category-selective VOT contained a degree of domain-specific information about aesthetic appeal, but did not generalize across domains. Activity patterns from the DMN, however, were predictive of aesthetic appeal across domains. Importantly, the ability to predict aesthetic appeal varied systematically; predictions were better for observers who gave more extreme ratings to images subsequently labeled as “high” or “low.” These findings support a model of aesthetic appreciation whereby domain-specific representations of the content of visual experiences in VOT feed in to a “core” domain-general representation of visual aesthetic appeal in the DMN. Whole-brain “searchlight” analyses identified additional prefrontal regions containing information relevant for appreciation of cultural artifacts (artwork and architecture) but not landscapes.
Neuroaesthetics is a rapidly developing interdisciplinary field of research that aims to understand the neural substrates of aesthetic experience: While understanding aesthetic experience has been an objective of philosophers for centuries, it has only more recently been embraced by neuroscientists. Recent work in neuroaesthetics has revealed that aesthetic experience with static visual art engages visual, reward and default-mode networks. Very little is known about the temporal dynamics of these networks during aesthetic appreciation. Previous behavioral and brain imaging research suggests that critical aspects of aesthetic experience have slow dynamics, taking more than a few seconds, making them amenable to study with fMRI. Here, we identified key aspects of the dynamics of aesthetic experience while viewing art for various durations. In the first few seconds following image onset, activity in the DMN (and high-level visual and reward regions) was greater for very pleasing images; in the DMN this activity counteracted a suppressive effect that grew longer and deeper with increasing image duration. In addition, for very pleasing art, the DMN response returned to baseline in a manner time-locked to image offset. Conversely, for non-pleasing art, the timing of this return to baseline was inconsistent. This differential response in the DMN may therefore reflect the internal dynamics of the participant's state: The participant disengages from art-related processing and returns to stimulus-independent thought. These dynamics suggest that the DMN tracks the internal state of a participant during aesthetic experience.
In recent years, many new cortical areas have been identified in the macaque monkey. The number of identified connections between areas has increased even more dramatically. We report here on (1) a summary of the layout of cortical areas associated with vision and with other modalities, (2) a computerized database for storing and representing large amounts of information on connectivity patterns, and (3) the application of these data to the analysis of hierarchical organization of the cerebral cortex. Our analysis concentrates on the visual system, which includes 25 neocortical areas that are predominantly or exclusively visual in function, plus an additional 7 areas that we regard as visual-association areas on the basis of their extensive visual inputs. A total of 305 connections among these 32 visual and visual-association areas have been reported. This represents 31% of the possible number of pathways it each area were connected with all others. The actual degree of connectivity is likely to be closer to 40%. The great majority of pathways involve reciprocal connections between areas. There are also extensive connections with cortical areas outside the visual system proper, including the somatosensory cortex, as well as neocortical, transitional, and archicortical regions in the temporal and frontal lobes. In the somatosensory/motor system, there are 62 identified pathways linking 13 cortical areas, suggesting an overall connectivity of about 40%. Based on the laminar patterns of connections between areas, we propose a hierarchy of visual areas and of somato sensory/motor areas that is more comprehensive than those suggested in other recent studies. The current version of the visual hierarchy includes 10 levels of cortical processing. Altogether, it contains 14 levels if one includes the retina and lateral geniculate nucleus at the bottom as well as the entorhinal cortex and hippocampus at the top. Within this hierarchy, there are multiple, intertwined processing streams, which, at a low level, are related to the compartmental organization of areas V1 and V2 and, at a high level, are related to the distinction between processing centers in the temporal and parietal lobes. However, there are some pathways and relationships (about 10% of the total) whose descriptions do not fit cleanly into this hierarchical scheme for one reason or another. In most instances, though, it is unclear whether these represent genuine exceptions to a strict hierarchy rather than inaccuracies or uncertainties in the reported assignment.
This paper investigates unified feature learning and classifier training approaches for image aesthetics assessment. Existing methods built upon handcrafted or generic image features and developed machine learning and statistical modeling techniques utilizing training examples. We adopt a novel deep neural network approach to allow unified feature learning and classifier training to estimate image aesthetics. In particular, we develop a double-column deep convolutional neural network to support heterogeneous inputs, i.e., global and local views, in order to capture both global and local characteristics of images. In addition, we employ the style and semantic attributes of images to further boost the aesthetics categorization performance. Experimental results show that our approach produces significantly better results than the earlier reported results on the AVA dataset for both the generic image aesthetics and content -based image aesthetics. Moreover, we introduce a 1.5-million image dataset (IAD) for image aesthetics assessment and we further boost the performance on the AVA test set by training the proposed deep neural networks on the IAD dataset.
In this study, professional art experts and nonexperts with an active interest in art rated sets of 10-20 slides of artworks made available by young artists. Each set was rated on bipolar scales, including not original - original; absence of craftsmanship - craftsmanship; and poor quality - good quality. Intraclass coefficients Ri for these three scales were .17, .21, and .22 for experts and .19, .08, and .16 for nonexperts, respectively. There was a significant agreement between experts and nonexperts with respect to originality, but no agreement with respect to craftsmanship and quality. The correlation between originality and quality was significantly (p < .01) higher for experts (r = .88) than for nonexperts (r = .40). Thus, experts seem to attach much more value to originality in determining aesthetic quality than nonexperts.
Contrast effects occur when an individual judges a target stimulus as being further removed from its position on some dimension than it actually is, due to exposure to a context stimulus presenting an opposite value on this dimension. In assimilation effects, on the contrary, the reverse occurs such that judgment about the target is brought nearer to the context stimulus. The objective of this article is to verify whether assimilation and contrast can be observed in aesthetic evaluation of visual artworks. The results demonstrated that when the context stimulus was formally similar to one of two artworks used in the comparison (the target) but aesthetically slightly inferior to it, an assimilation effect was observed. In contrast, when the context stimulus was formally similar to the target but definitely of inferior aesthetic quality, a contrast effect was observed. These results demonstrate the impact of contextual factors on aesthetic judgment.
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif-ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make train-ing faster, we used non-saturating neurons and a very efficient GPU implemen-tation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
With the ever-expanding volume of visual content available, the ability to organize and navigate such content by aesthetic preference is becoming increasingly important. While still in its nascent stage, research into computational models of aesthetic preference already shows great potential. However, to advance research, realistic, diverse and challenging databases are needed. To this end, we introduce a new large-scale database for conducting Aesthetic Visual Analysis: AVA. It contains over 250,000 images along with a rich variety of meta-data including a large number of aesthetic scores for each image, semantic labels for over 60 categories as well as labels related to photographic style. We show the advantages of AVA with respect to existing databases in terms of scale, diversity, and heterogeneity of annotations. We then describe several key insights into aesthetic preference afforded by AVA. Finally, we demonstrate, through three applications, how the large scale of AVA can be leveraged to improve performance on existing preference tasks.
