Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
With the increase in massive digitized datasets of cultural artefacts, social and cultural scientists have an unprecedented opportunity for the discovery and expansion of cultural theory. The WikiArt dataset is one such example, with over 250,000 high quality images of historically significant artworks by over 3000 artists, ranging from the 15th ce...
Context in source publication
Context 1
... dataset can search across artist, genre, style, and time period, retrieving similar images in each of these categories. In Figure 1, we show search results for Picasso's The Old Guitarist, painted during his blue period, based on color similarity, and style similarity. We note that what is retrieved is the artwork meta-data, and not the image of the artwork itself. ...
Citations
... Research in quantitative and computational aesthetics [21][22][23] , as well as the interplay of computation and human cultures 24, 25 , requires reliable benchmark datasets that are interpretable by machines and humans. Previous work has relied on embeddings of large amounts of well known artworks 26,27 or synthetic datasets of limited size [28][29][30][31] . ...
The notion of visual similarity is essential for computer vision, and in applications and studies revolving around vector embeddings of images. However, the scarcity of benchmark datasets poses a significant hurdle in exploring how these models perceive similarity. Here we introduce Style Aligned Artwork Datasets (SALADs), and an example of fruit-SALAD with 10,000 images of fruit depictions. This combined semantic category and style benchmark comprises 100 instances each of 10 easy-to-recognize fruit categories, across 10 easy distinguishable styles. Leveraging a systematic pipeline of generative image synthesis, this visually diverse yet balanced benchmark demonstrates salient differences in semantic category and style similarity weights across various computational models, including machine learning models, feature extraction algorithms, and complexity measures, as well as conceptual models for reference. This meticulously designed dataset offers a controlled and balanced platform for the comparative analysis of similarity perception. The SALAD framework allows the comparison of how these models perform semantic category and style recognition task to go beyond the level of anecdotal knowledge, making it robustly quantifiable and qualitatively interpretable.
... In addition, in Quadrant 2, the emerging term "Art History" stands out, where some studies propose a novel approach through the use of "Wikiartvectors", a tool that represents styles and colors of works of art for cultural analysis through information theory measures, also referring to the fact that a deep understanding of art history and its relationship with culture is essential for the development of machine learning models capable of capturing the essence and intrinsic meaning of artistic expressions (Srinivasa Desikan et al. 2022). ...
In the field of art, machine learning models have been used to predict artistic styles in paintings. The foregoing is somewhat advantageous for analysts, as these tools can provide more valuable results and help reduce bias in the results and conclusions provided. Therefore, the objective of this research was to examine research trends in the use of machine learning to predict artistic styles from a bibliometric review based on the PRISMA methodology. From the search equations, 268 documents were found, out of which, following the application of inclusion and exclusion criteria, 128 documents were analyzed. Through quantitative analysis, a growing research interest in the subject is evident, progressing from user perception approaches to the utilization of tools like deep learning for art studies. Among the main results, it is possible to identify that one of the most used techniques in the field has been neural networks for pattern recognition. Also, a large part of the research focuses on the use of design software for image creation and manipulation. Finally, it is found that the number of studies focused on contemporary modern art is still limited, this is due to the fact that a large part of the investigations has focused on historical artistic styles.
... One of the annotation examples is shown in Figure 5 and Table 3. WikiArt Emotions Dataset includes images and text information about the artworks, as well as the emotion tags and ratings obtained from the annotators. Some studies use WikiArt Emotion Dataset for emotion recognition [36] or style and color representation analysis [37]. Our purpose is to propose a method that can be broadly applicable to artwork recommendation and use the WikiArt Emotion Dataset to evaluate the proposed method. ...
Recommender systems help users obtain the content they need from massive amounts of information. Artwork recommender systems is a topic that has attracted attention. However, existing art recommender systems rarely consider user preferences and multimodal information at the same time, while utilizing all the information has the potential to help make better personalized recommendations. To better apply recommender systems to the artwork-recommendation scenario, we propose a new neural topic modeling (NTM)-based multimodal artwork recommender system (MultArtRec), that can take all the information into account at the same time and extract effective features representing user preferences from multimodal content. Also, to improve MultArtRec’s performance on monomodal feature extraction, we add a novel topic loss term to the conventional NTM loss. The first two experiments in this study compare the performances of different models with different monomodal inputs. The results show that MultArtRec can improve the performance with image modality inputs by up to 174.8% compared to the second-best model and improve the performance with text modality inputs by up to 10.7% compared to the second-best model. The third experiment is conducted to compare the performance of MultArtRec with monomodal inputs and multimodal inputs. The results show that the performance of MultArtRec with multimodal inputs can be improved by up to 15.9% compared to monomodal inputs. The last experiment preliminarily tests the versatility of MultArtRec on a fashion recommendation scenario that considers clothing image content and user preferences. The results show that MultArtRec outperforms the other methods across all the metrics.
... The quantification of visual aesthetics and artistic expression goes back to Birkhoff [1] and Bense [2], inspiring several computational approaches [3][4][5][6][7][8][9][10][11][12][13][14][15][16]. Research drawing on information theory has shown repeatedly and in parallel that visual complexity can be estimated with some accuracy using compression algorithms such as zip or gif [17][18][19][20][21][22][23][24][25][26][27][28][29]. ...
... This yields a vector of compression ratios. Further statistical transformations such as colorfulness metrics and fractal dimension [5,16,39,40] (compressions in a broader sense) can also be added to these vectors, and rescaled if the magnitudes differ, which we do (see Methods). Given the perspective of artists as (or as executing) algorithms -in the very broadest sense -our approach aims to capture the residual signal of the generating process, the "algorithmic fingerprints" of an artwork, through the operationalization of various aspects of visual aesthetic complexity, as estimated by compression ratios of the visual transformations. ...
... A scene in a film or a recorded theater play could be represented by the concatenation of a visual compression ensemble, an audio compression ensemble, an image embedding, and a language model embedding [90] of the spoken dialogue. A compression ensemble could also be combined with the larger apparatus of art history, in the form of socio-cultural context information from relevant databases and knowledge graphs [91] or any other visual feature vectors [13,16,92]. Furthermore, features or objects (e.g. ...
To the human eye, different images appear more or less complex, but capturing this intuition in a single aesthetic measure is considered hard. Here, we propose a computationally simple, transparent method for modeling aesthetic complexity as a multidimensional algorithmic phenomenon, which enables the systematic analysis of large image datasets. The approach captures visual family resemblance via a multitude of image transformations and subsequent compressions, yielding explainable embeddings. It aligns well with human judgments of visual complexity, and performs well in authorship and style recognition tasks. Showcasing the functionality, we apply the method to 125,000 artworks, recovering trends and revealing new insights regarding historical art, artistic careers over centuries, and emerging aesthetics in a contemporary NFT art market. Our approach, here applied to images but applicable more broadly, provides a new perspective to quantitative aesthetics, connoisseurship, multidimensional meaning spaces, and the study of cultural complexity.
Sustainability in art is crucial for fostering intercultural understanding and preserving cultural heritage, which is essential to promoting the Sustainable Development Goals (SDGs). In previous research on Western and Chinese art, studies typically focused on individual artists and summarized their aesthetic values, often suffering from a lack of comparative analysis, a unidimensional sensory perspective, and a deficiency in comprehensive aesthetic evaluation standards. Consequently, this study selected representative watercolor works from five master artists in Western and Chinese art history as an example, constructing a comprehensive aesthetic evaluation system focusing on composition, color, themes, and techniques. Beginning with the influence of aesthetic and non-aesthetic perspectives by natural experimental method, the research employs correlation analysis and structural equation modeling to analyze the functional relationships between evaluative factors, perspective forces, and the overall aesthetic appeal of the paintings. Furthermore, the study reveals the distinctions between Eastern and Western paintings through multi-group analysis. Key findings include the following: Evaluation factors have varying effects on the influence of aesthetic and non-aesthetic perspectives. All factors, except conceptual expression (X12), brushwork expressiveness (X14), and watercolor language (X16), positively impact the overall aesthetic appeal. In mediation effects, X16 positively mediates between the influence of aesthetic perspectives and the overall aesthetic appeal. Factors such as warm–cool relationship (X8), X12, emotional atmosphere (X11), X14, and X16 positively or negatively affect the relationship between non-aesthetic perspectives and the overall aesthetic appeal. Multi-group analysis reveals significant differences in the evaluation factors and mediation effects that influence the overall aesthetic appeal. This study demonstrates the relationship between evaluation factors from different perspectives and aesthetics, providing valuable insights into evaluating Eastern and Western art. This evaluation system is applicable to academic research and practice in cultural heritage preservation and evaluation and art education, facilitating a deeper understanding of artistic values and promoting cross-cultural exchanges.
Modern content-based image retrieval systems demonstrate rather good performance in identifying visually similar artworks. However, this task becomes more challenging when art history specialists aim to refine the list of similar artworks based on their criteria, thus we need to train the model to reproduce this refinement. In this paper , we propose an approach for improving the list of similar paintings according to specific simulated criteria. By this approach, we retrieve paintings similar to a request image using ResNet50 model and ANNOY algorithm. Then, we simulate re-ranking based on the two criteria, and use the re-ranked lists for training LambdaMART model. Finally , we demonstrate that the trained model reproduces the re-ranking for the query painting by the specific criteria. We plan to use the proposed approach for reproducing re-rankings made by art history specialists, when this data will be collected.
