ArticlePDF Available

Abstract Art by Shape Classification

Authors:
  • Five AI Ltd

Abstract and Figures

This paper shows that classifying shapes is a tool useful in nonphotorealistic rendering (NPR) from photographs. Our classifier inputs regions from an image segmentation hierarchy and outputs the "best" fitting simple shape such as a circle, square, or triangle. Other approaches to NPR have recognized the benefits of segmentation, but none have classified the shape of segments. By doing so, we can create artwork of a more abstract nature, emulating the style of modern artists such as Matisse and other artists who favored shape simplification in their artwork. The classifier chooses the shape that "best" represents the region. Since the classifier is trained by a user, the "best shape" has a subjective quality that can over-ride measurements such as minimum error and more importantly captures user preferences. Once trained, the system is fully automatic, although simple user interaction is also possible to allow for differences in individual tastes. A gallery of results shows how this classifier contributes to NPR from images by producing abstract artwork.
Content may be subject to copyright.
A preview of the PDF is not available
... For the second problem, started with the image parsing, the style transfer process aims to construct the mapping relations between image contents and rendering primitives and then arrange the rendering primitives on the canvas to obtain the final image. Existing works address this mapping problem by seriatim matching each primitive with the image content according to shape [34], tone [2,29,36] and texture [29,43] similarity. This means only the existing primitives can be selected when rendering image. ...
... They computed the texture structure for each character of that typeface, then the character which can reflect the local image texture faithfully was found by a deformationtolerant texture matching method. Song et al. [34] classified the image segments into circle, square, or triangle on the basis of shape similarity, which can be used to create a more abstract natural artwork. All these methods require seriatim primitive matching and subtle threshold setting. ...
... To get the final rendering result, we first select stitch for each sub-region which best fits its contents and then place the stitch with the same size and orientation of the sub-region on the canvas. Existing techniques for primitive selection [3,29,31,34,43,48] only use the pregenerated primitives, which are selected by seriatim matching each primitive with the image content. However, since it is hard to pre-generate a complete stitch library to fit continuous image contents, these techniques may not work well for stitch selection. ...
Article
Full-text available
Random-needle Embroidery is a graceful Chinese art designated as Intangible Cultural Heritage, which “draws” beautiful images with thousands of free-form threads. In this paper, we explore techniques for automatically translating an input image into an art image with the random-needle style. The key idea is to generate rendering primitives of this art first, from which the corresponding dictionary is learned to further sparsely code the contents in the input image. To this end, we first define the artistic style of Random-needle Embroidery by introducing the notion of “stitch”, i.e., collection of threads arranged in a certain pattern, as the basic rendering primitive. Then, we adopt sparse coding to generate a stitch dictionary which gives a compact representation of the generated stitches. During runtime, new and more image content-adaptive stitches can be synthesized by optimizing a linear combination of stitch dictionary atoms via sparse representation. Then, the synthesized stitches are placed on the canvas sequentially and connected to adjacent stitches by stitch quilting. After placing all the stitches, a blank filling strategy is proposed and adopted to fill the uncovered areas on the canvas. The experimental results show our method can generate engaging images with the random-needle style. Moreover, our rendering image is better than those obtained by using two other state-of-the art methods.
... Hall and Song's implementation is semi-automatic: a human being must provide input for the output to be meaningful. In similar work, Song et al. [35] modified geometric shapes to create abstract art. ...
Article
Full-text available
The human figure is important in art. I discuss examples of the abstract depiction of the human figure, from both impressionist painting and children’s book illustration, and the challenge faced in algorithmically mimicking what human artists can achieve. I demonstrate that there are excellent examples in both genres that provide insight into what a human artist sees as important in providing abstraction at different levels of detail. The challenge lies in the human brain having enormous knowledge about the world and an ability to make fine distinctions about other humans from posture, clothing and expression. This allows a human to make assumptions about human figures from a tiny amount of data, and allows a human artist to take advantage of this when creating art. The question for the computer graphics community is whether and how we could algorithmically mimic what a human artist can do. I provide evidence from both genres to suggest possible ways forward.
... In interactive Machine Learning (iML), a human operator and a machine collaborate to achieve a task, whether this is to classify or cluster a set of data points [1,10], to find interesting data projections [5,6,13], or to design creative art works [35,43]. The underlying assumption is that the human-machine co-operation yields better results than a fully automated or manual system. ...
Book
Full-text available
The evaluation of interactive machine learning systems remains a difficult task. These systems learn from and adapt to the human, but at the same time, the human receives feedback and adapts to the system. Getting a clear understanding of these subtle mechanisms of co-operation and co-adaptation is challenging. In this chapter, we report on our experience in designing and evaluating various interactive machine learning applications from different domains. We argue for coupling two types of validation: algorithm-centred analysis, to study the computational behaviour of the system; and human-centred evaluation, to observe the utility and effectiveness of the application for end-users. We use a visual analytics application for guided search, built using an interactive evolutionary approach, as an exemplar of our work. Our observation is that human-centred design and evaluation complement algorithmic analysis, and can play an important role in addressing the "black-box" effect of machine learning. Finally, we discuss research opportunities that require human-computer interaction methodologies, in order to support both the visible and hidden roles that humans play in interactive machine learning.
... With the rapid development of computer software and hardware technology, the applications of 3D technology are becoming increasingly wide [4][5][6][7] and its shadow can be found in such applications as tourism [1,8,9], movies [10], games [11,12], arts [13], and education [14] etc. Of these applications, head-mounted virtual reality applications [15,16] have become the most popular. ...
... With the rapid development of computer software and hardware technology, the applications of 3D technology are becoming increasingly wide [1][2][3][4][5][6] and its shadow can be found in such applications as movies [7], games [8,9], arts [10], education [11], and Arduino [12] etc. Of these applications, head-mounted virtual reality applications [13,14] have become the most popular. ...
Article
Full-text available
With the rapid development of computer software technology, more and more experts and scholars have begun to engage in research relating to head-mounted virtual reality. Although current head-mounted virtual reality devices can present 3D scenes in a stereoscopic manner, the interactive manipulation is still inconvenient. For example, the HTC Vive and Oculus Rift need to be manipulated through their specific interactive devices. However, most interactive devices of this type are in a joystick form and usually lack intuitive manipulation. Therefore, this research introduced a hand tracking system into the head-mounted virtual reality system that allows users to perform system manipulation directly through hand actions. In practice, this research utilized the HTC Vive VR system for practical operation and a Manus glove for hand tracking detection. The Unity3D development environment was applied for development and system integration. To show the effect of integration, this research developed a set of 3D models for the system display. This system could not only allow the user to view realistic 3D virtual scenes through the HTC Vive but also allow the user to open function options through the Manus glove to realize such manipulation functions as selecting, zooming, rotating, switching models, and displaying 3D model information, etc. The development of this system verified that the intuitive 3D interaction system provided a method that was closer to natural manipulation.
... In interactive Machine Learning (iML), a human operator and a machine collaborate to achieve a task, whether this is to classify or cluster a set of data points [1,10], to find interesting data projections [5,6,13], or to design creative art works [35,43]. The underlying assumption is that the human-machine co-operation yields better results than a fully automated or manual system. ...
Chapter
Full-text available
The evaluation of interactive machine learning systems remains a difficult task. These systems learn from and adapt to the human, but at the same time, the human receives feedback and adapts to the system. Getting a clear understanding of these subtle mechanisms of co-operation and co-adaptation is challenging. In this chapter, we report on our experience in designing and evaluating various interactive machine learning applications from different domains. We argue for coupling two types of validation: algorithm-centred analysis, to study the computational behaviour of the system; and human-centred evaluation, to observe the utility and effectiveness of the application for end-users. We use a visual analytics application for guided search, built using an interactive evolutionary approach, as an exemplar of our work. Our observation is that human-centred design and evaluation complement algorithmic analysis, and can play an important role in addressing the “black-box” effect of machine learning. Finally, we discuss research opportunities that require human-computer interaction methodologies, in order to support both the visible and hidden roles that humans play in interactive machine learning.
... In interactive Machine Learning (iML), a human operator and a machine collaborate to achieve a task, whether this is to classify or cluster a set of data points [1,10], to find interesting data projections [5,6,13], or to design creative art works [35,43]. The underlying assumption is that the human-machine co-operation yields better results than a fully automated or manual system. ...
Article
Full-text available
The evaluation of interactive machine learning systems remains a difficult task. These systems learn from and adapt to the human, but at the same time, the human receives feedback and adapts to the system. Getting a clear understanding of these subtle mechanisms of co-operation and co-adaptation is challenging. In this chapter, we report on our experience in designing and evaluating various interactive machine learning applications from different domains. We argue for coupling two types of validation: algorithm-centered analysis, to study the computational behaviour of the system; and human-centered evaluation, to observe the utility and effectiveness of the application for end-users. We use a visual analytics application for guided search, built using an interactive evolutionary approach, as an exemplar of our work. Our observation is that human-centered design and evaluation complement algorithmic analysis, and can play an important role in addressing the "black-box" effect of machine learning. Finally, we discuss research opportunities that require human-computer interaction methodologies, in order to support both the visible and hidden roles that humans play in interactive machine learning.
Article
Geometric art, an artwork that is structured by geometric shapes, was first made popular by the introduction of Cubism paintings by Pablo Picasso and Georges Braque in the early-20th-century. With the recent advancement in digital imaging technology coupled with the rising popularity of social media such as Instagram, automatic geometric abstraction that can automatically transform a photograph into a piece of geometric art starts to gain research attention. Several state-of-the-art works have been proposed. Notably, despite the fact that Cubism artworks by renown artists illustrate the importance of the main subject in a painting to be recognizable and should be given more detailed representation, most of the state-of-the-art abstraction algorithms are not subject-focused. In this paper, we present Pic2PolyArt, a unified subject-focused geometric abstraction framework that can support both triangle-based and polygon-based abstraction. Given an input photograph, our proposed algorithm first identifies the main subject and important features of an image with a combination of saliency, edge, and face detection techniques. It then generates a set of seed points that are used by Delaunay Triangulation and Voronoi Tessellation to generate triangle-based and polygon-based geometric abstraction respectively. Results from qualitative evaluation on benchmark dataset and empirical user studies demonstrate the effectiveness of our proposed abstraction framework in generating pleasant geometric abstraction from photographs and provide insightful knowledge on users preference with regards to the type of polygons and the level of abstraction used to represent the resulting geometric art.
Article
In this paper we propose a new area-based convexity measure. We assume that convexity evaluation of an arbitrary planar shape is related to the total influence of dents of the shape, and discover that those attributes of the dents, such as the position, area, and depth with respect to the Geometric Center of Convex Hull (GCCH) of the shape, determine the dent influence. We consider that the convex hull of the shape consists of infinitely small patches, to each of which we assign a weight showing the patch influence. We can simply integrate all the patch weights in any regions within the convex hull to calculate their total influence. We define this operation as the Distance Weighted Area Integration, if the weight is associated with the Euclidean distance from the patch to the GCCH. Our new measure is a distance weighted generalization of the most commonly used convexity measure, making this conventional measure fully replaceable for the first time. Experiments demonstrate advantages of the new convexity measure against the existing ones.
Conference Paper
Collage is a composite artwork made from assemblage of different material forms. In this work, we present a novel approach for creating a fantastic collage artform, namely Arcimboldo-like collage, which represents an input image with multiple thematically-related cutouts from the filtered Internet images. Due to the massive data of Internet images, competent image cutouts can almost always be discovered to match the segmented components of the input image. The selected cutouts are purposefully arranged such that as a whole assembly, they can represent the input image with disguise in both shape and color; but separately, individual cutout is still recognizable as its own being. Experimental results and user study show that our algorithm can effectively produce the entertaining Arcimboldo-like collages.
Chapter
Region-based stylization provides algorithms for traditional media types, including woodcuts, stencils, and stained glass, as well as image abstraction, that may not correspond to any historical style or medium. In region-based stylization, the primitives are two-dimensional areas of the image plane, possibly populated with conventional primitives such as strokes in a secondary stage. The archetypal region-based stylization method involves an initial segmentation of the image, with subsequent assignment of colors to the region, filling the regions with strokes, if applicable, and possibly stylization of the region shapes. Region-based stylization techniques are characterized by their preservation of and attention to the region boundaries, which may be the exaggerated, smoothed, or otherwise modified results of an initial automatic or semi-automatic segmentation pass.
Conference Paper
The Laplacian pyramid is ubiquitous for decomposing images into multiple scales and is widely used for image analysis. However, because it is constructed with spatially invariant Gaussian kernels, the Laplacian pyramid is widely believed as being unable to represent edges well and as being ill-suited for edge-aware operations such as edge-preserving smoothing and tone mapping. To tackle these tasks, a wealth of alternative techniques and representations have been proposed, e.g., anisotropic diffusion, neighborhood filtering, and specialized wavelet bases. While these methods have demonstrated successful results, they come at the price of additional complexity, often accompanied by higher computational cost or the need to post-process the generated results. In this paper, we show state-of-the-art edge-aware processing using standard Laplacian pyramids. We characterize edges with a simple threshold on pixel values that allows us to differentiate large-scale edges from small-scale details. Building upon this result, we propose a set of image filters to achieve edge-preserving smoothing, detail enhancement, tone mapping, and inverse tone mapping. The advantage of our approach is its simplicity and flexibility, relying only on simple point-wise nonlinearities and small Gaussian convolutions; no optimization or post-processing is required. As we demonstrate, our method produces consistently high-quality results, without degrading edges or introducing halos.
Article
This paper describes a technique that transforms ordinary video segments into animations that have a hand-painted look. Our method is the first to exploit temporal coherence in video clips to design an automatic filter with a hand-drawn animation quality, in this case, one that produces an Impressionist effect. Off-the-shelf image processing and rendering techniques are employed, modified and combined in a novel way. This paper proceeds through the process step by step, providing helpful hints for tuning the off-the-shelf parts as well as describing the new techniques and bookkeeping used to glue the parts together.
Conference Paper
In this paper, we introduce a completely new approach to fitting rectangles and squares to given closed regions using our published ideas in Rothe et al. (1996), Voss and Suesse (1997, 1999). In these papers, we have developed a new region-based fitting method using the method of normalization. There we demonstrate the zero-parametric fitting of lines, triangles, parallelograms, circles and ellipses, and the one-parametric fitting of elliptical segments, circular segments and super-ellipses. In the present paper, we discuss this normalization idea for fitting of closed regions using rectangles and squares. As features we use the area-based low order moments. The main problem is a stable normalization of the rotation. We show that we have to solve only an one-dimensional optimization problem in the case of rectangles. In the case of squares there are no free parameters to determine. The presented algorithm is used in practice for document recognition