A Novel Artificial Mosaic Generation Technique Driven by Local Gradient Analysis.

Full text

Page 1

A Novel Artificial Mosaic Generation Technique
Driven by Local Gradient Analysis
Sebastiano Battiato, Gianpiero Di Blasi, Giovanni Gallo,
Giuseppe Claudio Guarnera, and Giovanni Puglisi
Dipartimento di Matematica e Informatica
University of Catania, Italy
{battiato,gdiblasi,gallo,guarnera,puglisi}@dmi.unict.it
http://www.dmi.unict.it/~iplab
Abstract. Art often provides valuable hints for technological innova-
tions especially in the field of Image Processing and Computer Graphics.
In this paper we present a novel method to generate an artificial mosaic
starting from a raster input image. This approach, based on Gradient
Vector Flow computation and some smart heuristics, permit us to follow
the most important edges maintaining at the same time high frequency
details. Several examples and comparisons with other recent mosaic gen-
eration approaches show the effectiveness of our technique.
Key words: Artificial Mosaic, Non Photo-Realistic Rendering, Gradi-
ent Vector Flow.
1Introduction
Mosaics are artwork constituted cementing together small colored tiles. Probably
they can be considered the first example of image synthesis technique based
on discrete primitives. The creation of a digital mosaic of artistic quality is a
challenging task. Many factors like position, orientation, size and shape of tiles
must be taken into account in the mosaic generation in order to densely pack
the tiles and emphasize the orientation chosen by the artist. Digital mosaic
generation from a raster image can be formulated as an optimization problem
in the following way:
Given an image I in the plane R2and a vector field Φ(x,y) defined on that
region by the influence of the edges of I, find N sites Pi(xi,yi) in I and place N
rectangles, one at each Pi, oriented with sides parallel to Φ(xi,yi), such that all
rectangles are disjoint, the area they cover is maximized and each tile is colored
by a color which reproduces the image portion covered by the tile [1].
Many mosaic generation algorithms have been recently developed. Hausner
[1] uses Centroidal Voronoi Diagram together with user selected features and
Manhattan distance. This approach obtains good results but, due to the high
number of iterations necessary to reach convergence, it is computationally slow.
In [2,3] the authors present an approach based on directional guidelines and
distance transform. They obtain very realistic results with a linear complexity

Page 2

2 Battiato et al.
with respect to image size. A novel technique for ancient mosaics generation has
been presented in [4]. The authors, using graph-cut optimization algorithm, are
able to work on tiles positioning without an explicit edge detection phase. Other
related works can be found in [5–7]. A complete survey of the existing methods
in the field of artificial mosaic is available in [8].
In this paper we propose a novel approach based on Gradient Vector Flow
(GVF) [9] computation together with some smart heuristics used to drive tiles
positioning. Almost all previous approaches filter out high frequencies in order
to simplify mosaic generation. Preliminary works [10] have shown that GVF
properties permit us to preserve edge information and maintain image details.
The novelty of this paper is related to the heuristics used to follow principal
edges and to maximize the overall mosaic area covering. In particular the tiles
positioning is not based only on gradient magnitude [10] but makes use of local
considerations to link together vectors that share the same ”logical” edge. Ex-
perimental results confirm the better quality of the new technique with respect
to the state of the art proposals [8].
The paper is structured as follows. Next section describes in detail the pro-
posed methodology. Experiments are reported in Section 3 whereas Section 4
closes the paper tracking direction for future works.
2Proposed Algorithm
In order to emulate the style of an ancient artisan we have developed an auto-
matic technique based on the following two steps:
– GVF (Gradient Vector Flow) field computation based on [9] algorithm;
– rule based tile positioning.
GVF is a dense force field designed by the authors of [9] in order to solve the
classical problems affecting snakes (sensitivity to initialization and poor conver-
gence to boundary concavity). Starting from the gradient of an image, this field
is computed through diffusion equations.
GVF is a field of vectors v = [v,u] that minimizes the following energy
function:
? ?
where the subscripts represent partial derivates along x and y axes respec-
tively, μ is a regularization parameter and |∇f| is the gradient computed from
the intensity input image. Due to the formulation described above, GVF field
values are close to |∇f| values where this quantity is large (energy E, to be
minimized, is dominated by |∇f|2|v − ∇f|2) and are slow-varying in homoge-
neous regions (the energy E is dominated by sum of the squares of the partial
derivatives of GVF field). An example of GVF field is shown in Figure 1.
This vector field can be used to effectively drive tiles positioning. Edge infor-
mation is preserved, it is propagated in the close regions and merged together
in a smoothly way.
E =
μ(u2
x+ u2
y+ v2
x+ v2
y) + |∇f|2|v − ∇f|2.
(1)

Page 3

Artificial Mosaic Generation Driven by Gradient Analysis3
Fig.1. Input image and its corresponding GVF field.
Let I be the input image to be mosaicized. In order to simplify the algorithm
we work only on the luminance channel of the image I, eliminating hue and
saturation information. The luminance L(I) is then equalized and the discrete
gradient of the results is calculated, by means of crossing difference. The equal-
ization process, especially for natural images, allows to normalize the overall
gradient distribution. The horizontal and vertical components of the gradient
∇L(I) are used as input for the GVF algorithm. Notice that in the implemen-
tation gradient computation is performed using Robert’s Kernel. This choice is
more noise sensitive and hence incorporates in the final mosaic a little, aesthet-
ically pleasant, randomicity.
All the tiles have the same rectangular shape and the same size. Moreover
we impose that tiles do not overlap. Placing ordering is hence fundamental in
terms of visual overall effect. First we consider local |GV F(I)| maxima with
values greater than a threshold th. These pixels, sorted according to |GV F(I)|,
are selected together with their neighbors with |GV F(I)| greater than tl(chains
of tiles are built up and placed). We impose that the tile orientation is obtained
according to the GVF direction in its central point. In this way we locate first
the neighborhood of main edges in the input image, just to follow the perceptual
orientation of the image itself. The second step of the algorithm is devoted to
cover the homogeneous regions of the image. This is accomplished simply placing
each tile one by one following the order left to right, up to bottom, starting form
the upper-left corner of the image. This heuristic strategy, somewhat arbitrary,
is justified by the properties of the GVF: this technique leads to aesthetically
pleasant results, by preserving main orientations and covering a wide portion of
pixels with tiles densely packed. The algorithm can be summarized as follows:
1. Input: a raster image I
2. L(I) ← Luminance(I)
3. G(I) ← Robert’s Gradient(Equalize(L(I)))
4. [u,v] ← GV F(|G(I)|∞,μ,nIterations)

Page 4

4Battiato et al.
5. gvf(I) ← (u2+ v2)1/2
6. In(I) ← NonMaxSuppression(gvf(I))
7. let th, tl be threshold values, with th> tl
8. Sort in queue Q pixels (i,j) according to decreasing In(i,j) values.
Only pixels whose In is greater than the threshold th are put into Q.
9. while Q is not empty
10. Extract a pixel (i,j) from Q
11.if (i,j) is not marked
12. Place a tile centering it in (i,j) at angle α = tan−1(v(i,j)/u(i,j))
13.if in this way the tile overlaps with previously placed tiles
14. Skip tile positioning
15.Starting from (i,j) follow and mark as visited the chain of local
maxima (i.e. In(w,z) > tl) in both directions perpendicular to α,
to obtain a guideline
16.Place a tile centering it in each (w,z) in the chain at angle
β = tan−1(v(w,z)/u(w,z))
17.if in this way the tile overlaps with previously placed tiles
18.Skip tile positioning.
19. for j ← 1 to length(I)
20.for i ← 1 to width(I)
21.Place a tile in the pixel(i,j) at angle γ = tan−1(v(i,j)/u(i,j))
22.if in this way the tile overlaps with previously placed tiles
23.Skip tile positioning.
3Experimental Results
The above algorithm has been implemented in JAVA, using for GVF computa-
tion an external MATLAB module. The method as been compared, to assess the
aesthetic quality, with the mosaics obtained by [2] and [4] by using their original
implementations. We wish to thank all the authors of these papers for providing
the output of their techniques upon our request. For sake of comparison Figure
2 reports the mosaic obtained from the standard Lena picture. A first clear ad-
vantage of the novel technique is that it is able to better preserve fine details.
This happens because high frequency areas are prioritary on tiles placing. Ob-
serve, to support this claim, the areas around Lena’s nose, lips and high brow.
Another performance index of mosaic algorithms is the amount and spatial dis-
tribution of untiled space. The area left uncovered by the proposed technique
is comparable with the amount of uncovered area left by [2], but gaps are here
better distributed. For example the constraints of [2] force the appearance of a
long ”crack” in the vertical band on the wall behind Lena, while the proposed
approach achieves in the same region a pleasant smoothness. Relatively to [4] it
should be observed that the uncovered area left by our technique is considerably
less. Observe that a higher percentage of covered area leads to a better preser-
vation of the original colors of the same picture (see Table 1). The perceived
texture obtained with the proposed technique appears, finally, less chaotic than
the texture obtained with [4]. As for the parameter adopted to produce Figure
2, the tile size is 5×5 and the image size is 667×667. Typically, considering other
images, we obtain covered area greater than [4] but smaller than [2].

Page 5

Artificial Mosaic Generation Driven by Gradient Analysis5
Fig.2. Visual comparison between mosaics generated by our approach (B), [2] (C) and
[4] (D), applied on input image (A) considering in all cases the same tile size 5×5.

Page 6

6Battiato et al.
Table 1. Number of tiles and covered area comparison between various approaches.
Technique
Our Method
[2]
[4]
Number of tiles Covered area
13412
13994
11115
75.4%
78.6%
62.5%
Fig.3. Comparison between the proposed approach and [10] on a Lena image detail
(a). The novel heuristics (c) are able to follow the underlying edges (b) maintaining
higher fidelity than (d) also considering the original colors.
Figure 3 shows a comparison between the proposed approach and [10] on a
Lena image detail (a). Both approaches do not consider the final step of linear
tiles placing. The novel heuristics (c) are able to follow the underlying edges (b)
maintaining higher fidelity than (d) also considering the original colors.
In Figure 4 we show the same image processed using increasing tile size. The
right relation between image size (and its level of detail) of the input, and the
tile size to be used in the mosaicing process, can be derived only by aesthetic
considerations. Our algorithm is able to preserve the global appearance even with
higher tile size. Finally we show in Figure 5 an example of mosaicized image by
using rectangular tiles (3x9). This example shows how the proposed criteria are
able to preserve fine details (due to GVF capabilities) maintaining at the same
time the global orientation of almost all edge present in the original picture (due
to the tile positioning rules).
The overall complexity of the proposed technique is O(kn)+O(nlogn), where
n is the number of pixels in the source image. Further results are depicted in
Figures 6, 7. The mosaicized images can be also downloaded at the following
web address: http://svg.dmi.unict.it/iplab/download/

Page 7

Artificial Mosaic Generation Driven by Gradient Analysis7
Fig.4. Mosaics generated with increasing tiles size A (3x3), B (6x6), C (10x10), D
(14x14).

Page 8

8 Battiato et al.
Fig.5. An example of mosaic generated with rectangular tiles (3x9).

Page 9

Artificial Mosaic Generation Driven by Gradient Analysis9
Fig.6. Input image (A) and its mosaic (B) generated by our approach (image size
595x744, tile size 5x5 ).
Fig.7. Input image (A) and its mosaic (B) generated by our approach (image size
532x646, tile size 4x4 ).

Page 10

10Battiato et al.
4 Conclusions
We propose a novel technique to produce a traditionally looking mosaic from a
digital source picture. The new technique tries to overcome the difficulties that
rely on edge detection, using the Gradient Vector Flow. Tests show that the new
technique produces aesthetically pleasant images that have a greater fidelity in
dealing with fine details and a better management of gaps. The proposed tech-
nique does not cut tiles. Indeed the next research step will integrate heuristics
like those proposed in [2] and [3] to cut tiles with the proposed method. Fu-
ture works will be also devoted to color management and mosaic generation in
vectorial format without using raster-to-vector conversion techniques [11].
References
1. Hausner, A.: Simulating decorative mosaics. In: Proc. SIGGRAPH 2001. (2001)
573–580
2. Di Blasi, G., Gallo, G.: Artificial mosaics. The Visual Computer 21(6) (2005)
373–383
3. Battiato, S., Di Blasi, G., Farinella, G.M., Gallo, G.: A novel technique for opus
vermiculatum mosaic rendering. In: Proc. ACM/WSCG 2006. (2006)
4. Liu, Y., Veksler, O., Juan, O.: Simulating classic mosaics with graph cuts. In:
Proc. EMMCVPR. (2007) 55–70
5. Faustino, G.M., de Figueiredo, L.H.: Simple adaptive mosaic effects. In: Proc.
SIBGRAPI 2005. (2005) 315–322
6. Elber, E., Wolberg, G.: Rendering traditional mosaics. The Visual Computer 19(1)
(2003) 67–78
7. Schlechtweg, S., Germer, T., Strothotte, T.: Renderbots — multi-agent systems
for direct image generation. Computer Graphics Forum 24(2) (2005) 137–148
8. Battiato, S., Di Blasi, G., Farinella, G.M., Gallo, G.: Digital mosaic frameworks -
an overview. Computer Graphics Forum 26(4) (2007) 794–812
9. Xu, C., Prince, L.: Snakes, shapes, and gradient vector flow. IEEE Transactions
on Image Processing 7(3) (1998) 359–369
10. Battiato, S., Di Blasi, G., Gallo, G., Guarnera, G.C., Puglisi, G.: Artificial mosaics
by gradient vector flow. In: Proc. EUROGRAPHICS 2008. (2008)
11. Battiato, S., Farinella, G.M., Puglisi, G.: Statistical based vectorization for stan-
dard vector graphics. In: Proc. ICCS 2006, Lectures Notes in Computer Science
Vol. 3992 (2006)