Fig 1 - uploaded by Hendrik De Villiers
Content may be subject to copyright.
This image depicts the basic structure of a CNN [5]. 

This image depicts the basic structure of a CNN [5]. 

Source publication
Conference Paper
Full-text available
Procedural texture generation enables the creation of more rich and detailed virtual environments without the help of an artist. However, finding a flexible generative model of real world textures remains an open problem. We present a novel Convolutional Neural Network based texture model consisting of two summary statistics (the Gramian and Transl...

Context in source publication

Context 1
... Convolutional Neural Network is a type of feed-forward neural network that predominantly consists of convolutional, pooling, fully connected and softmax layers. The basic struc- ture of a neural network is depicted in Figure 1. Because only convolutional and pooling layers are employed during texture generation, we restrict our attention to these layers in the following outline of layer functionality. ...

Citations

... Apart from these work dealing with cross-correlation of features and closely related to the present paper, [25] proposed to incorporate the power spectrum in the loss function, thereby enabling the respect of highly structured textures. In a related work, [32], it is proposed to impose the spectrum constraint by using a windowed Fourier transform, enabling non-stationary behavior to be accounted for, at the cost of the inherent stationary nature of textures. ...
... Nevertheless, the size of the filters used in CNNs such as VGG-19, and therefore the size of the corresponding receptive fields, is small with respect to the size of the image especially when synthesizing high-resolution images (here 1024 × 1024). As we have mentioned in the introduction, several works have addressed this limitation [2,25,28,32,33], but, as we will see in the experimental section, none is fully satisfactory. In the following sections, we propose several improvement of the original neural texture synthesis method in order to address this limitation. ...
Article
Full-text available
The field of texture synthesis has witnessed important progresses over the last years, most notably through the use of convolutional neural networks. However, neural synthesis methods still struggle to reproduce large-scale structures, especially with high-resolution textures. To address this issue, we first introduce a simple multi-resolution framework that efficiently accounts for long-range dependency. Then, we show that additional statistical constraints further improve the reproduction of textures with strong regularity. This can be achieved by constraining both the Gram matrices of a neural network and the power spectrum of the image. Alternatively, one may constrain only the autocorrelation of the features of the network and drop the Gram matrices constraints. In an experimental part, the proposed methods are then extensively tested and compared to alternative approaches, both in an unsupervised way and through a user study. Experiments show the advantage of the multi-scale scheme for high-resolution textures and the advantage of combining it with additional constraints for regular textures.
... This work will be developed later in Section 3.2.3. In a related work, Schreiber et al. [2016] propose to impose the spectrum constraint by using a windowed Fourier Transform instead of a Fourier Transform. This enables non-stationary behaviors to be accounted for, at the cost of the inherent stationary nature of textures. ...
Thesis
In this thesis, we study the transfer of Convolutional Neural Networks (CNN) trained on natural images to related tasks. We follow two axes: texture synthesis and visual recognition in artworks. The first one consists in synthesizing a new image given a reference sample. Most methods are based on enforcing the Gram matrices of ImageNet-trained CNN features. We develop a multi-resolution strategy to take into account large scale structures. This strategy can be coupled with long-range constraints either through a Fourier frequency constraint, or the use of feature maps autocorrelation. This scheme allows excellent high-resolution synthesis especially for regular textures. We compare our methods to alternatives ones with quantitative and perceptual evaluations. In a second axis, we focus on transfer learning of CNN for artistic image classification. CNNs can be used as off-the-shelf feature extractors or fine-tuned. We illustrate the advantage of the last solution. Second, we use feature visualization techniques, CNNs similarity indexes and quantitative metrics to highlight some characteristics of the fine-tuning process. Another possibility is to transfer a CNN trained for object detection. We propose a simple multiple instance method using off-the-shelf deep features and box proposals, for weakly supervised object detection. At training time, only image-level annotations are needed. We experimentally show the interest of our models on six non-photorealistic.
... Apart from these work dealing with cross-correlation of features and closely related to the present paper, [24] proposed to incorporate the power spectrum in the loss function, thereby enabling the respect of highly structured textures. In a related work, [31], it is proposed to impose the spectrum constraint by using a windowed Fourier Transform, enabling non-stationnary behavior to be accounted for, at the cost of the inherent stationary nature of textures. ...
... Nevertheless, the size of the filters used in CNNs such as VGG-19, and therefore the size of the corresponding receptive fields, are small with respect to the size of the image especially when synthesizing high resolution images (here 1024×1024). As we have mentioned in the introduction, several works have addressed this limitation [24,2,27,31,32], but, as we will see in the experimental section, none is fully satisfactory. In the following sections, we propose several improvement of the original neural texture synthesis method in order to address this limitation. ...
Preprint
Full-text available
The field of texture synthesis has witnessed important progresses over the last years, most notably through the use of Convolutional Neural Networks. However, neural synthesis methods still struggle to reproduce large scale structures, especially with high resolution textures. To address this issue, we first introduce a simple multi-resolution framework that efficiently accounts for long-range dependency. Then, we show that additional statistical constraints further improve the reproduction of textures with strong regularity. This can be achieved by constraining both the Gram matrices of a neural network and the power spectrum of the image. Alternatively one may constrain only the autocorrelation of the features of the network and drop the Gram matrices constraints. In an experimental part, the proposed methods are then extensively tested and compared to alternative approaches, both in an unsupervised way and through a user study. Experiments show the interest of the multi-scale scheme for high resolution textures and the interest of combining it with additional constraints for regular textures.
Conference Paper
The huge amount of data resulting from the ac- quisition of medical images with multiple modalities can be overwhelming for storage and sharing through communication systems. Thus, efficient compression algorithms must be in- troduced to reduce the burden of storage and communication resources required by such amount of data. However, since in the medical context all details are important, the adoption of lossless image compression algorithms is paramount. This paper proposes a novel lossless compression scheme tailored to jointly compress the modality of computerized to- mography (CT) and that of positron emission tomography (PET). Different approaches are adopted, namely image-to-image trans- lation techniques and redundancies between both images are also explored. To perform the image-to-image translation approach, we resort to lossless compression of the original CT data and apply a cross-modality image translation generative adversarial network to obtain an estimation of the corresponding PET. Then, the residue that results from the differences between the original PET and its estimation is also compressed. Thus, instead of compressing two independent image modalities, i.e., both images of the original PET-CT pair, in the proposed approach only the CT is independently encoded along with the PET residue. The performed experiments using a publicly available PET- CT pair dataset show that the proposed scheme attains up to 8.9% compression gains for the PET data, in comparison with the naive approach, and up to 3.5% gains for the PET-CT pair.
Article
Online monitoring of pellet size distribution (PSD) of green pellets is an important work in product quality control of pelletization process. Conventionally, image segmentation technique is a preliminary step in computer vision-based PSD monitoring. However, haze, pellets overlapping, and uneven illumination contribute to the main challenges that severely impair the segmentation performance and PSD measurement accuracy. This article proposed a fully automatic online PSD monitoring method incorporating a K-means clustering-based haze judgment module, a lightweight U-net segmentation model with the fusion of none-weight VGG16 features (VGG16-LUnet), and a convex-hull detection and ellipse fitting model for adhesive pellet separation and contour fitting. The VGG16-LUnet model can accurately segment the pellets from both hazy and haze-free images with the help of haze judgment module. Thus, this model can be called VGG16-LUnet-TAdj. Then, a contour fitting model is applied to determine the pellets sizes based on the segmentation results, and the PSD is obtained as well. Extensive experiments on the segmentation of in situ captured green pellet images and the corresponding PSD curves demonstrate that our proposed method performs comparable or even favorable to the state-of-the-art methods.