Chapter

Seam Carving Detection and Localization Using Two-Stage Deep Neural Networks

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Seam carving is a method to resize an image in a content-aware fashion. However, this method can also be used to carve out objects from images. In this paper, we propose a two-step method to detect and localize seam carved images. First, we build a detector to detect small patches in an image that has been seam carved. Next, we compute a heatmap on an image based on the patch detector’s output. Using these heatmaps, we build another detector to detect if a whole image is seam carved or not. Our experimental results show that our approach is effective in detecting and localizing seam carved images.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... One may reapply seam carving to enlarge the image to its original size. Seam carving may highlight some texture artifacts in the resulting image as well [10,15,16], which can be considered for further automatic detection. Although these artifacts are mostly imperceptible to the human eye, they can be detected by computer vision techniques. ...
... One may reapply seam carving to enlarge the image to its original size. Seam carving may highlight some texture artifacts in the resulting image as well [16,10,15], which can be considered for further automatic detection. Although these artifacts are mostly imperceptible to the human eye, they can be detected by computer vision techniques. ...
Preprint
Full-text available
Seam carving is a computational method capable of resizing images for both reduction and expansion based on its content, instead of the image geometry. Although the technique is mostly employed to deal with redundant information, i.e., regions composed of pixels with similar intensity, it can also be used for tampering images by inserting or removing relevant objects. Therefore, detecting such a process is of extreme importance regarding the image security domain. However, recognizing seam-carved images does not represent a straightforward task even for human eyes, and robust computation tools capable of identifying such alterations are very desirable. In this paper, we propose an end-to-end approach to cope with the problem of automatic seam carving detection that can obtain state-of-the-art results. Experiments conducted over public and private datasets with several tampering configurations evidence the suitability of the proposed model.
... One may reapply seam carving to enlarge the image to its original size. Seam carving may highlight some texture artifacts in the resulting image as well [16,10,15], which can be considered for further automatic detection. Although these artifacts are mostly imperceptible to the human eye, they can be detected by computer vision techniques. ...
Conference Paper
Seam carving is a computational method capable of resizing images for both reduction and expansion based on its content, instead of the image geometry. Although the technique is mostly employed to deal with redundant information, i.e., regions composed of pixels with similar intensity, it can also be used for tampering images by inserting or removing relevant objects. Therefore, detecting such a process is of extreme importance regarding the image security domain. However, recognizing seam-carved images does not represent a straightforward task even for human eyes, and robust computation tools capable of identifying such alterations are very desirable. In this paper, we propose an end-to-end approach to cope with the problem of automatic seam carving detection that can obtain state-of-the-art results. Experiments conducted over public and private datasets with several tampering configurations evidence the suitability of the proposed model.
... There is extensive recent work on image forensics, including techniques to detect image splicing based forgeries ( [5], [6]), copy-move forgeries ( [7], [8], [1]), image retouching ( [9], [10]), seam carving ( [11], [12]) and image resampling ( [13], [14]). Other common manipulations include machine learning based forgeries, typically manipulated using Generative Adversarial Networks (GANs) ( [15], [16]). ...
Preprint
Full-text available
Recent advances in machine learning and computer vision have made it simple to manipulate a variety of images, including satellite images. Most of the commercially available satellite images go through the process of orthorectification to remove potential distortions due to terrain variations. This orthorectification process typically involves the use of rational polynomial coefficients (RPC) that geometrically remap the pixels in the original image to the rectified image. This paper proposes a new method to verify the authenticity of these orthorectified images with respect to the associated RPC metadata. The steps include calculating the Residual Discrete Fourier Transform (DFT) pattern from the image using a linear predictor based residual spectral analysis and comparing with expected residual DFT pattern using the RPC metadata associated with the image. If the metadata associated with an orthorectified image is the correct one, then both the DFT patterns should have high structural similarity. We use SSIM (Structural Similarity Index Metric) to quantify the similarity and thereby verify if the data has been tampered or not. Detailed experimental results are presented to demonstrate the high accuracy of the proposed method in detecting manipulations.<br
... There is extensive recent work on image forensics, including techniques to detect image splicing based forgeries ( [5], [6]), copy-move forgeries ( [7], [8], [1]), image retouching ( [9], [10]), seam carving ( [11], [12]) and image resampling ( [13], [14]). Other common manipulations include machine learning based forgeries, typically manipulated using Generative Adversarial Networks (GANs) ( [15], [16]). ...
Preprint
Recent advances in machine learning and computer vision have made it simple to manipulate a variety of images, including satellite images. Most of the commercially available satellite images go through the process of orthorectification to remove potential distortions due to terrain variations. This orthorectification process typically involves the use of rational polynomial coefficients (RPC) that geometrically remap the pixels in the original image to the rectified image. This paper proposes a new method to verify the authenticity of these orthorectified images with respect to the associated RPC metadata. The steps include calculating the Residual Discrete Fourier Transform (DFT) pattern from the image using a linear predictor based residual spectral analysis and comparing with expected residual DFT pattern using the RPC metadata associated with the image. If the metadata associated with an orthorectified image is the correct one, then both the DFT patterns should have high structural similarity. We use SSIM (Structural Similarity Index Metric) to quantify the similarity and thereby verify if the data has been tampered or not. Detailed experimental results are presented to demonstrate the high accuracy of the proposed method in detecting manipulations.<br
... The authors in [38] improved their network design with a new architecture, ILFNet [39]. The authors in [40] proposes a two stage model where stage-1 performs patch level localization and stage-2 performs image level classification. The method proposed in this paper localizes seams at a pixel level as opposed to the patch level strategies in prior work and focuses on satellite imagery, which is more resistant to seam carving artifacts than consumer images. ...
Preprint
Full-text available
Seam carving is a popular technique for content aware image retargeting. It can be used to deliberately manipulate images, for example, change the GPS locations of a building or insert/remove roads in a satellite image. This paper proposes a novel approach for detecting and localizing seams in such images. While there are methods to detect seam carving based manipulations, this is the first time that robust localization and detection of seam carving forgery is made possible. We also propose a seam localization score (SLS) metric to evaluate the effectiveness of localization. The proposed method is evaluated extensively on a large collection of images from different sources, demonstrating a high level of detection and localization performance across these datasets. The datasets curated during this work will be released to the public.
Conference Paper
Full-text available
Last-generation GAN models allow to generate synthetic images which are visually indistinguishable from natural ones, raising the need to develop tools to distinguish fake and natural images thus contributing to preserve the trustworthiness of digital images. While modern GAN models can generate very high-quality images with no visibile spatial artifacts, reconstruction of consistent relationships among colour channels is expectedly more difficult. In this paper, we propose a method for distinguishing GAN-generated from natural images by exploiting inconsistencies among spectral bands, with specific focus on the generation of synthetic face images. Specifically, we use cross-band co-occurrence matrices, in addition to spatial co-occurrence matrices, as input to a CNN model, which is trained to distinguish between real and synthetic faces. The results of our experiments confirm the goodness of our approach which outperforms a similar detection technique based on intra-band spatial co-occurrences only. The performance gain is particularly significant with regard to robustness against post-processing, like geometric transformations, filtering and contrast manipulations.
Article
Full-text available
With advanced image journaling tools, one can easily alter the semantic meaning of an image by exploiting certain manipulation techniques such as copy-clone, object splicing, and removal, which mislead the viewers. In contrast, the identification of these manipulations becomes a very challenging task as manipulated regions are not visually apparent. This paper proposes a high-confidence manipulation localization architecture which utilizes resampling features, Long-Short Term Memory (LSTM) cells, and encoder-decoder network to segment out manipulated regions from non-manipulated ones. Resampling features are used to capture artifacts like JPEG quality loss, upsampling, downsampling, rotation, and shearing. The proposed network exploits larger receptive fields (spatial maps) and frequency domain correlation to analyze the discriminative characteristics between manipulated and non-manipulated regions by incorporating encoder and LSTM network. Finally, decoder network learns the mapping from low-resolution feature maps to pixel-wise predictions for image tamper localization. With predicted mask provided by final layer (softmax) of the proposed architecture, end-to-end training is performed to learn the network parameters through back-propagation using ground-truth masks. Furthermore, a large image splicing dataset is introduced to guide the training process. The proposed method is capable of localizing image manipulations at pixel level with high precision, which is demonstrated through rigorous experimentation on three diverse datasets. Code and data are available in the following link-https://github.com/jawadbappy/forgery_localization_HLED
Article
Full-text available
Seam carving is widely used excellent content-aware image scaling method. When an image is processed by seam carving, its local texture changes. Local binary patterns is an excellent local descriptor for describing the local texture of an image. In this paper, a blind detection based uniform local binary patterns(ULBP) is proposed to detect seam-carved image. Firstly, the image is transformed into gray-scale image. Then the ULBP histogram features and seam features are extracted from the gray-scale image. Finally support vector machine (SVM) is adopted as classifier to train and test those features to identify whether an image is subjected to seam carving or not. Wei et al. (Pattern Recogn Lett 36:100–106 2014) method and Ryu et al. (IEICE Trans Inf Syst 97(5):1304–1311 2014) method are selected as the benchmark. Extensive compared experiments are conducted by the three methods, respectively. Experimental results show that the proposed method has the best performance among the three methods under a variety of setting.
Article
Full-text available
In this work, we propose a technique that utilizes a fully convolutional network (FCN) to localize image splicing attacks. We first evaluated a single-task FCN (SFCN) trained only on the surface label. Although the SFCN is shown to provide superior performance over existing methods, it still provides a coarse localization output in certain cases. Therefore, we propose the use of a multi-task FCN (MFCN) that utilizes two output branches for multi-task learning. One branch is used to learn the surface label, while the other branch is used to learn the edge or boundary of the spliced region. We trained the networks using the CASIA v2.0 dataset, and tested the trained models on the CASIA v1.0, Columbia Uncompressed, Carvalho, and the DARPA/NIST Nimble Challenge 2016 SCI datasets. Experiments show that the SFCN and MFCN outperform existing splicing localization algorithms, and that the MFCN can achieve finer localization than the SFCN.
Conference Paper
Full-text available
Resampling is an important signature of manipulated images. In this paper, we propose two methods to detect and localize image manipulations based on a combination of re-sampling features and deep learning. In the first method, the Radon transform of resampling features are computed on overlapping image patches. Deep learning classifiers and a Gaussian conditional random field model are then used to create a heatmap. Tampered regions are located using a Random Walker segmentation method. In the second method, resampling features computed on overlapping image patches are passed through a Long short-term memory (LSTM) based network for classification and localiza-tion. We compare the performance of detection/localization of both these methods. Our experimental results show that both techniques are effective in detecting and localizing digital image forgeries.
Article
Full-text available
Content-aware image resizing is currently widely used because it maintains the original appearance of important objects to the greatest extent when the aspect ratio of an image changes during resizing. Content-aware image resizing techniques, such as seam carving, are also used for image forgery. A new Benford’s law-based algorithm for detecting content-aware resized images is presented. The algorithm extracts features on the basis of the first digit distribution of the discrete cosine transform coefficients, which follow the standard Benford’s law. We trained these features from both normal images and content-aware resized images using a support vector machine. The experimental results show that the proposed method can efficiently distinguish a content-aware resized image from a normal image, and its precision is better than that of existing methods, including those based on Markov features and others.
Article
Full-text available
Content-aware image retargeting has been investigated since the last decade as a paradigm of image modification for proper display on the different screen sizes. Modifications, such as seam carving or seam insertion, have been introduced to achieve aforesaid image retargeting. The changes in an image are not easily recognizable by human eyes. Inspired by the Blocking Artifact Characteristics Matrix (BACM), a method to detect tampers caused by seam modification on JPEG retargeted images without knowledge of the original image is proposed in this paper. In a BACM block matrix, we found that the original JPEG image demonstrates a regular symmetrical data, whereas the symmetrical data in a block reconstructed by seam modification is destroyed. Twenty-two features are proposed to train the data by using a Support Vector Machine (SVM) classification method. The experimental results clearly demonstrate that the proposed method provides a very high recognition rate for those JPEG retargeted images. The source codes and the complete experimental data can be accessed at http://video.minelab.tw/DETS/.
Article
Full-text available
We propose a new algorithm for the accurate detection and localization of copy–move forgeries, based on rotation-invariant features computed densely on the image. Dense-field techniques proposed in the literature guarantee a superior performance with respect to their keypoint-based counterparts, at the price of a much higher processing time, mostly due to the feature matching phase. To overcome this limitation, we resort here to a fast approximate nearest-neighbor search algorithm, PatchMatch, especially suited for the computation of dense fields over images. We adapt the matching algorithm to deal efficiently with invariant features, so as to achieve higher robustness with respect to rotations and scale changes. Moreover, leveraging on the smoothness of the output field, we implement a simplified and reliable postprocessing procedure. The experimental analysis, conducted on databases available online, proves the proposed technique to be at least as accurate, generally more robust, and typically much faster than the state-of-the-art dense-field references.
Article
Full-text available
Seam carving is a content-aware multimedia retargeting technique to adaptively resize multimedia data for different display sizes. However, it can also be used to remove objects from digital object or video for malicious purposes. In this paper, a forensics hash-based tampering detection and localization approach is proposed for seam carving-based video retargeting. It extracts the invariant Speeded-up Robust Feature points from every spatiotemporal image to represent the matching surface, and the relative position change of the neighboring matching surface is used to build the forensic hash in a compact and scalable way. Experimental results show that the proposed forensics approach can effectively estimate the exact amount and rough locations of deleted seam carving surfaces. It achieves desirable detection performance even when there are frames deleted. If the hash length is reasonably increased, it can estimate the rough location and exact amount of deleted frames. Moreover, the built forensics hash is of good robustness, scalability, and compactness. Copyright © 2014 John Wiley & Sons, Ltd.
Article
Full-text available
Image inpainting refers to the process of restoring missing or damaged areas in an image. This field of research has been very active over recent years, boosted by numerous applications: restoring images from scratches or text overlays, loss concealment in a context of impaired image transmission, object removal in a context of editing, or disocclusion in image-based rendering (IBR) of viewpoints different from those captured by the cameras. Although earlier work dealing with disocclusion has been published in [1], the term inpainting first appeared in [2] by analogy with a process used in art restoration.
Article
Full-text available
We propose a new method to detect resampled imagery. The method is based on examining the normalized energy density present within windows of varying size in the second derivative of the image in the frequency domain, and exploiting this characteristic to derive a 19-D feature vector that is used to train a SVM classifier. Experimental results are reported on 7500 raw images from the BOSS database. Comparison with prior work reveals that the proposed algorithm performs similarly for resampling rates greater than 1, and is superior to prior work for resampling rates less than 1. Experiments are performed for both bilinear and bicubic interpolations, and qualitatively similar results are observed for each. Results are also provided for the detection of resampled imagery with noise corruption and JPEG compression. As expected, some degradation in performance is observed as the noise increases or the JPEG quality factor declines.
Article
Full-text available
Seam carving" is a recently introduced content aware image re-sizing algorithm. This method can also be used for image tamper-ing. In this paper, we explore techniques to detect seam carving (or seam insertion) without knowledge of the original image. We em-ploy a machine learning based framework to distinguish between seam-carved (or seam-inserted) and normal images. It is seen that the 324-dimensional Markov feature, consisting of 2D difference histograms in the block-based Discrete Cosine Transform domain, is well-suited for the classification task. The feature yields a detec-tion accuracy of 80% and 85% for seam carving and seam insertion, respectively. For seam insertion, each new pixel that is introduced is a linear combination of its neighboring pixels. We detect seam insertions based on this linear relation, with a high detection accu-racy of 94% even for very low seam insertion rates. We show that the Markov feature is also useful for scaling and rotation detection.
Article
Full-text available
JPEG is one of the most extensively used image formats. Understanding the inherent characteristics of JPEG may play a useful role in digital image forensics. In this paper, we introduce JPEG error analysis to the study of image forensics. The main errors of JPEG include quantization, rounding, and truncation errors. Through theoretically analyzing the effects of these errors on single and double JPEG compression, we have developed three novel schemes for image forensics including identifying whether a bitmap image has previously been JPEG compressed, estimating the quantization steps of a JPEG image, and detecting the quantization table of a JPEG image. Extensive experimental results show that our new methods significantly outperform existing techniques especially for the images of small sizes. We also show that the new method can reliably detect JPEG image blocks which are as small as 8 × 8 pixels and compressed with quality factors as high as 98. This performance is important for analyzing and locating small tampered regions within a composite image.
Conference Paper
Full-text available
Current image re-sampling detectors can reliably detect re-sampling in JPEG images only up to a Quality Factor (QF) of 95 or higher. At lower QFs, periodic JPEG blocking artifacts interfere with periodic patterns of re-sampling. We add a controlled amount of noise to the image before the re-sampling detection step. Adding noise suppresses the JPEG artifacts while the periodic patterns due to re-sampling are partially retained. JPEG images of QF range 75-90 are considered. Gaussian/Uniform noise in the range of 28-24 dB is added to the image and the images thus formed are passed to the re-sampling detector. The detector outputs are averaged to get a final output from which re-sampling can be detected even at lower QFs. We consider two re-sampling detectors - one proposed by Poposcu and Farid [1], which works well on uncompressed and mildly compressed JPEG images and the other by Gallagher [2], which is robust on JPEG images but can detect only scaled images. For multiple re-sampling operations (rotation, scaling, etc) we show that the order of re-sampling matters. If the final operation is up-scaling, it can still be detected even at very low QFs.
Conference Paper
Full-text available
In this paper, we propose a statistical test to discriminate between original and forged regions in JPEG images, under the hypothesis that the former are doubly compressed while the latter are singly compressed. New probability models for the DCT coefficients of singly and doubly compressed regions are proposed, together with a reliable method for estimating the primary quantization factor in the case of double compression. Based on such models, the probability for each DCT block to be forged is derived. Experimental results demonstrate a better discriminating behavior with respect to previously proposed methods.
Article
Full-text available
Traditional image resizing techniques are oblivious to the content of the image when changing its width or height. In contrast, media (i.e., image and video) retargeting take s content into account. For example, one would like to change the aspect ratio of a video without making human figures look too fat or too skinny, or change the size of an image by automatically removing "unnecessary" portions while keeping the "important" features intact. We propose a simple operator; we term seam carving to support image and video retargeting. A seam is an optimal 1D path of pixels in an image, or a 2D manifold in a video cube, going from top to bottom, or left to right. Optimality is defined by minimizing an energy function that assigns costs to pixels. We show that computing a seam reduces to a dynamic programming problem for images and a graph min-cut search for video. We demonstrate that several image and video operations, such as aspect ratio correction, size change, and object removal, can be recast as a successive operation of the seam carving operator.
Article
Full-text available
Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports content-aware image resizing for both reduction and expansion. A seam is an optimal 8-connected path of pixels on a single image from top to bottom, or left to right, where optimality is defined by an image energy function. By repeatedly carving out or inserting seams in one direction we can change the aspect ratio of an image. By applying these operators in both directions we can retarget the image to a new size. The selection and order of seams protect the content of the image, as defined by the energy function. Seam carving can also be used for image content enhancement and object removal. We support various visual saliency measures for defining the energy of an image, and can also include user input to guide the process. By storing the order of seams in an image we create multi-size images, that are able to continuously change in real time to fit a given size.
Article
Full-text available
In this paper, we analyze and analytically describe the specific statistical changes brought into the covariance structure of signal by the interpolation process. We show that interpolated signals and their derivatives contain specific detectable periodic properties. Based on this, we propose a blind, efficient, and automatic method capable of finding traces of resampling and interpolation. The proposed method can be very useful in many areas, especially in image security and authentication. For instance, when two or more images are spliced together, to create high quality and consistent image forgeries, almost always geometric transformations, such as scaling, rotation, or skewing are needed. These procedures are typically based on a resampling and interpolation step. By having a method capable of detecting the traces of resampling, we can significantly reduce the successful usage of such forgeries. Among other points, the presented method is also very useful in estimation of the geometric transformations factors.
Article
Seam carving is a representative content-aware image retargeting approach to adjust the size of an image. To preserve visually prominent content, seam-carving algorithms first calculate the connected path of pixels, referred to as the seam, according to a defined cost function and then adjust the size of an image by removing or duplicating repeatedly calculated seams. Seam carving is actively exploited to overcome diversity in the resolution of images between applications and devices; hence, detecting the distortion caused by seam carving has become important in image forensics. In this paper, we propose a convolutional neural network (CNN)-based approach to classifying seam-carving forgery. To attain the ability to learn low-level features, we designed a convolutional neural network (CNN) architecture comprising five types of network blocks specialized in capturing local artifacts caused by seam carving. An ensemble module is further adopted to both enhance performance and comprehensively analyze the features in the local areas. To validate the effectiveness of our work, extensive experiments based on various CNN-based baselines were conducted. Compared to the baselines, our work exhibits state-of-the-art performance in terms of three-class classification (original, seam inserted, and seam removed). The experimental results also demonstrate that our model with the ensemble module is robust for various unseen cases.
Article
Image retargeting refers to manipulating an input image to adapt its display on diverse screens with distinct spatial resolutions and aspect ratios. However, it might also be used for malicious image forgeries such as object removal. There exist several types of image retargeting techniques, yet most existing image retargeting detection approaches expose only single type of image retargeting. This paper proposes a blind forensics approach to identify three image retargeting techniques including seam carving, scaling and scale-and-stretch. Since image retargeting usually leads to local texture distortion and global structure deformation, a 512-dimensional feature set, which is made up of LBPCO (co-occurrence of local binary pattern) and BSIF (binarized statistical image features), is designed to expose image retargeting artifacts. Support vector machine (SVM) is exploited as pattern classifier to identify three image retarget-ing techniques. Compared with existing works, the proposed approach not only detects image retargeting forgery, but also identifies specific image retargeting technique.
Conference Paper
Deep Learning techniques have been widely used in the recent years, primarily because of their efficiency in several applications, such as engineering, medicine, and data security. Seam carving is a content-aware image resizing method that can also be used for image tampering, being not straightforward to be identified. In this paper, we combine Convolutional Neural Networks and Local Binary Patterns to recognize whether an image has been modified automatically or not by seam carving. The experimental results show that the proposed approach can achieve accuracies within the range [81% − 98%] depending on the severity of the tampering procedure.
Article
Seam carving has been successfully applied to resize and deliberately remove objects from digital images. The proliferation of seam carving makes a serious challenge in image forensics. In this paper, we propose a blind detection method for seam insertion images and try to recover the original image from seam inserted image. Because the corresponding pixel in every selected seam is removed and replaced by two pixels, whose values are computed by averaging its value with their left and right neighbors in a seam insertion image. The location of the inserted pixels can be estimated by analyzing correlation between adjacent pixels. Based on estimated insertion pixels, we make an attempt to derive the original value for recovering the original image. Our experimental results show that our proposed detection method achieves better detection rate compared with other existing seam insertion detection methods. Meanwhile, our method can locate most of the inserted pixels and recovery the original image.
Article
Seam-carving is used for adjusting the size of an image as well as removing objects from images. When seam-carving is applied to delete the object in the image, it may lead to the distortion of the information transmitted by the image, misleading the public, and endangering human health. Therefore, detecting the location of objects removed by seam-carving techniques is a scientific challenge to the image forensics. According to the position movement law of the related pixel in the seam-carving process, a method of the detecting and locating the objects deleted based on the block artifact grid (BAG) mispairing characteristic is proposed in this paper. Firstly, the BAG is extracted from the JPEG image; secondly, 10-dimensional features of the BAG chart is constructed; thirdly, clustering technique is applied on the extracted features of the BAG to obtain the classification results; finally, the location of the removed object is computed based on the previous classification results. Experiment result shows that the proposed method can efficiently detect and locate the object removed from an image by seam-carving. The proposed method in this paper opens up a new research direction for the location and detection of tampering, and has important application value for the intelligent forensics system based on seam carving.
Conference Paper
In this paper, we present a new image forgery detection method based on deep learning technique, which utilizes a convolutional neural network (CNN) to automatically learn hierarchical representations from the input RGB color images. The proposed CNN is specifically designed for image splicing and copy-move detection applications. Rather than a random strategy, the weights at the first layer of our network are initialized with the basic high-pass filter set used in calculation of residual maps in spatial rich model (SRM), which serves as a regularizer to efficiently suppress the effect of image contents and capture the subtle artifacts introduced by the tampering operations. The pre-trained CNN is used as patch descriptor to extract dense features from the test images, and a feature fusion technique is then explored to obtain the final discriminative features for SVM classification. The experimental results on several public datasets show that the proposed CNN based model outperforms some state-of-the-art methods.
Conference Paper
When creating a forgery, a forger can modify an image using many different image editing operations. Since a forensic examiner must test for each of these, significant interest has arisen in the development of universal forensic algorithms capable of detecting many different image editing operations and manipulations. In this paper, we propose a universal forensic approach to performing manipulation detection using deep learning. Specifically, we propose a new convolutional network architecture capable of automatically learning manipulation detection features directly from training data. In their current form, convolutional neural networks will learn features that capture an image's content as opposed to manipulation detection features. To overcome this issue, we develop a new form of convolutional layer that is specifically designed to suppress an image's content and adaptively learn manipulation detection features. Through a series of experiments, we demonstrate that our proposed approach can automatically learn how to detect multiple image manipulations without relying on pre-selected features or any preprocessing. The results of these experiments show that our proposed approach can automatically detect several different manipulations with an average accuracy of 99.10%.
Article
Seam-Carving is widely used for content-aware image resizing. To cope with the digital image forgery caused by Seam-Carving, a new detecting algorithm based on Benford's law is presented. The algorithm utilize the probabilities of the first digits of quantized DCT coefficients from individual AC modes to detect Seam-Carving images. The experimental result shows that the performance of proposed method is better than that of the method based on traditional Markov features and other existing methods.
Conference Paper
Digital forensics is a relatively new research area which aims at authenticating digital media by detecting possible digital forgeries. Indeed, the ever increasing availability of multimedia data on the web, coupled with the great advances reached by computer graphical tools, makes the modification of an image and the creation of visually compelling forgeries an easy task for any user. This in turns creates the need of reliable tools to validate the trustworthiness of the represented information. In such a context, we present here RAISE, a large dataset of 8156 high-resolution raw images, depicting various subjects and scenarios, properly annotated and available together with accompanying metadata. Such a wide collection of untouched and diverse data is intended to become a powerful resource for, but not limited to, forensic researchers by providing a common benchmark for a fair comparison, testing and evaluation of existing and next generation forensic algorithms. In this paper we describe how RAISE has been collected and organized, discuss how digital image forensics and many other multimedia research areas may benefit of this new publicly available benchmark dataset and test a very recent forensic technique for JPEG compression detection.
Article
Steganalysis and forgery detection in image forensics are generally investigated separately. We have designed a method targeting the detection of both steganography and seam-carved forgery in JPEG images. We analyze the neighboring joint density of the DCT coefficients and reveal the difference between the untouched image and the modified version. In realistic detection, the untouched image and the modified version may not be obtained at the same time, and different JPEG images may have different neighboring joint density features. By exploring the self-calibration under different shift recompressions, we propose calibrated neighboring joint density-based approaches with a simple feature set to distinguish steganograms and tampered images from untouched ones. Our study shows that this approach has multiple promising applications in image forensics. Compared to the state-of-the-art steganalysis detectors, our approach delivers better or comparable detection performances with a much smaller feature set while detecting several JPEG-based steganographic systems including DCT-embedding-based adaptive steganography and Yet Another Steganographic Scheme (YASS). Our approach is also effective in detecting seam-carved forgery in JPEG images. By integrating calibrated neighboring density with spatial domain rich models that were originally designed for steganalysis, the hybrid approach obtains the best detection accuracy to discriminate seam-carved forgery from an untouched image. Our study also offers a promising manner to explore steganalysis and forgery detection together.
Article
In this paper, we propose a scheme to detect the copy-move forgery in an image, mainly by extracting the keypoints for comparison. The main difference to the traditional methods is that the proposed scheme first segments the test image into semantically independent patches prior to keypoint extraction. As a result, the copy-move regions can be detected by matching between these patches. The matching process consists of two stages. In the first stage, we find the suspicious pairs of patches that may contain copy-move forgery regions, and we roughly estimate an affine transform matrix. In the second stage, an Expectation-Maximization-based algorithm is designed to refine the estimated matrix and to confirm the existence of copy-move forgery. Experimental results prove the good performance of the proposed scheme via comparing it with the state-of-the-art schemes on the public databases.
Conference Paper
The content-aware image retargeting algorithm is used for modifying the image size into the suitable size in different device. "Seam carving" is a kind of content aware image retargeting algorithm. In this paper, based on the blocking artifact characteristics matrix (BACM), we propose a method to detect seam carving in natural images without knowledge of the original image. In detail, for the original JPEG images, the BACM exhibits regular symmetrical shapes; for the images that are damaged, the regular symmetrical property of the BACM is destroyed. After found BACM from images, we define 18 features to detect the damage from BACM to train a support vector machine (SVM) classifier for recognizing whether an image is an original or it has been modified by seam-carving. We show that BACM is useful for detect the damage by seam-carving in JPEG format images.
Conference Paper
Seam carving is a content-aware image resizing method [Shamir and Avidan 2007], which assigns Sobel-operator-based energy to each pixel and describes seams as the eight-connected paths of pixels. Successive removal of the optimal seams, i.e., those seams with the lowest sum of energy, allows reduction in image size. Pixels with lower energy are generally removed earlier; implying that (1) the modifications to the image are difficult to identify and (2) low energy can be deliberately assigned to particular objects so that they can be removed from the image. These two observations reveal that, although difficult, it is important to design a seam carving detection method.
Conference Paper
Content-aware scaling is a method for image retargeting. It has been widely used in image manipulation including tampering. To improve the detection of the forgery in JPEG images, we propose to merge calibrated neighboring joint density and a rich models-based approach that was originally designed for steganalysis. A feature selection algorithm is utilized to reduce the feature dimensionality in the merged feature set. Experimental results show that the high-dimensional detector consisting of calibrated neighboring joint density and rich model features noticeably improves the detection accuracy; and the application of feature selection method to the high-dimensional detector can further improve the detection accuracy by using a much smaller and optimized feature set.
Article
Linear transformation, such as rotation, scaling, or any combinations of geometric attacks, is among the most common forms of image manipulation. This letter proposes a forensic technique that estimates the linear transformation of an investigated image. We exploited the periodic properties of interpolation by the second-derivative of the transformed image in both the row and column directions. Both the magnitude and phase information of the derived signals were analyzed to estimate the transformation matrix accurately. Empirical evidence from a large database of manipulated images indicates the superior performance of the proposed method.
Article
Seam carving is an adaptive multimedia retargeting technique to resize multimedia data for different display sizes. This technique has found promising applications in media consumption on mobile devices such as tablets and smartphones. However, seam carving can also be used to maliciously alter image content and when combined with other tampering operations, makes tampering detection very difficult by traditional multimedia forensic techniques. In this paper, we study the problem of seam carving estimation and tampering localization using very compact side information called forensic hash. The forensic hash technique bridges two related areas, namely robust image hashing and blind multimedia forensics, to answer a broader scope of forensic questions in a more efficient and accurate manner. We show that our recently proposed forensic hash construction can be extended to accurately estimate seam carving and detect local tampering.
Conference Paper
In Ref. 15, we took a critical view on the reliability of forensic techniques as tools to generate evidence of authenticity for digital images and presented targeted attacks against the state-of-the-art resampling detector by Popescu and Farid. We demonstrated that a correct detection of manipulations can be impeded by resampling with geometric distortion. However, we constrained our experiments to global image transformations. In a more realistic scenario, most forgeries will make use of local resampling operations, e.g., when pasting a beforehand scaled or rotated object. In this paper, we investigate the detectability of local resampling without and with geometric distortion and study the influence of the size both of the tampered and the analyzed image region. Although the detector might fail to reveal the characteristic periodic resampling artifacts, a forensic investigator can benefit from the generally increased correlation in resampled image regions. We present an adapted targeted attack, which allows for an increased degree of undetectability in the case of local resampling.
Article
When creating a digital forgery, it is often necessary to combine several images, for example, when compositing one person's head onto another person's body. If these images were originally of different JPEG compression quality, then the digital composite may contain a trace of the original compression qualities. To this end, we describe a technique to detect whether the part of an image was initially compressed at a lower quality than the rest of the image. This approach is applicable to images of high and low quality as well as resolution.
Article
The quick advance in image/video editing techniques has enabled people to synthesize realistic images/videos conveniently. Some legal issues may arise when a tampered image cannot be distinguished from a real one by visual examination. In this paper, we focus on JPEG images and propose detecting tampered images by examining the double quantization effect hidden among the discrete cosine transform (DCT) coefficients. To our knowledge, our approach is the only one to date that can automatically locate the tampered region, while it has several additional advantages: fine-grained detection at the scale of 8×8 DCT blocks, insensitivity to different kinds of forgery methods (such as alpha matting and inpainting, in addition to simple image cut/paste), the ability to work without fully decompressing the JPEG images, and the fast speed. Experimental results on JPEG images are promising.
Article
The unique stature of photographs as a definitive recording of events is being diminished due, in part, to the ease with which digital images can be manipulated and altered. Although good forgeries may leave no visual clues of having been tampered with, they may, nevertheless, alter the underlying statistics of an image. For example, we describe how resampling (e.g., scaling or rotating) introduces specific statistical correlations, and describe how these correlations can be automatically detected in any portion of an image. This technique works in the absence of any digital watermark or signature. We show the efficacy of this approach on uncompressed TIFF images, and JPEG and GIF images with minimal compression. We expect this technique to be among the first of many tools that will be needed to expose digital forgeries.
Conference Paper
Exemplar-based inpainting technique can be used to remove objects from an image and play visual tricks, which would affect the authenticity of images. In this paper, a blind detection method based on zero-connectivity feature and fuzzy membership is proposed to detect the specific doctoring. Firstly, zero-connectivity labeling is applied on block pairs to yield matching degree feature for all blocks in the region of suspicious, and fuzzy memberships are computed by constructing ascending semi-trapezoid membership function. Then the tampered regions are identified by a cut set. A num of natural and inpainted forged images are used to show the effectiveness of our method in detecting digital doctoring.
Article
The unique stature of photographs as a definitive recording of events is being diminished due, in part, to the ease with which digital images can be manipulated and altered. Although good forgeries may leave no visual clues of having been tampered with, they may, nevertheless, alter the underlying statistics of an image. For example, we describe how resampling (e.g., scaling or rotating) introduces specific statistical correlations, and describe how these correlations can be automatically detected in any portion of an image. This technique works in the absence of any digital watermark or signature. We show the efficacy of this approach on uncompressed TIFF images, and JPEG and GIF images with minimal compression. We expect this technique to be among the first of many tools that will be needed to expose digital forgeries.
2020) Detection, attribution and localization of gan generated images
  • M Goebel
  • L Nataraj
  • T Nanjundaswamy
  • T M Mohammed
  • S Chandrasekaran
  • B Manjunath
A convolutional neural network based seam carving detection scheme for uncompressed digital images
  • J Ye
  • Y Shi
  • G Xu
  • Y Q Shi
Ye, J., Shi, Y., Xu, G., Shi, Y.Q.: A convolutional neural network based seam carving detection scheme for uncompressed digital images. In: International Workshop on Digital Watermarking. pp. 3-13. Springer (2018)
Detection of image seam carving by using weber local descriptor and local binary patterns
  • D Zhang
  • Q Li
  • G Yang
  • L Li
  • X Sun
Zhang, D., Li, Q., Yang, G., Li, L., Sun, X.: Detection of image seam carving by using weber local descriptor and local binary patterns. Journal of information security and applications 36, 135-144 (2017)
Seam carving detection using convolutional neural networks
  • Lfs Cieslak
  • Da Costa
  • K A Paulopapa