Reference images in databases TID2008 and TID2013.

Reference images in databases TID2008 and TID2013.

Source publication
Article
Full-text available
This paper describes a recently created image database, TID2013, intended for evaluation of full-reference visual quality assessment metrics. With respect to TID2008, the new database contains a larger number (3000) of test images obtained from 25 reference images, 24 types of distortions for each reference image, and 5 levels for each type of dist...

Contexts in source publication

Context 1
... etalon colour images of various content. TID2008 contains 25 reference (distortion- free, etalon) colour images where 24 images were obtained (by cropping) from the Kodak database http:// r0k.us/graphics/kodak/. The 25-th reference image was artificially created and added to 24 natural scene images -see all 25 distortion-free images as shown in Fig. 1. As it can be seen, the test images are of different content, some of them are quite textural ones whilst others contain large quasi-homogeneous regions. Thus, the abovementioned requirements are ...
Context 2
... step set equal to 1.73σ. Such quantization step is chosen in order to provide a visibility of both distortions from compression noise and residual noise. Noise standard deviation has been indivi- dually adjusted for each test image to provide a desired value of PSNR. Examples of images with the considered type of distortions are presented in Fig. 10. As it can be seen, the distortions can be quite ...
Context 3
... particular attention nowadays [36]. Distortions of this type have been modelled using the Matlab function rgb2ind. It converts an RGB image to the indexed image using dither. To provide a desired PSNR, the number of quantization levels was adjusted individually for each test image. Examples of images with this type of distortions are shown in Fig. 11. Chromatic aberrations (distortion type #23) might take place at image acquisition stage but similar effects can also appear at stages of image transformations. It is quite annoying type of distortions especially in places of high contrasts and if a distortion level is high. Chromatic aberrations have been modelled by carrying out ...
Context 4
... at image acquisition stage but similar effects can also appear at stages of image transformations. It is quite annoying type of distortions especially in places of high contrasts and if a distortion level is high. Chromatic aberrations have been modelled by carrying out mutual shifting of R, G, and B components with respect to each other (see Fig. 12). ...
Context 5
... image Besides, further slight blurring of shifted components has been performed. Shifting and blurring parameters have been adjusted to provide a desired PSNR. An example is shown in Fig. ...
Context 6
... the last distortion type (#24) relates to compres- sive sensing (sparse sampling and reconstruction) that has become a hot research topic [37,38]. As far as we know, HVS- metrics have not been exploited in this area yet although their usefulness is expected. An example of distortions for this application is presented in Fig. 14 though they can depend upon a method of compressive sensing ...
Context 7
... us, it was convenient to use the method [38] and available software for obtaining reconstructed images with distortions. As earlier, modelling is carried out separately for components Y, Cb, and Cr. Some details for Y compo- nent are explained by Fig. ...
Context 8
... getting Reference image As it follows from the description presented above, the new types of distortions introduced in the database TID2013 are quite different. It took a lot of time to carry out extensive computations in order to provide desired levels of distortions. These computations have been partly automated to simplify the process (see Fig. ...
Context 9
... is interesting that in the resulting MOS there were no values equal to 0 or 9 (see MOS histogram in Fig. 17). Moreover, there were no MOS values larger than 7.5. This shows that conditions of comparisons were quite difficult especially for distorted images with rather high visual ...
Context 10
... us consider some other properties of MOS. Its values for all 3000 distorted images in the database are presented as scatterplot in Fig. 18 where first (leftmost) 120 points correspond to the reference image #1, next 120 points relate to the distorted images that have the same reference image #2 and so on. This scatterplot shows that MOS values are most dense within the interval from 3 to 6 and, thus, the task of comparing image visual quality was not ...
Context 11
... values averaged for all observers that carried out experiments are presented for each distortion type and level in Fig. 19. Each 5 level points for a given type of distortions (24 totally) are connected to see a tendency if it exists. For most types of distortions, the tendency is clear and obvious-average MOS decreases if distortion level becomes larger. The exceptions are Distortion types #15 and #17. Recall that distortion type #15 is Local block-wise ...
Context 12
... MOS decreases if distortion level becomes larger. The exceptions are Distortion types #15 and #17. Recall that distortion type #15 is Local block-wise distortions of different intensity where for level 1 one has a larger number of blocks than for other levels but contrasts of these blocks with respect to surrounding are smaller. The results in Fig. 19 show that for observers assessing visual quality it is more important what the total area of such blocks (that "hide" useful information) is than what the block contrasts are. Distortion type # 17 relates to Contrast change (see Table 1). Level 1 corresponds to small contrast decreasing, level 2-to small contrast increasing, level 3-to ...
Context 13
... information) is than what the block contrasts are. Distortion type # 17 relates to Contrast change (see Table 1). Level 1 corresponds to small contrast decreasing, level 2-to small contrast increasing, level 3-to a larger contrast decreasing, level 4-to a larger contrast increasing, level 5-to the largest contrast decreasing. The results in Fig. 19 clearly show that contrast increasing is perceived as better than contrast decreasing. However, there is certain "optimal" contrast change that approximately corresponds to level ...
Context 14
... now Mean RMSE of MOS depending upon reference image and distortion level. The corresponding data are represented in Fig. ...
Context 15
... our opinion, such results can be attributed to several factors. First, there is high density of MOS (see its histo- gram, in Fig. 17) for "Good quality" image group. For many images of this group, distortions are practically invisible and then experiment participants, having met with two such images presented at the monitor, select the best quality image in a random manner. Second, for "Bad quality" image group, experiment participants pay less attention to visual ...

Citations

... FCA principle involves obtaining the major features matrix and building a quality measurement reference. Extensive experiments have been performed on four common image databases (LIVE [33], CSIQ [34], and TID2013 [35]) and some other images are being investigated. The results are compared with existing approaches in the domain. ...
... We present our experimental configuration and examine the accuracy of our approach in this section. The proposed image quality assessment was evaluated using three well-known image databases (LIVE [33], CSIQ [34] and TID2013 [35]) and some other images. These databases are very popular as IQA datasets, which contain images with various types of distortions. ...
Preprint
Full-text available
The assessment of image quality is an imperative aspect that provides and uses information regarding the confidentiality, safety, quality, and transparency of the obtained images. There are a lot of existing quality metric approaches that belong to the group of modification alteration procedures (pixel-based metric). These techniques have not been well-matched with perceptual image quality. The paper presents a new reduced reference image quality metric in the spatial domain that reduces the complexity. Perceptual Human Visual System (HVS) aspect drives our approach, however, visual quality is heavily influenced by the instruments of the human brain and visual system. From the reference and distortion images, we extract four intrinsic features: contrast, entropy, histogram, and standard deviation. Then, we build the formal concept analysis matrix for reference and distortion images. Finally, we compare the obtained matrixes to evaluate the image quality. The performance of the proposed technique is assessed using LIVE, TID2013, and CSIQ datasets, and the obtained results are compared in terms of PSNR, SSIM, and NCC metrics. Also, a comparison with more recent and relevant approaches are performed to highlight the superior performance of our proposed approach, the experimental results indicated that the proposed approach provides efficient performance among the compression, Gaussian blur, Contrast, and add noise distortion types.
... Corresponding author: Shiqi Wang. [18]; (b) DNV of reference images in SIQAD dataset [19]; (c) The deep feature statistics of reference images in SIQAD dataset obtained by our proposed method. regions for SCIQA [9]. ...
Preprint
Full-text available
The statistical regularities of natural images, referred to as natural scene statistics, play an important role in no-reference image quality assessment. However, it has been widely acknowledged that screen content images (SCIs), which are typically computer generated, do not hold such statistics. Here we make the first attempt to learn the statistics of SCIs, based upon which the quality of SCIs can be effectively determined. The underlying mechanism of the proposed approach is based upon the wild assumption that the SCIs, which are not physically acquired, still obey certain statistics that could be understood in a learning fashion. We empirically show that the statistics deviation could be effectively leveraged in quality assessment, and the proposed method is superior when evaluated in different settings. Extensive experimental results demonstrate the Deep Feature Statistics based SCI Quality Assessment (DFSS-IQA) model delivers promising performance compared with existing NR-IQA models and shows a high generalization capability in the cross-dataset settings. The implementation of our method is publicly available at https://github.com/Baoliang93/DFSS-IQA.
... In this paper, we applied five publicly available IQA benchmark, such as CLIVE [21], KonIQ-10k [22], SPAQ [23], TID2013 [24], and KADID-10k [25], to evaluate and compare our proposed methods to the state-of-the-art. Specifically, CLIVE [21], KonIQ-10k [22], and SPAQ [23] contain unique quality labeled images with authentic distortions. ...
... On the other hand, there is no fixed resolution in SPAQ [23] but the images have high resolution which varies around 4000 × 4000. In contrast to CLIVE [21], KonIQ-10k [22], and SPAQ [23], TID2013 [24] and KADID-10k [25] consist of 24 and 25 reference images whose perceptual quality are considered perfect, respectively. The quality labeled distorted images were produced artificially by an image processing tool from the reference images using different distortion types (i.e., JPEG compression noise, salt & pepper noise, Gaussian blur, etc.) at multiple distortion levels. ...
... The empirical distributions of quality scores in the applied IQA databases. (a) CLIVE [21], (b) KonIQ-10k [22], (c) SPAQ [23], (d) TID2013 [24], (e) KADID-10k [25]. ...
Article
Full-text available
Objective quality assessment of natural images plays a key role in many fields related to imaging and sensor technology. Thus, this paper intends to introduce an innovative quality-aware feature extraction method for no-reference image quality assessment (NR-IQA). To be more specific, a various sequence of HVS inspired filters were applied to the color channels of an input image to enhance those statistical regularities in the image to which the human visual system is sensitive. From the obtained feature maps, the statistics of a wide range of local feature descriptors were extracted to compile quality-aware features since they treat images from the human visual system’s point of view. To prove the efficiency of the proposed method, it was compared to 16 state-of-the-art NR-IQA techniques on five large benchmark databases, i.e., CLIVE, KonIQ-10k, SPAQ, TID2013, and KADID-10k. It was demonstrated that the proposed method is superior to the state-of-the-art in terms of three different performance indices.
... • Firstly, we assessed our Neural Network's accuracy by predicting the quality score of the no-reference images from two datasets: LIVE [52], and TID2013 [47]. For each image in the dataset, a subjective quality score, i.e., the mean opinion score (MOS), is assigned to validate our proposed method. ...
... The objective of this test is to evaluate the performance of our neural network by comparing its prediction scores of the no-reference images to the subjective ratings. As mentioned before, the two largest publicly available subject-related databases used are: LIVE [52], and TID2013 [47]. To evaluate the performance of our method, two correlation coefficients between the prediction results and the subjective scores are adopted: the Spearman Rank Order Correlation Coefficient (SROCC), which is related to how well the relationship between two variables can be described using a monotonic function, and the Pearson Correlation Coefficient (PCC), which is related to the measurement of the linear correlation between two variables. ...
Article
Full-text available
These days, social media holds a large portion of our daily lives. Millions of people post their images using a social media platform. The enormous amount of images shared on social network presents serious challenges and requires massive computing resources to ensure efficient data processing. However, images are subject to a wide range of distortions in real application scenarios during the processing, transmission, sharing, or combination of many factors. So, there is a need to guarantee acceptable delivery content, even though some distorted images do not have access to their original version. In this paper, we present a framework developed to process a large amount of images in real-time while estimating and assisting in the enhancement of the No-Reference and Full-Reference image quality. Our quality evaluation is measured using a Convolutional Neural Network, which is tuned by the objective quality methods, in addition to the face alignment metric and enhanced with the help of a Super-Resolution Model. A set of experiments is conducted to evaluate our proposed approach.
... Five variants, namely JPEG, JPEG2000, Gblur, WN, and FF distortion variants from LIVE [34,36] are used for the experimentations. TID2008 [27,28] and TID2013 [29,30] both datasets do not have FF distortion variant, hence remaining four are considered. These training and testing datasets are used for the experimentations carried out throughout this research study. ...
Article
Full-text available
No reference method for image quality assessment using shape adaptive wavelet features by applying neuro-wavelet model is proposed in this paper. Images usually consist of visual objects. Degradation of an image ultimately causes distortions to the objects present in the image. Distortions can change the shape of these objects. Quality assessment of an image cannot be said to be complete without assessing the quality of individual objects present in the image. Therefore, deviation in shape has to be quantified along with the quality assessment of an image. Shape Adaptive Discrete Wavelet Transform offers a solution to shape identification problem. The variations in magnitude of feature values are found not proportional to the amount of degradation due to the presence of other artifacts. Wavelet decomposition is applied to capture the small variations observed in extracted features. Separate back propagation neural network models are trained for quality assessment of all kind of images ranging from pristine to bad. Results show improvement in accuracy independent of image databases. It has been observed that the predicted score correlates well with the mean opinion score with 90% accuracy for LIVE dataset, 93% and 95% for TID2008 and TID2013 respectively.
... IQA datasets are typically designed to evaluate existing IQMs on their ability to answer previously not considered questions such as IQ from two different view distances (CID dataset [58]), IQ on images perturbed with multiple types of distortions (MDID [64], MD-IVL [62] and VLC [73] datasets), evaluation of GAN-based image restoration algorithms (PIPAL [70] and NTIRE 2021 [71] challenges and datasets). Another commonly used datasets are LIVE [54], TID2013 [55] and KADID-10k [66]. Some works even try to propose a method to construct a general-case IQA dataset with extremely diverse image characteristics [74]. ...
... We selected TID2013 [55] and KADID-10k [66] databases as one of the most popular in the research community and PIPAL [70] as the one with a higher variety of introduced distortions. After that, we selected an evaluation criterion (type of correlation) that can be computed on selected datasets and compared with the reference values. ...
... MSE and PSNR have been repeatedly shown to poorly correlate with human judgements in controlled experiments [55,66]. The main reason for this is four strong underlying assumptions about visual quality [79]: 1. Independence of spatial relationships between samples, 2. Independence of relationship between signal and error, 3. Independence of sign of error samples, 4. Equal importance of all signal samples and errors. ...
Preprint
Full-text available
Image Quality Assessment (IQA) metrics are widely used to quantitatively estimate the extent of image degradation following some forming, restoring, transforming, or enhancing algorithms. We present PyTorch Image Quality (PIQ), a usability-centric library that contains the most popular modern IQA algorithms, guaranteed to be correctly implemented according to their original propositions and thoroughly verified. In this paper, we detail the principles behind the foundation of the library, describe the evaluation strategy that makes it reliable, provide the benchmarks that showcase the performance-time trade-offs, and underline the benefits of GPU acceleration given the library is used within the PyTorch backend. PyTorch Image Quality is an open source software: https://github.com/photosynthesis-team/piq/.
... According to CNN-SVR and CNN-LSTM, the evaluation is performed on KoNViD-1k [16]. In the case of DeepBIQ both authentic and artificial IQA datasets are considered with LIVEin-the-wild [17] and TID2013 [18], respectively. MultiGAP-NRIQA considers two more modern artificially distorted IQA datasets KADID-10k [19] and TID2013 [18], as well as the modern in-the-wild IQA dataset KonIQ-10k [20]. ...
... In the case of DeepBIQ both authentic and artificial IQA datasets are considered with LIVEin-the-wild [17] and TID2013 [18], respectively. MultiGAP-NRIQA considers two more modern artificially distorted IQA datasets KADID-10k [19] and TID2013 [18], as well as the modern in-the-wild IQA dataset KonIQ-10k [20]. For CNN-SVR, CNN-LSTM, and MultiGAP-NRIQA the best performance was achieved using an Inception-V3 network architecture as a baseline feature extraction network. ...
Article
Full-text available
Data used to train supervised machine learning models are commonly split into independent training, validation, and test sets. This paper illustrates that complex data leakage cases have occurred in the no-reference image and video quality assessment literature. Recently, papers in several journals reported performance results well above the best in the field. However, our analysis shows that information from the test set was inappropriately used in the training process in different ways and that the claimed performance results cannot be achieved. When correcting for the data leakage, the performances of the approaches drop even below the state-of-the-art by a large margin. Additionally, we investigate end-to-end variations to the discussed approaches, which do not improve upon the original.
... Existing OIQA methods have been successful, but are difficult in evaluating some specific image processing tasks [11,12]. Traditional assessment metrics such as Peak Signal-to-Noise Ratio (PSNR) [13] and structural similarity index (SSIM) [14] are widely used to evaluate image quality, but they are ineffective for images obtained by GANs-based methods. ...
... Comparison experiments are established based on three public databases, including the LIVE Image Quality Assessment Database (LIVE) [35], the Categorical Subjective Image Quality (CSIQ) database [25], and the Tampere image database 2013 (TID2013) [12]. Furthermore, according to the official information provided, we evaluated the performance of the proposed method with other IQA algorithms on the NTIRE 2022 FR Track. ...
... Using Pearson linear correlation coefficient (PLCC), the Spearman rank correlation coefficient (SRCC), and the Kendall rank correlation coefficient (KRCC) as evaluation criteria, results are reported in Tab. 3. As same as the NTIRE 2022 Percep- Table 3. Comparison experiment of IQA methods on three standard IQA databases, i.e., LIVE [35], CSIQ [25], and TID2013 [12], in terms of PLCC, SRCC, KRCC and MS. The top two performing methods are highlighted in bold face. ...
Preprint
In this work, we introduce Gradient Siamese Network (GSN) for image quality assessment. The proposed method is skilled in capturing the gradient features between distorted images and reference images in full-reference image quality assessment(IQA) task. We utilize Central Differential Convolution to obtain both semantic features and detail difference hidden in image pair. Furthermore, spatial attention guides the network to concentrate on regions related to image detail. For the low-level, mid-level and high-level features extracted by the network, we innovatively design a multi-level fusion method to improve the efficiency of feature utilization. In addition to the common mean square error supervision, we further consider the relative distance among batch samples and successfully apply KL divergence loss to the image quality assessment task. We experimented the proposed algorithm GSN on several publicly available datasets and proved its superior performance. Our network won the second place in NTIRE 2022 Perceptual Image Quality Assessment Challenge track 1 Full-Reference.
... TID2013 [27] LIVE [35] CSIQ [21] IVC [25] PLCC ...
... We evaluate the performance of DeepWSD on six IQA datasets, including CSIQ [21], TID2013 [27], LIVE [35], IVC [25], KADID-10k [22] and LIVE-MultiDist [18]. Those databases contain various distortion types and distortion levels. ...
Preprint
Full-text available
Existing deep learning-based full-reference IQA (FR-IQA) models usually predict the image quality in a deterministic way by explicitly comparing the features, gauging how severely distorted an image is by how far the corresponding feature lies from the space of the reference images. Herein, we look at this problem from a different viewpoint and propose to model the quality degradation in perceptual space from a statistical distribution perspective. As such, the quality is measured based upon the Wasserstein distance in the deep feature domain. More specifically, the 1DWasserstein distance at each stage of the pre-trained VGG network is measured, based on which the final quality score is performed. The deep Wasserstein distance (DeepWSD) performed on features from neural networks enjoys better interpretability of the quality contamination caused by various types of distortions and presents an advanced quality prediction capability. Extensive experiments and theoretical analysis show the superiority of the proposed DeepWSD in terms of both quality prediction and optimization.
... Finally, on the basis of the integrated features, the IQA model is trained via SVR [35]. The performance comparisons are conducted on seven publicly accessible image databases: LIVE [36], CSIQ [37], TID2013 [38], LIVE multiply distorted image database (LIVEMD) [39], LIVE in the wild image quality challenge database (LIVEwild) [40], SIQAD [41] and QACS [42]. The results demonstrate that the proposed method is suitable for both natural image and SCI. ...
... The performance comparisons are conducted on seven publicly available databases: LIVE [36], CSIQ [37], TID2013 [38], LIVEMD [39], LIVEwild [40], SIQAD [41] and QACS [42]. LIVEMD is divided into LiveMD1 and LiveMD2, which are corresponding to two multiple distortions. ...
Article
Full-text available
Image quality is an important metric for measuring multimedia services; thus, analysing image quality accurately has high practicability. The existing image quality assessment (IQA) methods mainly focus on grey information, which underutilize the colour information. In this paper, a new IQA method is proposed to make full use of colour information. The method calculates the colour information fluctuation (CF) around each pixel in different detection directions, and obtain CF maps (CFMs). Meanwhile, the greyscale fluctuations (GFs) are analysed to extract GF maps (GFMs). Based on CFMs and GFMS, the direction information is calculated to form CF direction map (CFD) and GF direction map (GFD). After that, the histogram-based features are extracted from CFMs, GFMs, CFDs and GFDs. Finally, different features are combined to measure quality variations comprehensively. The performance comparisons demonstrate the proposed method is competitive with prevalent methods, and is suitable for both natural image and screen content image.