No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Color laparoscopic image region segmentation after contrast enhancement including SRCNN by image regions
Abstract
As one of image pre-processing method to detect, recognize, and estimate lesion or characteristic region in medical image processing, there are many studies improved performance and precision of processing by contrast enhancement or super-resolution.
However, it is not clarified how condition is better to apply these methods. Therefore, we experimented and discussed on affect for color laparoscopic image quality by the difference of contrast enhancement method.
As a result, we obtained knowledge of high similarity among patterns of adaptive histogram equalization in three methods.
However, under these conditions, in the case of considering the region segmentation, it is not clarified how processing precision is better.
In this paper, first we processed the contrast enhancement for the color laparoscopic frame image cut from surgery video under laparoscopy.
Next, we processed super-resolution for generated image.
Finally, we compared and discussed by Peak Signal to Noise Ratio (PSNR), Structural SIMilarity (SSIM), and texture features for contrast.
... As a previous study, the author conducted research on Region Of Interest (ROI) estimation and visual attention in 3D CG images based on saliency maps. 3 As a result, we did not find much dependence on 3D CG contents, and we thought that it might be possible to identify saliency by utilizing both Spectral Residual (SR) and Fine Grained (FG). 4,5 On the other hand, in electronic medical record systems and computer-aided surgery in the medical field, it is necessary to handle medical information by actively utilizing artificial intelligence and obtaining some clues in order to make more accurate image diagnosis in addition to the judgment of medical personnel's eyes on areas in medical images that are assumed to be important in some way. ...
... In this study, we first attempted to contrast-enhance images using laparoscopic images. 3 Then, we experimentally verified how the estimated region of interest changed before and after image processing using saliency maps, and discussed the results. In addition, we measured the area of the image and extracted specific numerical values to visualize the estimated region of interest. ...
... In order to reproduce this very high level of human intelligence on a computer, thus far, there were many studies using computer image processing to select image information according to its importance [1]. In this study, we first attempted to contrast-enhance images using laparoscopic images [2]. Then, we experimentally verified how the estimated region of interest changed before and after image processing using saliency maps, and discussed the results. ...
... Next, we studied on affect to color laparoscopic image quality for the difference of contrast enhancement method. As a result of comparison using three types of contrast enhancement methods (gamma correction, histogram equalization, and adaptive histogram equalization), the difference among each results is seen, in particular, contents of adaptive histogram equalization are similar to high, and similar to low for the difference among patterns [4], [5]. From overall of this background, it is not enough to compare to other neural network methods. ...
The Generative Adversarial Networks (GAN) is unsupervised learning enabled to transform according to data characteristics, though this generate unreal data by learning characteristics from data. As past our study, we discussed from the viewpoint of image quality for super-resolution of color laparoscopic image including SRCNN (Super-Resolution Convolutional Neural Network). However, it was not enough to compare to other neural network methods in our discussion. We consider that it is possible to support the medical image diagnosis by measuring whether the difference of both neural network method and image contents is affected or not for image quality. In this paper, first we carried out the objective image quality assessment by designing optimally of color laparoscopic super-resolution image using Generative Adversarial Networks (GAN). And then, we discussed for performance between methods comparing to result of SRCNN. On the other hand, from a view of information science, we consider that we need to verify experimentally for affect between network learning effect and generated image, and then to improve method. Therefore, we also discussed relationship between image quality and learning effect in color laparoscopic image generation using SRGAN.
In recent years, the rapid advancement of image generation techniques has resulted in the widespread abuse of manipulated images, leading to a crisis of trust and affecting social equity. Thus, the goal of our work is to detect and localize tampered regions in images. Many deep learning based approaches have been proposed to address this problem, but they can hardly handle the tampered regions that are manually fine-tuned to blend into image background. By observing that the boundaries of tempered regions are critical to separating tampered and non-tampered parts, we present a novel boundary-guided approach to image manipulation detection, which introduces an inherent bias towards exploiting the boundary information of tampered regions. Our model follows an encoder-decoder architecture, with multi-scale localization mask prediction, and is guided to utilize the prior boundary knowledge through an attention mechanism and contrastive learning. In particular, our model is unique in that 1) we propose a boundary-aware attention module in the network decoder, which predicts the boundary of tampered regions and thus uses it as crucial contextual cues to facilitate the localization and 2) we propose a multi-scale contrastive learning scheme with a novel boundary-guided sampling strategy, leading to more discriminative localization features. Our state-of-art performance on several public benchmarks demonstrates the superiority of our model over prior works.
Owing to refraction, absorption, and scattering of light by suspended particles in water, raw underwater images have low contrast, blurred details, and color distortion. These characteristics can significantly interfere with visual tasks such as segmentation and tracking. This paper proposes an underwater image enhancement solution through a deep residual framework. First, the cycle-consistent adversarial networks (CycleGAN) is employed to generate synthetic underwater images as training data for convolution neural network models. Second, the very-deep super-resolution reconstruction model (VDSR) is introduced to underwater resolution applications; with it, the Underwater Resnet model is proposed, which is a residual learning model for underwater image enhancement tasks. Furthermore, the loss function and training mode are improved. A multi-term loss function is formed with mean squared error loss and a proposed edge difference loss. An asynchronous training mode is also proposed to improve the performance of the multi-term loss function. Finally, the impact of batch normalization is discussed. According to the underwater image enhancement experiments and a comparative analysis, the color correction and detail enhancement performance of the proposed methods are superior to that of previous deep learning models and traditional methods.
High-definition display technology for underwater images is of great significance for many applications, such as marine animal observation, seabed mining and marine fishery production. The traditional underwater visual display systems have problems such as low visibility, poor real-time performance, and low resolution, and cannot meet the needs of real-time high-definition displays in extreme environments. To solve these issues, we propose an underwater image enhancement method and a corresponding image superresolution algorithm. To improve the quality of underwater images, we modify the Retinex algorithm and combine it with a neural network. The Retinex algorithm is used to defog the underwater image, and then the image brightness is improved by applying a gamma correction. Then, by combining with the dark channel prior and multilayer perceptron, the transmission map is further refined to improve the dynamic range of the image. In the superresolution process, the current convolutional neural network reconstruction algorithm is only trained on the Y channel, which will lead to problems due to the insufficient acquisition of the color feature. Therefore, an image superresolution reconstruction algorithm that is based on color features is proposed. The experimental results show that the proposed method improves the reconstruction effect of the image edges and texture details, increases the image clarity, and enhances the image color recovery.
The authors propose a novel model called concatenated recursive compressor-decompressor network (CRCDNet) for contrast-enhanced super-resolution. The characteristics of authors' model can be summarised as follows. First, a compression-decompression process reduces the computational complexity compared with the general fully convolutional model. Second, an internal/external skip-connection is used to preserve information of the preceding layers. Finally, by employing a recursive module, authors' model has a small number of parameters, yet is a deep and robust network. The authors apply authors' proposed network to license plate images. As a real application, license plates can provide important evidence for investigation of crimes and for security, but it is very difficult to collect the vast amounts of license plates required for analysis based on a data-driven approach. To solve this problem, the authors generated virtual datasets to train authors' model, while analysing the performance with real license plate datasets. Authors' method achieves better performance than the state-of-the-art models on license plate images.
Training a deep convolutional neural network (CNN) from scratch is difficult because it requires a large amount of labeled training data and a great deal of expertise to ensure proper convergence. A promising alternative is to fine-tune a CNN that has been pre-trained using, for instance, a large set of labeled natural images. However, the substantial differences between natural and medical images may advise against such knowledge transfer. In this paper, we seek to answer the following central question in the context of medical image analysis: Can the use of pre-trained deep CNNs with sufficient fine-tuning eliminate the need for training a deep CNN from scratch? To address this question, we considered four distinct medical imaging applications in three specialties (radiology, cardiology, and gastroenterology) involving classification, detection, and segmentation from three different imaging modalities, and investigated how the performance of deep CNNs trained from scratch compared with the pre-trained CNNs fine-tuned in a layer-wise manner. Our experiments consistently demonstrated that 1) the use of a pre-trained CNN with adequate fine-tuning outperformed or, in the worst case, performed as well as a CNN trained from scratch; 2) fine-tuned CNNs were more robust to the size of training sets than CNNs trained from scratch; 3) neither shallow tuning nor deep tuning was the optimal choice for a particular application; and 4) our layer-wise fine-tuning scheme could offer a practical way to reach the best performance for the application at hand based on the amount of available data.
A novel hybrid region-based active contour model is presented to segment medical images with intensity inhomogeneity. The energy functional for the proposed model consists of three weighted terms: global term, local term, and regularization term. The total energy is incorporated into a level set formulation with a level set regularization term, from which a curve evolution equation is derived for energy minimization. Experiments on some synthetic and real images demonstrate that our model is more efficient compared with the localizing region-based active contours (LRBAC) method, proposed by Lankton, and more robust compared with the Chan-Vese (C-V) active contour model.
Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu/∼lcv/ssim/.
In medical images, since there are body region and border that it is hard for medical worker to distinguish by the only image diagnosis, we estimate that the progress of work, time, and emergency are needed. Therefore, it is problem of emergency to develop the medical information system enable to support medical workers by using high performance computer. In this study, first, we carried out texture analysis for region of laparoscopic image. Next, based on results, we experimented whether region segmentation of laparoscopic image is possible or impossible. Finally, we discussed how each texture features are affected to region segmentation.
High-resolution diffusion imaging with
submillimeter isotropic voxels requires long scan
times that are usually clinically impractical. Even
with those long scans, the image quality can still
suffer from low signal-to-noise ratio (SNR) and
severe geometric distortion due to long echo spacing
in echo-planar imaging sequences. In this study, we
proposed and validated the efficacy of using a state-of-
the-art deep-learning method, super-resolution
convolutional neural network (SRCNN), to achieve
submillimeter super-resolution diffusion-weighted
(DW) images. The 2D-based deep-learning method
was validated by comparing with the ground truth
using numerical simulations and by studying
region-of-interest (ROI) using real human data of
three healthy volunteers. Furthermore, we
interrogated the proposed method under different
real-life SNR conditions. The results demonstrated
that the proposed deep- learning method was able to
reproduce sufficient details in the anatomy that can
only be detected using high-resolution diffusion
imaging. The percentage errors in diffusion tensor
imaging (DTI) derived metrics were less than 8%
when the baseline SNR larger than 20. The ROI
results demonstrated that the proposed method
produced comparable values of diffusion metrics to
the matched high-resolution diffusion metrics of
real human data. Particularly, the patterns of
distributions of the subjects were similar between
the proposed method and real data across whole brain
gray-matter and white-matter ROIs. A deep learned
submillimeter resolution of 0.625 mm
diffusion directional image showed high image
quality, particularly in the cortical gray matter. We
demonstrated the feasibility of using a deep learning
algorithm based on SRCNN in DTI. This
approach can be a robust alternative when
acquiring the true sub-millimeter diffusion MRI is
not available.
Traditional Visual Secret Sharing (VSS) scheme encrypts the secret image into multiple shares. It recovers the secret image based on the “all or nothing” methodology (i.e. all shares must be stacked together to recover the secret image else nothing will be revealed). The modern VSS schemes differ from traditional VSS by progressively recovering the secret image by stacking number of shares. These are also referred as Progressive VSS (PVSS). PVSS schemes generate two types of shares namely 1) meaningful (are the shares which have a meaningful image embedded as the cover image on top of the shares) and 2) noise-like (shares have a random noise-like appearance). In the previous work, we have proposed PVSS based Randomized VSS (RVSS) scheme to recover the secret image by hiding random data into the shares. RVSS achieves the maximum contrast of 70–80% and 70–90% for meaningful and noise-like shares respectively. In this paper, we propose a novel PVSS based super-resolution technique to improve the contrast of RVSS scheme. The experimental results showed that the proposed scheme achieves the contrast of 70–80% for meaningful shares and 99% for noise-like shares. Also, proposed scheme recovers the secret image free from blocking artifacts, for noise-like shares.
Up to now, the diagnostic imaging was carried out based on manual handling by professional doctors and health care workers. However, it is enable to diagnose images automatically by development of the computer systems. Therefore, it is required for approach from information science and engineering fields. For generating multi-view 3D images, if we are able to support whether workers are able to use correctly or not, and if the coded defect detection and restoration for their images are possible, we consider possibility towards medical applications in the near future. In this paper, first, we diagnosed automatically for multi-view 3D images in the case of occurring defect by encoded and decoded degradation at all or certain viewpoints by H.265/HEVC. Next, we assessed and estimated quantitatively in terms of the coded image quality in order to clarify how we are able to detect the coded defect.
We can come to approach on super-resolution processing based on deep learning by appearing deep learning tools. These performance are shown by applying the only deep learning theory for super-resolution processing. However, we consider that the optimal condition and design for super-resolution processing are achieved better by improving these algorithms and setting parameter appropriately. In this paper, first, we carried out experiments on optimal condition and design of super-resolution processing for the multi-view 3D images encoded and decoded by H.265/HEVC, focused on structure of convolutional neural network by using Chainer. And then, we assessed for the generated images quality objectively, and compare to each image. Finally, we discussed for experimental results.
Previously, it is not obvious to what extent was accepted for the assessors when we see the 3D image (including multi-view 3D) which the luminance change may affect the stereoscopic effect and assessment generally. We think that we can conduct a general evaluation, along with a subjective evaluation, of the luminance component using both the S-CIELAB color space and CIEDE2000. In this study, first, we performed three types of subjective evaluation experiments for contrast enhancement in an image by using the eight viewpoints parallax barrier method. Next, we analyzed the results statistically by using a support vector machine (SVM). Further, we objectively evaluated the luminance value measurement by using CIEDE2000 in the S-CIELAB color space. Then, we checked whether the objective evaluation value was related to the subjective evaluation value. From results, we were able to see the characteristic relationship between subjective assessment and objective assessment.
There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at
http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net
.
A novel super-resolution reconstruction (SRR) framework in magnetic resonance imaging (MRI) is proposed. Its purpose is to produce images of both high resolution and high contrast desirable for image-guided minimally invasive brain surgery. The input data are multiple 2-D multislice inversion recovery MRI scans acquired at orientations with regular angular spacing rotated around a common frequency encoding axis. The output is a 3-D volume of isotropic high resolution. The inversion process resembles a localized projection reconstruction problem. Iterative algorithms for reconstruction are based on the projection onto convex sets (POCS) formalism. Results demonstrate resolution enhancement in simulated phantom studies, and ex vivo and in vivo human brain scans, carried out on clinical scanners. A comparison with previously published SRR methods shows favorable characteristics in the proposed approach.
Many applications of histograms for the purposes of image processing are well known. However, applying this process to the transform domain by way of a transform coefficient histogram has not yet been fully explored. This paper proposes three methods of image enhancement: a) logarithmic transform histogram matching, b) logarithmic transform histogram shifting, and c) logarithmic transform histogram shaping using Gaussian distributions. They are based on the properties of the logarithmic transform domain histogram and histogram equalization. The presented algorithms use the fact that the relationship between stimulus and perception is logarithmic and afford a marriage between enhancement qualities and computational efficiency. A human visual system-based quantitative measurement of image contrast improvement is also defined. This helps choose the best parameters and transform for each enhancement. A number of experimental results are presented to illustrate the performance of the proposed algorithms.
In this paper, a novel contrast enhancement algorithm is proposed. The proposed approach enhances the contrast without losing the original histogram characteristics, which is based on the histogram specification technique. It is expected to eliminate the annoying side effects effectively by using the differential information from the input histogram. The experimental results show that the proposed dynamic histogram specification (DHS) algorithm not only keeps the original histogram shape features but also enhances the contrast effectively. Moreover, the DHS algorithm can be applied by simple hardware and processed in real-time system due to its simplicity.