Content uploaded by Vassili Kovalev
Author content
All content in this area was uploaded by Vassili Kovalev on Apr 07, 2018
Content may be subject to copyright.
Highlighting Tumor Borders Using Generalized Gradient
Vassili Kovalev
1(✉)
, Eduard Snezhko
1
, Siarhei Kharuzhyk
2
, and Vitali Liauchuk
1
1Biomedical Image Analysis Group and Mathematical Cybernetics Department,
United Institute of Informatics Problems, Surganova Street, 6, 220012 Minsk, Belarus
vassili.kovalev@gmail.com
2N.N. Alexandrov National Cancer Center of Belarus, 223043 Minsk, Lesnoy District, Belarus
Abstract. This paper presents a generalized approach for computing image
gradient. It is predominantly aimed at detecting unclear and in certain circumstances
even completely invisible borders in large 2D and 3D texture images. The method
exploits the conventional approach of sliding window. Once two pixel/voxel sets are
sub-sampled from orthogonal window halves, they are compared by a suitable tech‐
nique (e.g., statistical t-test, SVM classier, comparison of parameters of two distribu‐
tions) and the resultant measure of difference (e.g., t-value, the classification accu‐
racy, skewness difference of two distributions etc.) is treated as the gradient magni‐
tude. The bootstrap procedure is employed for increasing the accuracy of difference
assessment of two pixel/voxel sets.
Keywords: Lung tumor · CT · Edge detection · Generalized gradient
1 Introduction
In many occasions there is a strong need for detection of borders of objects which look like
patterns of random textures. Such borders could be hardly detected by human visual system
when textures differ by their high-order statistics only [1, 2]. Recently, some advanced
methods of detecting hardly visible borders between the random image textures have been
suggested [1, 3]. Moreover, it was experimentally proven that these methods capitalizing on
so-called “generalized gradient” are able to highlight the border which is completely invis‐
ible for human eye [4, 5].
This paper addresses the problem of detecting borders of malignant tumors in native lung
CT images under conditions of presence of atelectasis. The atelectasis term denotes the
collapse of all or part of a lung due to bronchial plugging or the chest cavity being opened
to atmospheric pressure. This can happen when the vacuum between the lung and chest
wall is broken, allowing the lung to collapse within the chest (e.g., pneumothorax), when the
lung is compressed by masses in the chest, or when an airway is blocked, leading to slow
absorption of the distal air into the blood without replenishment. In this work we were
dealing with the bronchial compression caused by lung cancer tumors, the most common
cause of the atelectasis.
Computed tomography (CT) is the primary modality for imaging lung cancer patients.
However, the problem is that on CT scans the lung regions with the atelectasis and malig‐
nant tumors have quite similar attenuation values. Therefore the visual discrimination and
© Springer International Publishing AG 2017
V.V. Krasnoproshin and S.V. Ablameyko (Eds.): PRIP 2016, CCIS 673, pp. 86–96, 2017.
DOI: 10.1007/978-3-319-54220-1_9
separation of the atelectasis and tumor is hardly possible. Yet, accurate tumor segmentation
is strongly necessary by the following two reasons. First, the correct tumor localization,
segmentation, and precise measurement of tumor diameter play a crucial role in therapy
planning and choosing suitable surgery technique. Second, if the radiation therapy is
prescribed, an exact separation of tumor border is required for precise targeting and delivery
of the ionizing radiation dose accurately to the tumor but not to the surrounding tissues.
Thus, the purpose of this particular paper is to present results of an experimental study
of the ability of the generalized gradient method to highlight hardly visible borders of
objects. The study was conducted using three different groups of images. They were
comprised by 3D synthetic images and specially-designed physical gelatin phantom made
by authors and scanned using Siemens Somatom Definition AS scanner. Finally, the utility
of the method was examined on the problem of borders detection between malignant lung
tumors and the atelectasis regions based on 3D CT images of 40 lung cancer patients.
The first version of the generalized gradient method was introduced in [2] as so-called
classification gradient and slightly improved afterwards. The classification gradient method
makes use conventional technique of calculating image gradient at each pixel position by
means of comparing pixel/voxel values taken from orthogonal halves of appropriately sized
sliding window. However, apart from the traditional approaches where the gradient magni‐
tude is computed simply as the intensity difference (estimated by convolution with one or
other matrix of weights), the generalized gradient method treats the voxel values taken from
window halves as two samples which need to be compared in a suitable way. Once it is
done, the value of the corresponding dissimilarity measure is treated as a “gradient” value
at the current sliding window position for a given orientation X, Y or Z.
One may prefer to employ a sophisticated technique of comparing two samples of
voxels such as the voxel classification procedure performed with the help of an appropriate
classifier [3]. In these circumstances the resultant classification accuracy is treated as the
local image gradient magnitude which is varied in the range of 0–100%. Along with recent
classifiers, the sets of voxels may, for example, be compared in a statistical manner using
conventional t-test. This case the resultant t-value is treated as a measure of dissimilarity that
is as the signed local “gradient” value.
It should be noted that despite the fact that t-test also compares mean values of two
voxel samples, it proceeds in a more correct way taking into account the variances of two
distributions. In addition, the t-test has an inherit threshold of significance at |t| = 1.96;
p < 0.05 what is often very problematic to set up in conventional intensity convolutions.
2 Materials
In this study we used three kinds of images containing regions with weak borders which are
difficult to detect by human visual system: synthetic 3D images, CT image of the physical
gelatin phantom and CT images of chest of 40 patients. Image regions did not form coherent
spatial pattern, but rather looked like random textures with difference being the probability
density functions of values inside them.
Highlighting Tumor Borders Using Generalized Gradient 87
2.1 Synthetic Images
For this experiment, we created a synthetic 3D image with size (512 × 512 × 50) voxels.
Inside this volume a parallelepiped was placed with distances along the corresponding
volume margins equal to 128, 128 and 12 voxels. The grey values of the voxels of the inner
and outer regions were drawn from two Pearson distributions with different parameters,
having the same mean value of μ = 200 and standard deviation σ = 20, but different skew‐
ness values. The inner part was filled with values to have the skewness ω
in
to be as close as
possible to 1 taken throughout all the image slices, and voxels from the outer part were filled
with values to have the global skewness ω
out
= 2.
It should be noted that due to the probabilistic technique of values generation the
exact equality of their mean, standard deviation and skewness to the expected ones is
hardly possible.
2.2 Physical Gelatin Phantom
The purpose of creating physical phantom was to obtain CT image of some real object,
consisting of several adjacent parts with low relative contrast (layers). The phantom was
supposed to simulate the commonly encountered problem when objects present on radio‐
logical images have barely visible boundaries.
To create such a phantom, we used a cylindrical container filled with several hori‐
zontal layers of gelatin. Different levels of CT brightness of each layer were obtained
by means of dissolving certain pre-calculated amount of radiocontrast agent Omnipaque
in liquid gelatin before its solidification. To control the amounts of radiocontrast agent
some provisional measurements of Omnipaque solutions’ CT-brightness have been
made (see Fig. 1(a) and (b)).
To the amounts of dissolved Omnipaque solution were chosen to increase pure gelatin
(reference) CT-brightness by 4, 8, 16 and 32 Hounsfield unit (HU) for different layers rela‐
tive to the brightness of the reference layer. The reference layer was located at the most
bottom of the container. The brightest layer was placed next, then the others (see Fig. 1(c)).
Besides, an additional layer of water with Omnipaque solution introduced was
poured to the most top. Thus, one more low-contrast border was made between the upper
gelatin layer and the liquid layer.
2.3 Malignant Lung Tumors
In this study, we used 40 CT images of thorax of patients with lung cancer and the atelec‐
tasis of a portion of the lung as diagnosed by a qualified radiologist and confirmed histo‐
logically. Thirty-three of them were males and remaining seven were females. The age of
patients ranged from 41 to 80 years with the mean value of 61.7 years and standard devia‐
tion of 8.7 years. CT scanning was performed on a multi-slice Volume Zoom Siemens
scanner with the standard clinical kV and mA settings during the one-breath hold. The voxel
size of 9 tomograms was in the range of 0.65–0.74 mm in the axial image plane with the
slice thickness equal to the inter-slice distance of 1.5 mm. The voxel size of 31 remaining
88 V. Kovalev et al.
tomograms was 0.68 mm in the axial image plane with the slice thickness equal to the inter-
slice distance of 5.0 mm. No intravenous contrast agent was administered before the collec‐
tion of scan data what is a significant detail of present study. Typical examples of original
CT image slices are shown in Fig. 2.
Fig. 2. Example slices of typical lung CT images of two patients with atelectasis (ATL) and
malignant tumor (TUM). Patient 1 (left image) suffering from the cancer of middle bronchus with
atelectasis of the right middle lobe of the lung. Patient 2 (right image) with the cancer of right
upper bronchus and atelectasis of the back segment of the upper lung lobe.
Fig. 1. (a) General view of the installation; (b) cups with different amounts of dissolved
Omnipaque solution at the calibration stage; (c) phantom scheme; (d) one slice of the phantom
CT image.
Highlighting Tumor Borders Using Generalized Gradient 89
3Methods
The present study was performed in two main stages. The first, exploratory stage was dedi‐
cated to experimental assessment of intensity differences between the regions of malignant
tumors and atelectasis. In the second stage we examined the abilities of generalized gradient
techniques to highlight borders between the two.
3.1 Exploring the Intensity Differences
The approach followed in this stage was to sub-sample image voxels from two types of lung
regions at random and to evaluate the significance of the intensity differences as a function
of the sample size (i.e., the number of voxels in each voxel subset). In order to ease the
interpretability of the results, the sample sizes were selected so that they correspond to the
number of voxels in square-shaped image slice patches with the side size of 3, 4,…, 10, 15,
20, and 30 voxels that is 9, 16,…, 100, 225, 400, and 900 sample voxels respectively. This
does not mean that the analysis methodology we developing is 2D-oriented, though. In all
the occasions image voxels were sampled from the atelectasis and tumor regions at random.
All statistical and pattern recognition analyses described in this work were performed using
R, a language and environment for statistical computing which is available for free.
The atelectasis and tumor classes were compared by various ways to eliminate possible
bias of one singe method. First, the significance of intensity differences between the two
classes was assessed statistically using a two-tailed unpaired t-test with the significance
level of t-statistics set to p < 0.05. The resultant t-values, which depend on the degree of
freedom (sample size) were converted into z-scores to enable direct comparison of statis‐
tical significance obtained in different experiments as well as to calculate the mean signifi‐
cance scores over all 40 patients correctly. For each patient and each sample size the proce‐
dure consisting of random voxel sub-sampling and performing t-test was replicated 100
times in order to obtain reliable results.
At the second step, the atelectasis and tumor voxel samples (i.e., vectors of voxels sorted
in descending order) were clustered using commonly known Hierarchical Clustering,
Support Vector Machines, and Random Forests methods. For each sample size and each
patient the classifiers were trained on a training sets consisting of 10 atelectasis and 10
tumor samples and tested on the datasets of the same size. Training and test sets were
sampled independently. There was no voxels included in both training and test sets simul‐
taneously. The three classifiers were run on exactly the same data. Each test was replicated
100 times in order to obtain statistically representative estimates of the classification accu‐
racy.
The classification accuracy was corrected for agreement by chance using the classA‐
greement function provided with R. For two classes this particularly means that the minimal
accuracy value is 0 but not 50%. The corrected classification accuracy was used as a
measure of the dissimilarity of two lung regions as well as the basic value for estimating
possible image segmentation accuracy. The total number of performed classification tests
was: 40 patients × 11 sample sizes × 3 methods × 100 replications = 132 000.
90 V. Kovalev et al.
3.2 Detecting Tumor Borders Using Generalized Gradient
The above informal definition of the generalized gradient gives the essence of the method
used in present study. The exact computational procedure is a bit more complicated. A list
of key details which needs to be considered for better understanding and correct implemen‐
tation of the method is given below.
Despite the method may be used for computing generalized gradient maps of 2D
images, it is better suited for 3D because it is supposed to deal with relatively larger samples
of voxels taken from sliding window halves.
It is clear that with no respect to the nature and underlying mechanism of the procedure
used for comparing two voxel sets taken from adjacent window halves, it is highly desir‐
able to have the resultant dissimilarity estimate as precise as possible. In order to achieve
this, a bootstrap multi-step meta-procedure can be employed (see, for example, a good tuto‐
rial [6] written for non-statisticians). In practice it particularly means that at each computa‐
tional step not the whole amount but a fraction of voxels should be sub-sampled in a random
manner from window halves for executing chosen comparison procedure such as t-test. And
this step should be repeated about 100 times.
The final dissimilarity measure is computed as a mean value of corresponding partic‐
ular dissimilarity values that is as the mean t-value computed over the all 100 particular
trials in case the t-test procedure is employed. The same holds true in case the final clus‐
tering accuracy value is calculated based on particular classification steps, etc. The natural
payment for the increased accuracy of assessing the difference by means of bootstrap is the
growth of computational expenses for about two orders. For instance, in case of 3D images
the total number of elementary t-tests which need to be performed resides around 300 with
about 100 tests accomplished for computing gradient components GX, GY and GZ along
each of three orthogonal image axes X, Y and Z.
Fig. 3. Configuration of the gap sliding window.
Once the generalized gradient components GX, GY and GZ are computed using the
procedure of voxel set comparison, the gradient magnitude G
x,y,z
at a particular 3D voxel
position (x, y, z) is calculated as the Euclidean norm of the vector. In general, the sliding
window may have not three orthogonal orientations of voxel sampling like traditional axes
X, Y and Z but some alternative configurations too. In this study we also utilized a bit more
sophisticated configuration of sliding window depicted in Fig. 3. It supposes to use six
directions equally-spaced in 3D. Sampling in each direction is performed using
Highlighting Tumor Borders Using Generalized Gradient 91
corresponding spherical sub-windows with radius R. Moreover, the sub-windows are moved
apart from the central voxel at the distance d. This was done to address the problem of
smooth and wide object borders. Finally, the resulting generalized gradient value at a partic‐
ular 3D voxel position (x, y, z) is calculated from the particular values in each direction
G
i
,i∈{1,…,6
}
as
G
x,y,z
=(
∑
6
i
=1
G
2
i
)
1∕
2
.
4Results
4.1 The Intensity Differences Discovered
Results of statistical assessment of the significance of intensity differences between the
atelectasis and tumor regions of lung CT scans of 40 patients are reported in Fig. 4. As
it can be seen from the figure, the fraction of significance different voxel samples and
the mean significance scores varied considerably depending on the patient. For instance,
for one patient the percentage of significance different samples exceeds notable 60%
already on 9 voxels and achieves 100% with the sample size as little as 36 voxels (see
the left panel of Fig. 4) while in other it starts close to zero with 9 voxels and finishes
at about 10% only. Similarly, for some patients the mean z-score achieves the signifi‐
cance threshold z > 1.96 which is equivalent to p < 0.05 on the sample size of 9–25
voxels (see the right panel of Fig. 4) while for others these values remain insignificantly
low even on reasonably large samples consisting of 400–900 voxels.
Fig. 4. Significance of the intensity differences of lung atelectasis and tumor voxel samples for
40 patients (curves) as a function of the voxel sample size. Left panel: percentage of voxel samples
for which the intensity difference is statistically significant at p < 0.05. Right panel: the mean
value of significance score z. In both occasions image voxels were sampled from atelectasis and
tumor regions at random and each measurement is replicated 100 times.
On the contrary, the voxel sample classification results demonstrate much more
consistent behavior (see Fig. 5). As it can be revealed from the figure, a very useful
property of the classification approach for separating the atelectasis and tumor regions
is that the results are converged to 90–100% of the classification accuracy for relatively
large samples in each patient.
As for the comparative efficiency of the three classification methods, it is easy to see
from Fig. 5 that the Hierarchical Clustering algorithm outperforms both SVM and
Random Forests for each voxel sample size. Moreover, in case of Hierarchical Clus‐
tering, the classification accuracy corrected for the agreement by chance starts from the
92 V. Kovalev et al.
value above 50% almost for each patient and achieves 90% on the sample size of 225
voxels for all 40 patients except for 2 outliers. The mean and standard deviation values
of the classification accuracy computed over 40 patients (see the bottom right plot of
Fig. 5) make the superiority of Hierarchical Clustering method evident and renders other
two as almost identical in the voxel sample classification task. Considering that the one
possible segmentation technique could be based on a direct voxel sample classification
using sliding window of suitable size, the mean accuracy threshold should be set to a
reasonably high value, say 95%. If so, the minimal sample size should be set to approx‐
imately 100–200 voxels. This corresponds to the window size of about 12 × 12 voxels
(i.e., the half window size is 4.1 mm) for 2D and less than 6 × 6 × 6 voxels (2.0 mm)
for 3D case.
4.2 Detected Tumor Borders
The results of application of generalized gradient to synthetic images are depicted on
Fig. 6. This experiment shows the capability of the generalized gradient (GG) maps
calculated with different presets to detect weak borders, and the results are as they were
expected. Figure 6(c) and (d) show the clear border between inner and outer regions.
We used the SVM classification accuracy as the difference measure improved by the
bootstrap procedure and the gap sliding window. No a priory information about border
orientation, width, smoothness or values distribution was used.
Figure 6(b) depicts the GG map calculated over gelatin phantom using conventional
t-test to estimate the dissimilarity measure between values sampled from the gap window
halves. Though this map calculation is much faster than of the previous ones, in this
Fig. 5. Dependence of the classification accuracy on sample size of lung atelectasis and tumor
voxels for 40 patients (curves) when using Hierarchical Clustering (top left plot), Support Vector
Machines (top right plot), and Random Forests (bottom left plot) clustering methods. Each test
was replicated 100 times for the reliability of results. The mean and standard deviation accuracy
computed over 40 patients is given on the bottom right panel.
Highlighting Tumor Borders Using Generalized Gradient 93
particular case it gives no positive outcome, because t-test does not react on the differ‐
ence of skewness and higher orders moments. However, further we will show that it also
provides useful results retaining the same relative advance in speed when used for
processing of real images.
Fig. 7. Gelatin phantom: (a), (d) – GG maps calculated using gap window with R = 4, d = 2 and
R = 5, d = 3 respectively, t-test of voxel samples used for dissimilarity measure estimation; (b),
(e) – GG maps calculated using spherical window with r = 5 and r = 8 respectively, t-test of voxel
samples used for dissimilarity measure estimation; (c), (f) – GG maps calculated using spherical
window, dissimilarity measure is the difference of mean values sampled from window halves.
The resultant GG maps of the image in Fig. 1(d) are depicted in Fig. 7. Left column
contains maps calculated using t-test to estimate dissimilarity measure and gap sliding
Fig. 6. (a) Original synthetic image; (b) GG map using t-test, R = 4, d = 2; (c) GG map using
SVM, gap window’s R = 3, d = 1; (d) GG map using SVM, R = 4, d = 2.
94 V. Kovalev et al.
window, middle column – also t-test and spherical sliding window, right column –
spherical sliding window and dissimilarity measure is the difference of mean values
sampled from window halves. Sizes of all sliding windows along first and second rows
were chosen to have almost the same number of voxels. Unlike the previous synthetic
images, the gelatin phantom layers have definite differences of mean HU values, this is
why it is fairly easy problem to detect weak borders using different presets of the method.
Nevertheless, this figure may help to choose the preferable method’s parameters
depending on the desired result. The middle layers of gelatin seem to be interdiffused
and there were no detectable borders.
Quantitative assessment of the utility of generalized gradient maps in highlighting
lung tumor borders was performed separately for the first subgroup of 31 native CT
images with the slice thickness of 5.0 mm and remaining 9 images of the second
subgroup with the slice thickness of about 1.5 mm. Typical examples of original CT
image ROIs and corresponding gradient map regions are presented in Fig. 8.
Fig. 8. Example ROIs of the original CT images of lungs (left column) and corresponding
generalized gradient maps (right column). The first row represent case where the gradient map is
definitely useful for detecting tumor border whereas the second and the third rows illustrate cases
where the utility of maps is unclear and useless respectively.
As a result of the experiment, on the first subgroup of patients it was revealed that
the generalized gradient maps were definitely useful for detecting tumor border in 17
patients (54.8%) whereas in 9 other cases (29.0%) they did not provide any help for
solving the problem of separation the malignant tumor from adjacent atelectasis. The
efficacy of maps in the rest 5 cases (16.1%) was found to be unclear. The results of the
similar examination of CT scans with reasonably thin slices of about 1.5 mm suggest
that it appears to be unlikely the slice thickness is an important parameter for the method.
In particular, the distribution of cases between the “yes”, “no”, and “unclear” categories
was 5 (55.6%), 3 (33.3%), and 1 (11.1%) respectively. This is well comparable with
corresponding results obtained for the first subgroup.
Highlighting Tumor Borders Using Generalized Gradient 95
5Conclusions
In this work we have documented results of statistical assessment of CT image intensity
differences between the lung atelectasis and malignant tumors. The significance scores
and classification accuracy results reported here are based on the advanced statistical
and pattern recognition methods. Our results suggest that it is unlikely that the use of
statistical significance scores for separating lung atelectasis and tumor regions would
produce good quality discrimination for all patients. However, the recent clustering
algorithms demonstrate some encouraging classification accuracy on the CT intensity
samples consisting of few hundred voxels. The Hierarchical Clustering method is found
to be better suited for CT voxels classification task comparing to SVM and Random
Forests classifiers. This is in agreement with other studies where classes overlap in
feature space substantially. The voxel sample classification accuracy potentially allows
to reliably discriminate atelectasis and tumor regions using relatively small sliding
window of 12 × 12 voxels (i.e., the half window size is 4.1 mm) in 2D and no more than
6 × 6 × 6 voxels (2.0 mm) in 3D case.
Also we have introduced the basic concept of so-called generalized gradient and
demonstrated its abilities and key details on synthetic images, 3D CT images of physical
phantom as well as CT scans of lung of 40 patients with clinically confirmed diagnosis
of lung cancer.
References
1. Petrou, M., Kovalev, V., Reichenbach, J.: Three-dimensional nonlinear invisible boundary
detection. IEEE Trans. Image Proc. 15(10), 3020–3032 (2006)
2. Kovalev, V., Petrou, M.: The classification gradient. In: Proceedings of 18th International
Conference on Pattern Recognition, ICPR-2006, vol. 3, pp. 830–833, Hong Kong (2006)
3. Petrou, M., Kovalev, V., Reichenbach, J.: High Order Statistics for Tissue Segmentation.
Handbook of Medical Image Processing and Analysis, 2nd edn, pp. 245–257. Academic Press,
San Diego (2009)
4. Wedding, M., Gylys, B.: Medical Terminology Systems: A Body Systems Approach, 5th edn,
p. 559. FA Davis Company, Philadelphia (2004)
5. Mira, J., Fullerton, G., Ezekiel, J., Potter, J.: Evaluation of computed tomography numbers for
treatment planning of lung cancer. Int. J. Radiat. Oncol. Biol. Phys. 8(9), 1625–1628 (1982)
6. Wehrens, R., Putter, H., Buydens, L.: The bootstrap: a tutorial. Chemometr. Intell. Lab. Syst.
54, 35–52 (2000)
96 V. Kovalev et al.
