Content uploaded by Vassili Kovalev

Author content

All content in this area was uploaded by Vassili Kovalev on Apr 07, 2018

Content may be subject to copyright.

Highlighting Tumor Borders Using Generalized Gradient

Vassili Kovalev

1(✉)

, Eduard Snezhko

1

, Siarhei Kharuzhyk

2

, and Vitali Liauchuk

1

1Biomedical Image Analysis Group and Mathematical Cybernetics Department,

United Institute of Informatics Problems, Surganova Street, 6, 220012 Minsk, Belarus

vassili.kovalev@gmail.com

2N.N. Alexandrov National Cancer Center of Belarus, 223043 Minsk, Lesnoy District, Belarus

Abstract. This paper presents a generalized approach for computing image

gradient. It is predominantly aimed at detecting unclear and in certain circumstances

even completely invisible borders in large 2D and 3D texture images. The method

exploits the conventional approach of sliding window. Once two pixel/voxel sets are

sub-sampled from orthogonal window halves, they are compared by a suitable tech‐

nique (e.g., statistical t-test, SVM classier, comparison of parameters of two distribu‐

tions) and the resultant measure of difference (e.g., t-value, the classification accu‐

racy, skewness difference of two distributions etc.) is treated as the gradient magni‐

tude. The bootstrap procedure is employed for increasing the accuracy of difference

assessment of two pixel/voxel sets.

Keywords: Lung tumor · CT · Edge detection · Generalized gradient

1 Introduction

In many occasions there is a strong need for detection of borders of objects which look like

patterns of random textures. Such borders could be hardly detected by human visual system

when textures differ by their high-order statistics only [1, 2]. Recently, some advanced

methods of detecting hardly visible borders between the random image textures have been

suggested [1, 3]. Moreover, it was experimentally proven that these methods capitalizing on

so-called “generalized gradient” are able to highlight the border which is completely invis‐

ible for human eye [4, 5].

This paper addresses the problem of detecting borders of malignant tumors in native lung

CT images under conditions of presence of atelectasis. The atelectasis term denotes the

collapse of all or part of a lung due to bronchial plugging or the chest cavity being opened

to atmospheric pressure. This can happen when the vacuum between the lung and chest

wall is broken, allowing the lung to collapse within the chest (e.g., pneumothorax), when the

lung is compressed by masses in the chest, or when an airway is blocked, leading to slow

absorption of the distal air into the blood without replenishment. In this work we were

dealing with the bronchial compression caused by lung cancer tumors, the most common

cause of the atelectasis.

Computed tomography (CT) is the primary modality for imaging lung cancer patients.

However, the problem is that on CT scans the lung regions with the atelectasis and malig‐

nant tumors have quite similar attenuation values. Therefore the visual discrimination and

© Springer International Publishing AG 2017

V.V. Krasnoproshin and S.V. Ablameyko (Eds.): PRIP 2016, CCIS 673, pp. 86–96, 2017.

DOI: 10.1007/978-3-319-54220-1_9

separation of the atelectasis and tumor is hardly possible. Yet, accurate tumor segmentation

is strongly necessary by the following two reasons. First, the correct tumor localization,

segmentation, and precise measurement of tumor diameter play a crucial role in therapy

planning and choosing suitable surgery technique. Second, if the radiation therapy is

prescribed, an exact separation of tumor border is required for precise targeting and delivery

of the ionizing radiation dose accurately to the tumor but not to the surrounding tissues.

Thus, the purpose of this particular paper is to present results of an experimental study

of the ability of the generalized gradient method to highlight hardly visible borders of

objects. The study was conducted using three different groups of images. They were

comprised by 3D synthetic images and specially-designed physical gelatin phantom made

by authors and scanned using Siemens Somatom Definition AS scanner. Finally, the utility

of the method was examined on the problem of borders detection between malignant lung

tumors and the atelectasis regions based on 3D CT images of 40 lung cancer patients.

The first version of the generalized gradient method was introduced in [2] as so-called

classification gradient and slightly improved afterwards. The classification gradient method

makes use conventional technique of calculating image gradient at each pixel position by

means of comparing pixel/voxel values taken from orthogonal halves of appropriately sized

sliding window. However, apart from the traditional approaches where the gradient magni‐

tude is computed simply as the intensity difference (estimated by convolution with one or

other matrix of weights), the generalized gradient method treats the voxel values taken from

window halves as two samples which need to be compared in a suitable way. Once it is

done, the value of the corresponding dissimilarity measure is treated as a “gradient” value

at the current sliding window position for a given orientation X, Y or Z.

One may prefer to employ a sophisticated technique of comparing two samples of

voxels such as the voxel classification procedure performed with the help of an appropriate

classifier [3]. In these circumstances the resultant classification accuracy is treated as the

local image gradient magnitude which is varied in the range of 0–100%. Along with recent

classifiers, the sets of voxels may, for example, be compared in a statistical manner using

conventional t-test. This case the resultant t-value is treated as a measure of dissimilarity that

is as the signed local “gradient” value.

It should be noted that despite the fact that t-test also compares mean values of two

voxel samples, it proceeds in a more correct way taking into account the variances of two

distributions. In addition, the t-test has an inherit threshold of significance at |t| = 1.96;

p < 0.05 what is often very problematic to set up in conventional intensity convolutions.

2 Materials

In this study we used three kinds of images containing regions with weak borders which are

difficult to detect by human visual system: synthetic 3D images, CT image of the physical

gelatin phantom and CT images of chest of 40 patients. Image regions did not form coherent

spatial pattern, but rather looked like random textures with difference being the probability

density functions of values inside them.

Highlighting Tumor Borders Using Generalized Gradient 87

2.1 Synthetic Images

For this experiment, we created a synthetic 3D image with size (512 × 512 × 50) voxels.

Inside this volume a parallelepiped was placed with distances along the corresponding

volume margins equal to 128, 128 and 12 voxels. The grey values of the voxels of the inner

and outer regions were drawn from two Pearson distributions with different parameters,

having the same mean value of μ = 200 and standard deviation σ = 20, but different skew‐

ness values. The inner part was filled with values to have the skewness ω

in

to be as close as

possible to 1 taken throughout all the image slices, and voxels from the outer part were filled

with values to have the global skewness ω

out

= 2.

It should be noted that due to the probabilistic technique of values generation the

exact equality of their mean, standard deviation and skewness to the expected ones is

hardly possible.

2.2 Physical Gelatin Phantom

The purpose of creating physical phantom was to obtain CT image of some real object,

consisting of several adjacent parts with low relative contrast (layers). The phantom was

supposed to simulate the commonly encountered problem when objects present on radio‐

logical images have barely visible boundaries.

To create such a phantom, we used a cylindrical container ﬁlled with several hori‐

zontal layers of gelatin. Diﬀerent levels of CT brightness of each layer were obtained

by means of dissolving certain pre-calculated amount of radiocontrast agent Omnipaque

in liquid gelatin before its solidiﬁcation. To control the amounts of radiocontrast agent

some provisional measurements of Omnipaque solutions’ CT-brightness have been

made (see Fig. 1(a) and (b)).

To the amounts of dissolved Omnipaque solution were chosen to increase pure gelatin

(reference) CT-brightness by 4, 8, 16 and 32 Hounsfield unit (HU) for different layers rela‐

tive to the brightness of the reference layer. The reference layer was located at the most

bottom of the container. The brightest layer was placed next, then the others (see Fig. 1(c)).

Besides, an additional layer of water with Omnipaque solution introduced was

poured to the most top. Thus, one more low-contrast border was made between the upper

gelatin layer and the liquid layer.

2.3 Malignant Lung Tumors

In this study, we used 40 CT images of thorax of patients with lung cancer and the atelec‐

tasis of a portion of the lung as diagnosed by a qualified radiologist and confirmed histo‐

logically. Thirty-three of them were males and remaining seven were females. The age of

patients ranged from 41 to 80 years with the mean value of 61.7 years and standard devia‐

tion of 8.7 years. CT scanning was performed on a multi-slice Volume Zoom Siemens

scanner with the standard clinical kV and mA settings during the one-breath hold. The voxel

size of 9 tomograms was in the range of 0.65–0.74 mm in the axial image plane with the

slice thickness equal to the inter-slice distance of 1.5 mm. The voxel size of 31 remaining

88 V. Kovalev et al.

tomograms was 0.68 mm in the axial image plane with the slice thickness equal to the inter-

slice distance of 5.0 mm. No intravenous contrast agent was administered before the collec‐

tion of scan data what is a significant detail of present study. Typical examples of original

CT image slices are shown in Fig. 2.

Fig. 2. Example slices of typical lung CT images of two patients with atelectasis (ATL) and

malignant tumor (TUM). Patient 1 (left image) suﬀering from the cancer of middle bronchus with

atelectasis of the right middle lobe of the lung. Patient 2 (right image) with the cancer of right

upper bronchus and atelectasis of the back segment of the upper lung lobe.

Fig. 1. (a) General view of the installation; (b) cups with diﬀerent amounts of dissolved

Omnipaque solution at the calibration stage; (c) phantom scheme; (d) one slice of the phantom

CT image.

Highlighting Tumor Borders Using Generalized Gradient 89

3Methods

The present study was performed in two main stages. The first, exploratory stage was dedi‐

cated to experimental assessment of intensity differences between the regions of malignant

tumors and atelectasis. In the second stage we examined the abilities of generalized gradient

techniques to highlight borders between the two.

3.1 Exploring the Intensity Diﬀerences

The approach followed in this stage was to sub-sample image voxels from two types of lung

regions at random and to evaluate the significance of the intensity differences as a function

of the sample size (i.e., the number of voxels in each voxel subset). In order to ease the

interpretability of the results, the sample sizes were selected so that they correspond to the

number of voxels in square-shaped image slice patches with the side size of 3, 4,…, 10, 15,

20, and 30 voxels that is 9, 16,…, 100, 225, 400, and 900 sample voxels respectively. This

does not mean that the analysis methodology we developing is 2D-oriented, though. In all

the occasions image voxels were sampled from the atelectasis and tumor regions at random.

All statistical and pattern recognition analyses described in this work were performed using

R, a language and environment for statistical computing which is available for free.

The atelectasis and tumor classes were compared by various ways to eliminate possible

bias of one singe method. First, the significance of intensity differences between the two

classes was assessed statistically using a two-tailed unpaired t-test with the significance

level of t-statistics set to p < 0.05. The resultant t-values, which depend on the degree of

freedom (sample size) were converted into z-scores to enable direct comparison of statis‐

tical significance obtained in different experiments as well as to calculate the mean signifi‐

cance scores over all 40 patients correctly. For each patient and each sample size the proce‐

dure consisting of random voxel sub-sampling and performing t-test was replicated 100

times in order to obtain reliable results.

At the second step, the atelectasis and tumor voxel samples (i.e., vectors of voxels sorted

in descending order) were clustered using commonly known Hierarchical Clustering,

Support Vector Machines, and Random Forests methods. For each sample size and each

patient the classifiers were trained on a training sets consisting of 10 atelectasis and 10

tumor samples and tested on the datasets of the same size. Training and test sets were

sampled independently. There was no voxels included in both training and test sets simul‐

taneously. The three classifiers were run on exactly the same data. Each test was replicated

100 times in order to obtain statistically representative estimates of the classification accu‐

racy.

The classification accuracy was corrected for agreement by chance using the classA‐

greement function provided with R. For two classes this particularly means that the minimal

accuracy value is 0 but not 50%. The corrected classification accuracy was used as a

measure of the dissimilarity of two lung regions as well as the basic value for estimating

possible image segmentation accuracy. The total number of performed classification tests

was: 40 patients × 11 sample sizes × 3 methods × 100 replications = 132 000.

90 V. Kovalev et al.

3.2 Detecting Tumor Borders Using Generalized Gradient

The above informal definition of the generalized gradient gives the essence of the method

used in present study. The exact computational procedure is a bit more complicated. A list

of key details which needs to be considered for better understanding and correct implemen‐

tation of the method is given below.

Despite the method may be used for computing generalized gradient maps of 2D

images, it is better suited for 3D because it is supposed to deal with relatively larger samples

of voxels taken from sliding window halves.

It is clear that with no respect to the nature and underlying mechanism of the procedure

used for comparing two voxel sets taken from adjacent window halves, it is highly desir‐

able to have the resultant dissimilarity estimate as precise as possible. In order to achieve

this, a bootstrap multi-step meta-procedure can be employed (see, for example, a good tuto‐

rial [6] written for non-statisticians). In practice it particularly means that at each computa‐

tional step not the whole amount but a fraction of voxels should be sub-sampled in a random

manner from window halves for executing chosen comparison procedure such as t-test. And

this step should be repeated about 100 times.

The final dissimilarity measure is computed as a mean value of corresponding partic‐

ular dissimilarity values that is as the mean t-value computed over the all 100 particular

trials in case the t-test procedure is employed. The same holds true in case the final clus‐

tering accuracy value is calculated based on particular classification steps, etc. The natural

payment for the increased accuracy of assessing the difference by means of bootstrap is the

growth of computational expenses for about two orders. For instance, in case of 3D images

the total number of elementary t-tests which need to be performed resides around 300 with

about 100 tests accomplished for computing gradient components GX, GY and GZ along

each of three orthogonal image axes X, Y and Z.

Fig. 3. Conﬁguration of the gap sliding window.

Once the generalized gradient components GX, GY and GZ are computed using the

procedure of voxel set comparison, the gradient magnitude G

x,y,z

at a particular 3D voxel

position (x, y, z) is calculated as the Euclidean norm of the vector. In general, the sliding

window may have not three orthogonal orientations of voxel sampling like traditional axes

X, Y and Z but some alternative configurations too. In this study we also utilized a bit more

sophisticated configuration of sliding window depicted in Fig. 3. It supposes to use six

directions equally-spaced in 3D. Sampling in each direction is performed using

Highlighting Tumor Borders Using Generalized Gradient 91

corresponding spherical sub-windows with radius R. Moreover, the sub-windows are moved

apart from the central voxel at the distance d. This was done to address the problem of

smooth and wide object borders. Finally, the resulting generalized gradient value at a partic‐

ular 3D voxel position (x, y, z) is calculated from the particular values in each direction

G

i

,i∈{1,…,6

}

as

G

x,y,z

=(

∑

6

i

=1

G

2

i

)

1∕

2

.

4Results

4.1 The Intensity Diﬀerences Discovered

Results of statistical assessment of the signiﬁcance of intensity diﬀerences between the

atelectasis and tumor regions of lung CT scans of 40 patients are reported in Fig. 4. As

it can be seen from the ﬁgure, the fraction of signiﬁcance diﬀerent voxel samples and

the mean signiﬁcance scores varied considerably depending on the patient. For instance,

for one patient the percentage of signiﬁcance diﬀerent samples exceeds notable 60%

already on 9 voxels and achieves 100% with the sample size as little as 36 voxels (see

the left panel of Fig. 4) while in other it starts close to zero with 9 voxels and ﬁnishes

at about 10% only. Similarly, for some patients the mean z-score achieves the signiﬁ‐

cance threshold z > 1.96 which is equivalent to p < 0.05 on the sample size of 9–25

voxels (see the right panel of Fig. 4) while for others these values remain insigniﬁcantly

low even on reasonably large samples consisting of 400–900 voxels.

Fig. 4. Signiﬁcance of the intensity diﬀerences of lung atelectasis and tumor voxel samples for

40 patients (curves) as a function of the voxel sample size. Left panel: percentage of voxel samples

for which the intensity diﬀerence is statistically signiﬁcant at p < 0.05. Right panel: the mean

value of signiﬁcance score z. In both occasions image voxels were sampled from atelectasis and

tumor regions at random and each measurement is replicated 100 times.

On the contrary, the voxel sample classiﬁcation results demonstrate much more

consistent behavior (see Fig. 5). As it can be revealed from the ﬁgure, a very useful

property of the classiﬁcation approach for separating the atelectasis and tumor regions

is that the results are converged to 90–100% of the classiﬁcation accuracy for relatively

large samples in each patient.

As for the comparative eﬃciency of the three classiﬁcation methods, it is easy to see

from Fig. 5 that the Hierarchical Clustering algorithm outperforms both SVM and

Random Forests for each voxel sample size. Moreover, in case of Hierarchical Clus‐

tering, the classiﬁcation accuracy corrected for the agreement by chance starts from the

92 V. Kovalev et al.

value above 50% almost for each patient and achieves 90% on the sample size of 225

voxels for all 40 patients except for 2 outliers. The mean and standard deviation values

of the classiﬁcation accuracy computed over 40 patients (see the bottom right plot of

Fig. 5) make the superiority of Hierarchical Clustering method evident and renders other

two as almost identical in the voxel sample classiﬁcation task. Considering that the one

possible segmentation technique could be based on a direct voxel sample classiﬁcation

using sliding window of suitable size, the mean accuracy threshold should be set to a

reasonably high value, say 95%. If so, the minimal sample size should be set to approx‐

imately 100–200 voxels. This corresponds to the window size of about 12 × 12 voxels

(i.e., the half window size is 4.1 mm) for 2D and less than 6 × 6 × 6 voxels (2.0 mm)

for 3D case.

4.2 Detected Tumor Borders

The results of application of generalized gradient to synthetic images are depicted on

Fig. 6. This experiment shows the capability of the generalized gradient (GG) maps

calculated with diﬀerent presets to detect weak borders, and the results are as they were

expected. Figure 6(c) and (d) show the clear border between inner and outer regions.

We used the SVM classiﬁcation accuracy as the diﬀerence measure improved by the

bootstrap procedure and the gap sliding window. No a priory information about border

orientation, width, smoothness or values distribution was used.

Figure 6(b) depicts the GG map calculated over gelatin phantom using conventional

t-test to estimate the dissimilarity measure between values sampled from the gap window

halves. Though this map calculation is much faster than of the previous ones, in this

Fig. 5. Dependence of the classiﬁcation accuracy on sample size of lung atelectasis and tumor

voxels for 40 patients (curves) when using Hierarchical Clustering (top left plot), Support Vector

Machines (top right plot), and Random Forests (bottom left plot) clustering methods. Each test

was replicated 100 times for the reliability of results. The mean and standard deviation accuracy

computed over 40 patients is given on the bottom right panel.

Highlighting Tumor Borders Using Generalized Gradient 93

particular case it gives no positive outcome, because t-test does not react on the diﬀer‐

ence of skewness and higher orders moments. However, further we will show that it also

provides useful results retaining the same relative advance in speed when used for

processing of real images.

Fig. 7. Gelatin phantom: (a), (d) – GG maps calculated using gap window with R = 4, d = 2 and

R = 5, d = 3 respectively, t-test of voxel samples used for dissimilarity measure estimation; (b),

(e) – GG maps calculated using spherical window with r = 5 and r = 8 respectively, t-test of voxel

samples used for dissimilarity measure estimation; (c), (f) – GG maps calculated using spherical

window, dissimilarity measure is the diﬀerence of mean values sampled from window halves.

The resultant GG maps of the image in Fig. 1(d) are depicted in Fig. 7. Left column

contains maps calculated using t-test to estimate dissimilarity measure and gap sliding

Fig. 6. (a) Original synthetic image; (b) GG map using t-test, R = 4, d = 2; (c) GG map using

SVM, gap window’s R = 3, d = 1; (d) GG map using SVM, R = 4, d = 2.

94 V. Kovalev et al.

window, middle column – also t-test and spherical sliding window, right column –

spherical sliding window and dissimilarity measure is the diﬀerence of mean values

sampled from window halves. Sizes of all sliding windows along ﬁrst and second rows

were chosen to have almost the same number of voxels. Unlike the previous synthetic

images, the gelatin phantom layers have deﬁnite diﬀerences of mean HU values, this is

why it is fairly easy problem to detect weak borders using diﬀerent presets of the method.

Nevertheless, this ﬁgure may help to choose the preferable method’s parameters

depending on the desired result. The middle layers of gelatin seem to be interdiﬀused

and there were no detectable borders.

Quantitative assessment of the utility of generalized gradient maps in highlighting

lung tumor borders was performed separately for the ﬁrst subgroup of 31 native CT

images with the slice thickness of 5.0 mm and remaining 9 images of the second

subgroup with the slice thickness of about 1.5 mm. Typical examples of original CT

image ROIs and corresponding gradient map regions are presented in Fig. 8.

Fig. 8. Example ROIs of the original CT images of lungs (left column) and corresponding

generalized gradient maps (right column). The ﬁrst row represent case where the gradient map is

deﬁnitely useful for detecting tumor border whereas the second and the third rows illustrate cases

where the utility of maps is unclear and useless respectively.

As a result of the experiment, on the ﬁrst subgroup of patients it was revealed that

the generalized gradient maps were deﬁnitely useful for detecting tumor border in 17

patients (54.8%) whereas in 9 other cases (29.0%) they did not provide any help for

solving the problem of separation the malignant tumor from adjacent atelectasis. The

eﬃcacy of maps in the rest 5 cases (16.1%) was found to be unclear. The results of the

similar examination of CT scans with reasonably thin slices of about 1.5 mm suggest

that it appears to be unlikely the slice thickness is an important parameter for the method.

In particular, the distribution of cases between the “yes”, “no”, and “unclear” categories

was 5 (55.6%), 3 (33.3%), and 1 (11.1%) respectively. This is well comparable with

corresponding results obtained for the ﬁrst subgroup.

Highlighting Tumor Borders Using Generalized Gradient 95

5Conclusions

In this work we have documented results of statistical assessment of CT image intensity

diﬀerences between the lung atelectasis and malignant tumors. The signiﬁcance scores

and classiﬁcation accuracy results reported here are based on the advanced statistical

and pattern recognition methods. Our results suggest that it is unlikely that the use of

statistical signiﬁcance scores for separating lung atelectasis and tumor regions would

produce good quality discrimination for all patients. However, the recent clustering

algorithms demonstrate some encouraging classiﬁcation accuracy on the CT intensity

samples consisting of few hundred voxels. The Hierarchical Clustering method is found

to be better suited for CT voxels classiﬁcation task comparing to SVM and Random

Forests classiﬁers. This is in agreement with other studies where classes overlap in

feature space substantially. The voxel sample classiﬁcation accuracy potentially allows

to reliably discriminate atelectasis and tumor regions using relatively small sliding

window of 12 × 12 voxels (i.e., the half window size is 4.1 mm) in 2D and no more than

6 × 6 × 6 voxels (2.0 mm) in 3D case.

Also we have introduced the basic concept of so-called generalized gradient and

demonstrated its abilities and key details on synthetic images, 3D CT images of physical

phantom as well as CT scans of lung of 40 patients with clinically conﬁrmed diagnosis

of lung cancer.

References

1. Petrou, M., Kovalev, V., Reichenbach, J.: Three-dimensional nonlinear invisible boundary

detection. IEEE Trans. Image Proc. 15(10), 3020–3032 (2006)

2. Kovalev, V., Petrou, M.: The classiﬁcation gradient. In: Proceedings of 18th International

Conference on Pattern Recognition, ICPR-2006, vol. 3, pp. 830–833, Hong Kong (2006)

3. Petrou, M., Kovalev, V., Reichenbach, J.: High Order Statistics for Tissue Segmentation.

Handbook of Medical Image Processing and Analysis, 2nd edn, pp. 245–257. Academic Press,

San Diego (2009)

4. Wedding, M., Gylys, B.: Medical Terminology Systems: A Body Systems Approach, 5th edn,

p. 559. FA Davis Company, Philadelphia (2004)

5. Mira, J., Fullerton, G., Ezekiel, J., Potter, J.: Evaluation of computed tomography numbers for

treatment planning of lung cancer. Int. J. Radiat. Oncol. Biol. Phys. 8(9), 1625–1628 (1982)

6. Wehrens, R., Putter, H., Buydens, L.: The bootstrap: a tutorial. Chemometr. Intell. Lab. Syst.

54, 35–52 (2000)

96 V. Kovalev et al.