# 3D Assessment of Lymph Nodes vs. RECIST 1.1

**Abstract**

In today's clinical practice, the size of lymph nodes is assessed by measuring the long and the short axis in the axial plane. This study compares this approach with three-dimensional (3D) assessment.
For a representative set of 49 lymph nodes, the axes in the axial plane have been measured and a 3D model has been created manually. Based on the 3D model, the real axial long and short axis as well as the three 3D axes and the volume have been computed and compared to the measured axial axes.
The inter-observer variability is around 10% for all measured lengths and almost 16% for the computed volume. The average relative error of the measured long (short) axial axis is 9.73% (24.57%) to the computed axial axis and 25.05% (19.97%) to the computed 3D axis, respectively. The product of the axial long axis and the square of the axial short axis provides best correlation to the volume.
This study confirms Response Evaluation Criteria In Solid Tumours 1.1 that measuring the short axis is more robust than measuring the long axis because of less impact of the node's spatial orientation. Nonetheless it is shown that considering both axes is a better prognostic factor for the volume than measuring the short axis only.

# Figures

Technical Report

3D Assessment of Lymph Nodes vs.

RECIST 1.1

Sebastian Steger, Dipl.-Ing., Fabio Franco, MD, Nicola Sverzellati, MD, Gianfranco Chiari, MD,

Ramon Colomer, MD

Rationale and Objectives: In today’s clinical practice, the size of lymph nodes is assessed by measuring the long and the short axis in the

axial plane. This study compares this approach with three-dimensional (3D) assessment.

Materials and Methods: For a representative set of 49 lymph nodes, the axes in the axial plane have been measured and a 3D model has

been created manually. Based on the 3D model, the real axial long and short axis as well as the three 3D axes and the volume have been

computed and compared to the measured axial axes.

Results: The inter-observer variability is around 10% for all measured lengths and almost 16% for the computed volume. The average

relative error of the measured long (short) axial axis is 9.73% (24.57%) to the computed axial axis and 25.05% (19.97%) to the computed

3D axis, respectively. The product of the axial long axis and the square of the axial short axis provides best correlation to the volume.

Conclusion: This study conﬁrms Response Evaluation Criteria In Solid Tumours 1.1 that measuring the short axis is more robust than

measuring the long axis because of less impact of the node’s spatial orientation. Nonetheless it is shown that considering both axes is

a better prognostic factor for the volume than measuring the short axis only.

Key Words: Lymph node staging; RECIST; volumetry.

ªAUR, 2011

T

he presence and location of lymph node metastases (N

descriptor) is an important component of radiologic

staging in patients with cancer of different types.

Malignant lymph nodes are common target and nontarget

lesions, whose sizes have to be assessed for staging and disease

progression moni toring.

For example in oral cancer, accurate staging is the most

impor tant f actor that guides therapeutic decision making.

Staging methods inclu de radiological assessment that usually

entails imaging of the neck with computed tomography

(CT) or magnetic resonance imaging, or both. Lymphatic

metastases will develop in 20%–30% of patients with early

oral cancers and imply decreased survival (1).

The size can be measured using unidimensional, bidimen-

sional, and volumetric techniques (2). For liver malignomas

volumetric analysis is superior (3). However, in current clinical

practice the size is assessed in CT scans by measuring the axis in

the axial plane according to the Response Evaluation Criteria

In Solid Tumours (RECIST) criteria (4).

Besides general limitations of planar assessment of three-

dimensional (3D) structures, the limitation to the axial plane

for assessing arbitrarily spatial oriented structures reduces

reliability. This issue was addressed by the revised RECIST

criteria (5,6), by considering the short axis only.

In this study, we further explored the degree of inter-observer

variability in the assessment of the lymph nodes’ diameters and

tested which diameters in the axial plane better correlate with

volumetric assessment.

MATERIALS AND METHODS

An experienced radiologist (I.R. with 15 years of experience)

identiﬁed a representative set of 49 lymph nodes in 13 head

and neck CT scans of oral cancer patients (5 male, 8 female,

age range 36–93 years). The set contains benign as well as path-

ological lymph nodes, their short axial axis ranging between 4.2

and 24.3 mm. To avoid any kind of selection bias, the study

subjects were consecutive.

Five different radiologists (G.C., F.F., G.G., I.S. and J.G., with

15, 3, 3, 3, and 6 years in oncologic imaging, respectively) from

two different hospitals evaluated those lymph nodes by

measuring their long and short axis in the axial plane of the slice

in which those measurements appeared to be maximal in the

whole node. Then the boundaries of each lymph node in

each slice the lymph node is visible in were manually delineated

using an electronic free-hand region of interest, enabling the

creation of an accurate 3D model of the node. The time for

delineating was recorded for each node.

Acad Radiol 2011; -:1–4

From the Competence Center Medical Imaging, Fraunhofer IGD,

Fraunhoferstrasse 5, 64283 Darmstadt, Germany (S.S.); Department of

Clinical Sciences, Section of Radiology, Parma Hospital, Parma, Italy (F.F.,

N.S., G.C.); Centro Oncol

ogico MD Anderson International, Madrid, Spain

(R.C.). Received August 19, 2010; accepted November 10, 2010. This work

is partially funded within the NeoMark project (FP7-ICT-2007-2-224483) by

the European Commission. Address correspondence to: S.S. e-mail:

sebastian.steger@igd.fraunhofer.de

ªAUR, 2011

doi:10.1016/j.acra.2010.11.010

1

From this model the ‘‘real’’ axial diameters were automati-

cally extracted by the following algorithm. In each slice, the

minimal area bounding rectangle is computed by applying

the rotating calipers algorithm (7) on the convex hull of all

pixels that are said to be part of the lymph node in the current

slice. The real long axial axis of the node is then said to be the

length of that rectangle with the largest length of all slices.

The real short axial axis is deﬁned as the orthogonal axis to

the long axial axis in the slice where this axis is maximized.

Furthermore a similar computation is carried out to auto-

matically extract the real 3D axes of the lymph nodes. An

approximation of the minimum volume bounding box is

extracted by eigenvector analysis of the points of the lymph

node model’s convex hull. Please note that this box may have

an arbitrary spatial orientation in the 3D space (Fig 1).

Then the measured axial axes were compared to the real axial

axes as well as to the real 3D axes in terms of relative difference

and Pearson product-moment correlation coefﬁcient (PMCC).

Moreover the PMCC between the extracted volume and the

following measurements was computed:

(a) Third power of the real axial short axis

(b) Third power of the real 3D short axis

(c) Product of the real long axial axis and the second power of

the real short axial axis

(d) Product of all three 3D axes

Note that all of these expressions are volumes, justifying

a linear correlation to the volume of the node (prerequisite

for computing the PMCC).

RESULTS

The average duration for the manual 3D model creation was

roughly 4.5 minutes per lymph node. For large lymph nodes

this duration went up to 25 minutes. The average inter-

observer variability of all lymph nodes is given in Table 1

for all measurements. In the average case the measured long

axial axis differed by 9.73% from the model’s automatically

extracted long axial axis and even 24.57% for the short axial

axis. The reason for the short axis’ huge difference is mostly

due to the fact that long and short axis are always measured

in the same axial slice, whereas the automated extraction

from the 3D model used the slice in which the axis orthogonal

to the node’s computed long axial axis is maximal.

Comparing the measured axial axes to the automatically

computed 3D axes of the model, it tur ns out that the relative

difference is 25.05% for the long axis and 19.97% for the short

axis, respectively. The larger average difference conﬁrms that

measuring the short axis is less sensitive to the node’s spatial

orientation than the long axis, which was one of the motivations

for revising the RECIST criteria.

For the comparability of measurements, the relative differ-

ence between the measured and the actual size is not as impor-

tant as the correlation. Imagine a measuring technique that

results in a similar overestimation of the real size for all lymph

nodes. Although the relative difference to the actual size may

be quite large, this measuring technique is still a valuable

predictor, because the error is predictable, resulting in a high

correlation between the measured and the actual size.

Figure 2 shows the measured axial axis versus the real axis

extracted from the 3D model for the long and the short axis.

Measuring the long axis results in an underestimation of the

real size of the node whereas the short axis sometimes is overes-

timated. This observation is a consequence of the node’s spatial

orientation. Only in the case, when the real long axis is in the

axial plane, the radiologist is able to assess the correct length.

In all other cases, the radiologist can only measure the projection

in the axial plane which in general is smaller.

The linear correlation coefﬁcients are 0.8885 for the long

axis and 0.9325 for the short axis, respectively. This indicates

Figure 1. Axial two-dimensional vs. three-dimensional assessment

of the dimensions of a lymph node.

Figure 2. Measured axial axis vs. automatically extracted three-

dimensional axis.

STEGER ET AL Academic Radiology, Vol -,No-, - 2011

2

that the underestimation of the real long axis by measuring the

axial long axis is not predictably and therefore once more

conﬁrms that the short axis is less sensitive to the node’s spatial

orientation than the long axis.

Because of the different shapes of lymph nodes, the axes by

themselves do not perfectly correlate with the node’s volume.

Figure 3 points this out by comparing the volume of boxes

whose size corresponds to various combinations of measured

or computed axes with the actual volume of the node.

The correlation coefﬁcient is 0.9379 for the third power of

the axial short axis, 0.9979 for the third power of the 3D short

axis, 0.9797 for the product of the long axial axis and the squared

short axial axis, and 0.9990 for the product of all three 3D axes.

3D assessment performs a lot better than asses sing the lymph

nodes in the axial plane. But if a 3D model was available the

volume could be assessed directly and the axes wouldn’t be

needed to estimate the volume. However, if a 3D model is

not available—as it is not in today’s clinical practice—the

node can only be assessed in the axial plane. The presented

linear correlation coefﬁcients indicate that the product of the

long axial axis and the squared short axial axis is a better

predictor the just the short axial axis. This is contrary to

RECIST 1.1, which requires assessment of the short axis only.

DISCUSSION

Although this study has conﬁrmed that measuring the short

axes is less sensitive to the spatial orientation of the node

than measuring the long axis (as suggested by RECIST 1.1),

it indicates that measuring the short axis in the axial plane

by itself as a prognostic factor for the size of a lymph node is

worse than also incorporating the long axis. However, a really

good estimation of the size can only be achieved by measuring

the axes in the 3D space or measuring the volume.

Even though 3D assessment of lymph nodes has many advan-

tages, manually delineating the lymph node in each CT slice is

not feasible in clinical routine because of the long time it takes

for the creation of the 3D model. The application of accurate

and robust automated segmentation algorithms, as reported

elsewhere (8), could compensate for that, thus signiﬁcantly

improving the assessment of lymph node size while decreasing

evaluation time, thereby increasing the signiﬁcance of disease

progression monitoring in cancer.

FUTURE WORK

Future work includes the automated 3D model creation using

a variation of the previously mentioned segmentation algorithm

and applying it to the datasets used in this study. Moreover, the

signiﬁcance of this study needs to be increased by applying it to

a larger dataset.

ACKNOWLEDGM ENTS

The authors like to thank Inmaculada Rodriguez Jimenez and

Francisco Javier Garcia Prado from the Radio-Diagnosis

Department, Centro Oncologico MD Anderson International

Spain for selecting the lymph nodes and assessing their proper-

ties, respectively, as well as Giuseppe Gafa and Ilaria Stellati from

the Department of Clinical Sciences, Section of Radiology,

Parma Hospital for assessing the lymph nodes’ properties.

REFERENCES

1. Civantos FJ, Zitsch RP, Schuller DE, et al. Sentinel lymph node biopsy

accurately stages the regional lymph nodes for t1-t2 oral squamous cell

carcinomas: results of a prospective multi-institutional trial. J Clin Oncol

2010; 28:1395–1400.

2. Prasad SR, Jhaveri KS, Saini S, et al. CT tumor measurement for therapeutic

response assessment: comparison of unidimensional, bidimensional, and

volumetric techniques initial observations. Radiology 2002; 225:416–419.

3. Heussel CP, Meier S, Wittelsberger S, et al. Follow-up CT meas urement of

liver malignoma according to RECIST and WHO vs. volumetry. Rofo 2007;

179:958–964.

4. Therasse P, Arbuck SG, Eisenhaue r EA, et al. New guidelines to evaluate the

response to treatment in solid tumors. European Organization for Research

and Treatment of Cancer, National Cancer Institute of the United States,

National Cancer Institute of Canada. J Natl Cancer Inst 2000; 92:205–216.

TABLE 1. Inter-observer Variability

Absolute Relative

Standard

Deviation

Standard

Deviation (%)

Measured long axial axis 1.03 mm 9.01

Measured short axial axis 1.02 mm 11.45

Computed long axial axis 1.07 mm 8.00

Computed short axial axis 0.89 mm 8.37

Computed long three-

dimensional axis

1.74 mm 9.80

Computed short three-

dimensional axis

0.81 mm 8.83

Computed volume 0.17 mL 15.94

Figure 3. Axes vs. volume.

Academic Radiology, Vol -,No-, - 2011 3D ASSESSMENT OF LYMPH NODES VS. RECIST 1.1

3

5. Eisenhauer EA, Therasse P, Bogaerts J, et al. New Response Evaluation

Criteria In Solid Tumours: revised RECIST guideline (version 1.1). Eur J

Cancer 2009; 45:228–247.

6. Schwartz LH, Bogaerts J, Ford R, et al. Evaluation of lymph nodes with

RECIST 1.1. Eur J Cancer 2009; 45:261–267.

7. Toussaint G. Solving geometric problems with the rotating calipers. In:

Protonotarios EN, ed. MELECON 1983. NY: IEEE, 1983; A10.02/1-4.

8. Dornheim L, Dornheim J, Rossling I, et al. Model-based segmenta-

tion of pathological lymph nodes in CT data. SPIE 2010; 7623.

76234V.

STEGER ET AL Academic Radiology, Vol -,No-, - 2011

4

- CitationsCitations9
- ReferencesReferences14

- "Lymph node involvement determines cancer staging. Nodal stage also affects the patient survival rate and the time interval to development of distant metastasis (4, 7, 22, 23). Therefore, an evaluation of the lymph node stage is important to precisely assess the cancer burden and to correlate the tumor burden with the tumor stage. "

[Show abstract] [Hide abstract]**ABSTRACT:**To compare the diagnostic performance in evaluating the response of neoadjuvant chemotherapy (NAC), between the response evaluation criteria in solid tumor (RECIST) 1.0 and RECIST 1.1, on magnetic resonance imaging (MRI) for advance breast cancer patients. Breast cancer patients, who underwent NAC between 2005 and 2010, were included. Both prechemotherapy and post-chemotherapy MRIs were performed within 1-4 weeks before and after NAC. Only the patients with subsequent surgery were included. The response to NAC was assessed by using RECIST 1.0 and RECIST 1.1. Patients with a complete or partial response on MRI were considered as responders, and those with stable or progressive disease were considered as non-responders. Tumor necrosis > 50% on pathology was defined as responders and necrosis < 50% was defined as non-responders. The diagnostic accuracy of both RECIST 1.0 and RECIST 1.1 was analyzed and compared by receiver operating characteristic curve analysis. Seventy-nine females (mean age 51.0 ± 9.3 years) were included. Pathology showed 45 responders and 34 non-responders. There were 49 responders and 30 non-responders on RECIST 1.0, and in 55 patients, RECIST 1.0 results agreed with pathologic results (69.6%). RECIST 1.1 showed 52 responders and 27 non-responders. In 60 patients, RECIST 1.1 results were in accordance with pathology results (75.9%). The area under the ROC curve was 0.809 for RECIST 1.0 and 0.853 for RECIST 1.1. RECIST 1.1 showed better diagnostic performance than RECIST 1.0, although there was no statistically significant difference between the two.- [Show abstract] [Hide abstract]
**ABSTRACT:**Evaluation of tumor response is a vital element in clinical oncology research, particularly in the development of new drugs. Tumor response also plays a significant role in treatment decisions made by clinicians in practice. The underlying concept of tumor response, however, was developed as a result of limited understanding of tumor biology coupled with restricted availability of both effective treatments and imaging modalities. In recent years, impressive advances have been made in the treatment of cancer. Groundbreaking advances in our understanding of the molecular biology of tumor growth and proliferation have been made. New biologic agents have been approved for the treatment of several malignancies and, in many cases, biomarkers have been identified that can help predict those patients who will benefit. Pre-operative chemotherapy is now established for a number of tumor types. Modern imaging technologies allowing functional characterization of tumors have been introduced into clinical practice. In this new therapeutic landscape, the existing concept of tumor response risks becoming an anachronism, and revision of the criteria used to define tumor response is warranted. In this paper, we critically review the limitations of the classic criteria for tumor response assessment, and briefly discuss the potential role of alternative methodologies in providing a new, functional definition of tumor response. - [Show abstract] [Hide abstract]
**ABSTRACT:**Purpose: Therapy response evaluation in oncological patient care requires reproducible and accurate image evaluation. Today, common standard in measurement of tumour growth or shrinkage is one-dimensional RECIST 1.1. A proposed alternative method for therapy monitoring is computer aided volumetric analysis. In lung metastases volumetry proved high reliability and accuracy in experimental studies. High reliability and accuracy of volumetry in lung metastases has been proven. However, other metastatic lesions such as enlarged lymph nodes are far more challenging. The aim of this study was to investigate the reproducibility of semi-automated volumetric analysis of lymph node metastases as a function of both slice thickness and reconstruction kernel. In addition, manual long axis diameters (LAD) as well as short axis diameters (SAD) were compared to automated RECIST measurements. Materials and methods: Multislice-CT of the chest, abdomen and pelvis of 15 patients with lymph node metastases of malignant melanoma were included. Raw data were reconstructed using different slice thicknesses (1-5 mm) and varying reconstruction kernels (B20f, B40f, B60f). Volume and RECIST measurements were performed for 85 lymph nodes between 10 and 60 mm using Oncology Prototype Software (Fraunhofer MEVIS, Siemens, Germany) and were compared to a defined reference volume and diameter by calculating absolute percentage errors (APE). Variability of the lymph node sizes was computed as relative measurement differences, precision of measurements was computed as relative measurement deviation. Results: Mean absolute percentage error (APE) for volumetric analysis varied between 3.95% and 13.8% and increased significantly with slice thickness. Differences between reconstruction kernels were not significant, however, a trend towards middle soft tissue kernel could be observed.. Between automated and manual short axis diameter (SAD, RECIST 1.1) and long axis diameter (LAD, RECIST 1.0) no significant differences were found. The most unsatisfactory segmentation results occurred in higher slice thickness (3 and 5 mm) and sharp tissue kernel. Conclusion: Volumetric analysis of lymph nodes works satisfying in a clinical setting. Thin slice reconstructions (≤3 mm) and a middle soft tissue reconstruction kernel are recommended. LAD and SAD did not show significant differences regarding APE. Automated RECIST measurement showed lower APE than manual measurement in trend.

Data provided are for informational purposes only. Although carefully collected, accuracy cannot be guaranteed. Publisher conditions are provided by RoMEO. Differing provisions from the publisher's actual policy or licence agreement may be applicable.

This publication is from a journal that may support self archiving.

Learn more