3D Assessment of Lymph Nodes vs. RECIST 1.1
In today's clinical practice, the size of lymph nodes is assessed by measuring the long and the short axis in the axial plane. This study compares this approach with three-dimensional (3D) assessment. For a representative set of 49 lymph nodes, the axes in the axial plane have been measured and a 3D model has been created manually. Based on the 3D model, the real axial long and short axis as well as the three 3D axes and the volume have been computed and compared to the measured axial axes. The inter-observer variability is around 10% for all measured lengths and almost 16% for the computed volume. The average relative error of the measured long (short) axial axis is 9.73% (24.57%) to the computed axial axis and 25.05% (19.97%) to the computed 3D axis, respectively. The product of the axial long axis and the square of the axial short axis provides best correlation to the volume. This study confirms Response Evaluation Criteria In Solid Tumours 1.1 that measuring the short axis is more robust than measuring the long axis because of less impact of the node's spatial orientation. Nonetheless it is shown that considering both axes is a better prognostic factor for the volume than measuring the short axis only.
3D Assessment of Lymph Nodes vs.
Sebastian Steger, Dipl.-Ing., Fabio Franco, MD, Nicola Sverzellati, MD, Gianfranco Chiari, MD,
Ramon Colomer, MD
Rationale and Objectives: In today’s clinical practice, the size of lymph nodes is assessed by measuring the long and the short axis in the
axial plane. This study compares this approach with three-dimensional (3D) assessment.
Materials and Methods: For a representative set of 49 lymph nodes, the axes in the axial plane have been measured and a 3D model has
been created manually. Based on the 3D model, the real axial long and short axis as well as the three 3D axes and the volume have been
computed and compared to the measured axial axes.
Results: The inter-observer variability is around 10% for all measured lengths and almost 16% for the computed volume. The average
relative error of the measured long (short) axial axis is 9.73% (24.57%) to the computed axial axis and 25.05% (19.97%) to the computed
3D axis, respectively. The product of the axial long axis and the square of the axial short axis provides best correlation to the volume.
Conclusion: This study conﬁrms Response Evaluation Criteria In Solid Tumours 1.1 that measuring the short axis is more robust than
measuring the long axis because of less impact of the node’s spatial orientation. Nonetheless it is shown that considering both axes is
a better prognostic factor for the volume than measuring the short axis only.
Key Words: Lymph node staging; RECIST; volumetry.
he presence and location of lymph node metastases (N
descriptor) is an important component of radiologic
staging in patients with cancer of different types.
Malignant lymph nodes are common target and nontarget
lesions, whose sizes have to be assessed for staging and disease
progression moni toring.
For example in oral cancer, accurate staging is the most
impor tant f actor that guides therapeutic decision making.
Staging methods inclu de radiological assessment that usually
entails imaging of the neck with computed tomography
(CT) or magnetic resonance imaging, or both. Lymphatic
metastases will develop in 20%–30% of patients with early
oral cancers and imply decreased survival (1).
The size can be measured using unidimensional, bidimen-
sional, and volumetric techniques (2). For liver malignomas
volumetric analysis is superior (3). However, in current clinical
practice the size is assessed in CT scans by measuring the axis in
the axial plane according to the Response Evaluation Criteria
In Solid Tumours (RECIST) criteria (4).
Besides general limitations of planar assessment of three-
dimensional (3D) structures, the limitation to the axial plane
for assessing arbitrarily spatial oriented structures reduces
reliability. This issue was addressed by the revised RECIST
criteria (5,6), by considering the short axis only.
In this study, we further explored the degree of inter-observer
variability in the assessment of the lymph nodes’ diameters and
tested which diameters in the axial plane better correlate with
MATERIALS AND METHODS
An experienced radiologist (I.R. with 15 years of experience)
identiﬁed a representative set of 49 lymph nodes in 13 head
and neck CT scans of oral cancer patients (5 male, 8 female,
age range 36–93 years). The set contains benign as well as path-
ological lymph nodes, their short axial axis ranging between 4.2
and 24.3 mm. To avoid any kind of selection bias, the study
subjects were consecutive.
Five different radiologists (G.C., F.F., G.G., I.S. and J.G., with
15, 3, 3, 3, and 6 years in oncologic imaging, respectively) from
two different hospitals evaluated those lymph nodes by
measuring their long and short axis in the axial plane of the slice
in which those measurements appeared to be maximal in the
whole node. Then the boundaries of each lymph node in
each slice the lymph node is visible in were manually delineated
using an electronic free-hand region of interest, enabling the
creation of an accurate 3D model of the node. The time for
delineating was recorded for each node.
Acad Radiol 2011; -:1–4
From the Competence Center Medical Imaging, Fraunhofer IGD,
Fraunhoferstrasse 5, 64283 Darmstadt, Germany (S.S.); Department of
Clinical Sciences, Section of Radiology, Parma Hospital, Parma, Italy (F.F.,
N.S., G.C.); Centro Oncol
ogico MD Anderson International, Madrid, Spain
(R.C.). Received August 19, 2010; accepted November 10, 2010. This work
is partially funded within the NeoMark project (FP7-ICT-2007-2-224483) by
the European Commission. Address correspondence to: S.S. e-mail:
From this model the ‘‘real’’ axial diameters were automati-
cally extracted by the following algorithm. In each slice, the
minimal area bounding rectangle is computed by applying
the rotating calipers algorithm (7) on the convex hull of all
pixels that are said to be part of the lymph node in the current
slice. The real long axial axis of the node is then said to be the
length of that rectangle with the largest length of all slices.
The real short axial axis is deﬁned as the orthogonal axis to
the long axial axis in the slice where this axis is maximized.
Furthermore a similar computation is carried out to auto-
matically extract the real 3D axes of the lymph nodes. An
approximation of the minimum volume bounding box is
extracted by eigenvector analysis of the points of the lymph
node model’s convex hull. Please note that this box may have
an arbitrary spatial orientation in the 3D space (Fig 1).
Then the measured axial axes were compared to the real axial
axes as well as to the real 3D axes in terms of relative difference
and Pearson product-moment correlation coefﬁcient (PMCC).
Moreover the PMCC between the extracted volume and the
following measurements was computed:
(a) Third power of the real axial short axis
(b) Third power of the real 3D short axis
(c) Product of the real long axial axis and the second power of
the real short axial axis
(d) Product of all three 3D axes
Note that all of these expressions are volumes, justifying
a linear correlation to the volume of the node (prerequisite
for computing the PMCC).
The average duration for the manual 3D model creation was
roughly 4.5 minutes per lymph node. For large lymph nodes
this duration went up to 25 minutes. The average inter-
observer variability of all lymph nodes is given in Table 1
for all measurements. In the average case the measured long
axial axis differed by 9.73% from the model’s automatically
extracted long axial axis and even 24.57% for the short axial
axis. The reason for the short axis’ huge difference is mostly
due to the fact that long and short axis are always measured
in the same axial slice, whereas the automated extraction
from the 3D model used the slice in which the axis orthogonal
to the node’s computed long axial axis is maximal.
Comparing the measured axial axes to the automatically
computed 3D axes of the model, it tur ns out that the relative
difference is 25.05% for the long axis and 19.97% for the short
axis, respectively. The larger average difference conﬁrms that
measuring the short axis is less sensitive to the node’s spatial
orientation than the long axis, which was one of the motivations
for revising the RECIST criteria.
For the comparability of measurements, the relative differ-
ence between the measured and the actual size is not as impor-
tant as the correlation. Imagine a measuring technique that
results in a similar overestimation of the real size for all lymph
nodes. Although the relative difference to the actual size may
be quite large, this measuring technique is still a valuable
predictor, because the error is predictable, resulting in a high
correlation between the measured and the actual size.
Figure 2 shows the measured axial axis versus the real axis
extracted from the 3D model for the long and the short axis.
Measuring the long axis results in an underestimation of the
real size of the node whereas the short axis sometimes is overes-
timated. This observation is a consequence of the node’s spatial
orientation. Only in the case, when the real long axis is in the
axial plane, the radiologist is able to assess the correct length.
In all other cases, the radiologist can only measure the projection
in the axial plane which in general is smaller.
The linear correlation coefﬁcients are 0.8885 for the long
axis and 0.9325 for the short axis, respectively. This indicates
Figure 1. Axial two-dimensional vs. three-dimensional assessment
of the dimensions of a lymph node.
Figure 2. Measured axial axis vs. automatically extracted three-
STEGER ET AL Academic Radiology, Vol -,No-, - 2011
that the underestimation of the real long axis by measuring the
axial long axis is not predictably and therefore once more
conﬁrms that the short axis is less sensitive to the node’s spatial
orientation than the long axis.
Because of the different shapes of lymph nodes, the axes by
themselves do not perfectly correlate with the node’s volume.
Figure 3 points this out by comparing the volume of boxes
whose size corresponds to various combinations of measured
or computed axes with the actual volume of the node.
The correlation coefﬁcient is 0.9379 for the third power of
the axial short axis, 0.9979 for the third power of the 3D short
axis, 0.9797 for the product of the long axial axis and the squared
short axial axis, and 0.9990 for the product of all three 3D axes.
3D assessment performs a lot better than asses sing the lymph
nodes in the axial plane. But if a 3D model was available the
volume could be assessed directly and the axes wouldn’t be
needed to estimate the volume. However, if a 3D model is
not available—as it is not in today’s clinical practice—the
node can only be assessed in the axial plane. The presented
linear correlation coefﬁcients indicate that the product of the
long axial axis and the squared short axial axis is a better
predictor the just the short axial axis. This is contrary to
RECIST 1.1, which requires assessment of the short axis only.
Although this study has conﬁrmed that measuring the short
axes is less sensitive to the spatial orientation of the node
than measuring the long axis (as suggested by RECIST 1.1),
it indicates that measuring the short axis in the axial plane
by itself as a prognostic factor for the size of a lymph node is
worse than also incorporating the long axis. However, a really
good estimation of the size can only be achieved by measuring
the axes in the 3D space or measuring the volume.
Even though 3D assessment of lymph nodes has many advan-
tages, manually delineating the lymph node in each CT slice is
not feasible in clinical routine because of the long time it takes
for the creation of the 3D model. The application of accurate
and robust automated segmentation algorithms, as reported
elsewhere (8), could compensate for that, thus signiﬁcantly
improving the assessment of lymph node size while decreasing
evaluation time, thereby increasing the signiﬁcance of disease
progression monitoring in cancer.
Future work includes the automated 3D model creation using
a variation of the previously mentioned segmentation algorithm
and applying it to the datasets used in this study. Moreover, the
signiﬁcance of this study needs to be increased by applying it to
a larger dataset.
The authors like to thank Inmaculada Rodriguez Jimenez and
Francisco Javier Garcia Prado from the Radio-Diagnosis
Department, Centro Oncologico MD Anderson International
Spain for selecting the lymph nodes and assessing their proper-
ties, respectively, as well as Giuseppe Gafa and Ilaria Stellati from
the Department of Clinical Sciences, Section of Radiology,
Parma Hospital for assessing the lymph nodes’ properties.
1. Civantos FJ, Zitsch RP, Schuller DE, et al. Sentinel lymph node biopsy
accurately stages the regional lymph nodes for t1-t2 oral squamous cell
carcinomas: results of a prospective multi-institutional trial. J Clin Oncol
2. Prasad SR, Jhaveri KS, Saini S, et al. CT tumor measurement for therapeutic
response assessment: comparison of unidimensional, bidimensional, and
volumetric techniques initial observations. Radiology 2002; 225:416–419.
3. Heussel CP, Meier S, Wittelsberger S, et al. Follow-up CT meas urement of
liver malignoma according to RECIST and WHO vs. volumetry. Rofo 2007;
4. Therasse P, Arbuck SG, Eisenhaue r EA, et al. New guidelines to evaluate the
response to treatment in solid tumors. European Organization for Research
and Treatment of Cancer, National Cancer Institute of the United States,
National Cancer Institute of Canada. J Natl Cancer Inst 2000; 92:205–216.
TABLE 1. Inter-observer Variability
Measured long axial axis 1.03 mm 9.01
Measured short axial axis 1.02 mm 11.45
Computed long axial axis 1.07 mm 8.00
Computed short axial axis 0.89 mm 8.37
Computed long three-
1.74 mm 9.80
Computed short three-
0.81 mm 8.83
Computed volume 0.17 mL 15.94
Figure 3. Axes vs. volume.
Academic Radiology, Vol -,No-, - 2011 3D ASSESSMENT OF LYMPH NODES VS. RECIST 1.1
5. Eisenhauer EA, Therasse P, Bogaerts J, et al. New Response Evaluation
Criteria In Solid Tumours: revised RECIST guideline (version 1.1). Eur J
Cancer 2009; 45:228–247.
6. Schwartz LH, Bogaerts J, Ford R, et al. Evaluation of lymph nodes with
RECIST 1.1. Eur J Cancer 2009; 45:261–267.
7. Toussaint G. Solving geometric problems with the rotating calipers. In:
Protonotarios EN, ed. MELECON 1983. NY: IEEE, 1983; A10.02/1-4.
8. Dornheim L, Dornheim J, Rossling I, et al. Model-based segmenta-
tion of pathological lymph nodes in CT data. SPIE 2010; 7623.
STEGER ET AL Academic Radiology, Vol -,No-, - 2011