A Novel Image Based Verification Method for Respiratory Motion Management in Radiation Therapy
ABSTRACT Precise localization of moving targets is essential to increase local control of the cancer via dose escalation while reducing the severity of normal tissue complication. Localization of targets in real time with radioopaque marker is less favorable considering the excess radiation dose to the patient and potential complications of implantation. Various external surrogates could provide indications of the targets' positions during the breathing process. However, there is a great deal of uncertainty in the correlation between external surrogates and internal target positions/trajectory during respiratory cycles. In order to address this problem, we have developed an algorithm that automatically establishes correspondences between the fluoroscopic sequence frames taken from the patient on the day of treatment and the various phases of a 4DCT planning data set. Image based mapping/synchronization procedure is performed using an underlying Markov model established for the breathing process. The mapping procedure is formulated as an optimization process and is solved efficiently using a dynamic programming technique. Results on the phantom, synthetic, and real patient data demonstrate the effectiveness of the proposed method in coping with respiratory correlation variations. The approach could primarily be used for automatic gating interval adaptation in the gated radiotherapy.

Conference Paper: SurfaceBased Respiratory Motion Classification and Verification.
[Show abstract] [Hide abstract]
ABSTRACT: To ensure precise tumor irradiation in radiotherapy a stable breathing pattern is mandatory as tumors are moving due to respiratory motion. Consequentially, irregularities of respiratory patterns have to be detected immediately. The causal motion of tissue also difiers due to difierent physiological types of respiration, e.g., chest or abdominal breathing. Currently used devices to measure respiratory motion do not incorporate complete surface deformations. Instead only small regions of interest are considered. Thereby, valuable information to detect difierent breathing patterns and types are lost. In this paper we present a system that uses a novel camera sensor called TimeofFlight (ToF) for auto matic classiflcation and veriflcation of breathing patterns. The proposed algorithm calculates multiple volume signals of difierent anatomical re gions of the upper part of the patient's body. Therefore disjoint regions of interest are deflned for both, the patient's abdomen and thorax. Us ing the calculated volume signals the type of respiration is determined in realtime by computing an energy coecient. Changing breathing patterns can be visualized using a 2D histogram, which is also used to classify and detect abnormal breathing phases. We evaluated the pro posed method on flve persons and obtained a reliable difierentation of chest and abdominal breathing in all test cases. Furthermore, we could show that the introduced 2D histogram enables an accurate determina tion of changing breathing patterns.Bildverarbeitung für die Medizin 2009: Algorithmen  Systeme  Anwendungen, Proceedings des Workshops vom 22. bis 25. März 2009 in Heidelberg; 01/2009
Page 1
A Novel Image Based Verification Method for
Respiratory Motion Management in Radiation
Therapy
Ali Khamene1, Christian Schaller2, Mamadou Diallo1, Juan Carlos Celi3,
Barbara Ofstad3, Joachim Hornegger2, X. Allen Li4, An Tai4, and John
Bayouth5
1Imaging and Visualization Dept., Siemens Corporate Research, 755 College Road
East, Princeton NJ 08540, USA
2Lehrstuhl fuer Mustererkennung, University of ErlangenNuremberg, Marten Str. 3,
91058 Erlangen Germany
3Siemens Oncology Care Systems, Hans Bunte Str. 10, 69123 Heidelberg, Germany
4Medical College of Wisconsin, Department of Radiation Oncology, 8701 Watertown
Plan Road, Milwaukee WI 53226, USA
5University of Iowa, Dept. of Radiation Oncology, 200 Hawkins Dr., LL West
Addition, Iowa City IA 52242, USA
ali.khamene@siemens.com
Abstract. Precise localization of moving targets in the thorax and ab
domen during the course of treatment can be used to increase local con
trol via dose escalation while reducing the severity of normal tissue com
plication. Localization of targets in real time with radioopaque marker
is not practical considering the excess radiation dose to the patient and
potential complications of implantation. Various external surrogates can
be used to give indications of locations of the targets during the breath
ing process. However, there is a great deal of uncertainty in correlation
between external surrogates and internal target positions/trajectory dur
ing respiratory cycles. This is due to the fact that the correlation tends
to vary through time as the patient’s breathing pattern changes. In this
paper, we address the problem of establishing the correlation between a
surrogate signal and an internal target prior to treatment. We have de
veloped an algorithm that automatically establishes correspondences be
tween the fluoroscopic sequence frames taken from the patient on the day
of treatment and the various phases of a 4DCT planning data set. Image
based mapping/synchronization procedure is performed using an under
lying Markov model established for the breathing process. The mapping
procedure is formulated as an optimization process and is solved effi
ciently using a dynamic programming technique. Results on the phan
tom, synthetic, and real patient data demonstrate the effectiveness of the
proposed method in coping with respiratory correlation variations. The
approach could primarily be used for automatic gating interval adapta
tion in the gated radiotherapy.
Page 2
1Introduction and Background
Radiation therapy is a treatment modality with primary goal of precisely deliv
ering the prescribed dose to the tumor while minimizing the dose to surrounding
healthy tissues and critical structures. Recent technological advances in radiation
therapy enable delivering of a highly conformal radiation dose distribution to a
morphologically complex target volume. Main sources of delivery errors include
the inter and intrafraction target and organ motions.
1.1 Respiratory Motion in Radiotherapy
Intrafraction organ motion is mainly caused by patient respiration. Respiration
induced tumor motion has been studied indirectly by movements of organs,
radioopaque markers (implanted at the vicinity of the tumor) [12], and surrogate
organ/structure [6]. Realtime imaging technologies can be used to observe either
tumor motion or indirectly perceive the tumor motion [10]. It has been shown
that the tumor motion can be as large as 26 cm [7]. The variation of the motion
size as well as the breathing pattern makes the prediction of the tumor location
very difficult.
There are three major approaches for dealing with tumor motion in radiation
treatment. In the first method, which is clinically used, the patient is asked to
hold the breath or do a forced shallow breathing. The drawback here is that
the method requires patient compliance. Furthermore, the implementation of
the strategy is rather subjective and it depends on the skill of the therapist
and participation of the patient. In the second and third method, the patient
is required to do free breathing and no compliance is expected. In the second
method, which is usually referred to as “gating”, the radiation beam is gated
in a way that the tumor is only treated at the “prescribed or planned” position
[8]. Whereas in the third method, which usually referred to as “tracking”, the
radiation beam or table is moved so that the tumor always remains at a fixed
point within the treatment field [11]. Although, tracking has the advantage of
a higher duty cycle and a shorter delivery time, it is technically challenging to
implement [13]. In contrast, respiratory gating is more practical, and has been
adopted in clinic practice by a number of cancer centers [7].
1.2Gated Radiotherapy
We focus on the external gating approach that it is noninvasive and the exter
nal surrogate signals could be generated from strain gauges around the patient’s
thorax, optical markers on the patient skin, or airbag systems measuring air
flow or temperature. On the other hand, having a real time external signal is
not equivalent to knowing the location of the tumor at all times. Exploiting
external signal blindly to turn the radiation beam on and off can cause signif
icant errors. In particular, the correlation between the tumor motion and the
surrogate signal may change from the planning phase to the treatment sessions.
The major drawback of this method is the variability of the correlation between
the external signal amplitude (marker position) and the internal target position.
Page 3
The main challenge of the gated treatment is to adapt the gating intervals based
on uptodate breathing pattern of the patient. This paper focuses on an image
based method, which maps the uptodate images of the patient to the various
phases captured during the planning phase. The mapping function is then used
to determine optimal gating interval for the breathing surrogate for a specific
treatment session.
2Image Based Verification Technique for Gated
Radiotherapy
At the planning phase, in order to have a better understanding of the tumor
motion, time resolved volumetric CT images (socalled 4DCT) are acquired (see
figures 1a). A surrogate signal is usually used to resolve the projections prior
to reconstruction into a set of phases (usually 812) [14]. After reconstruction,
various phases of the 4DCT are represented by either an amplitude or phase
from the surrogate signal. The 4DCT scans are processed and the gross target
volume (GTV) of the tumor for each phase is determined. Furthermore, based
on the tumor location, residual motion, and duty cycle, a gating phase interval
that is delimited by two phases of the 4DCT is selected. The union of the GTVs
from the phases within the gating interval forms a new volume that is usually
referred to as internal target volume (ITV). Based on the prescribed dose and
shape of the ITV, a treatment plan is devised. Furthermore, amplitude or phase
of the surrogate signal at the two end phases of the gating phase interval signifies
the planned gating surrogate interval.
At the day of treatment, the initial patient setup is performed. . In order to
reduce the positioning bias due to respiratory motion, we select a single phase
out of 4DCT that roughly resembles the same breathing phase apparent from
the acquired 2D setup xray or portal images. An image guided method is then
used to correct for the residual positioning error [5]. After the setup verification
process, the patient is at the optimal treatment position. Since the surrogates
signal correlation with the target position varies day by day, the gating intervals
set during the planning phase might not be valid anymore. It is desirable to
check and possibly to adapt the gating intervals using an uptodate respiratory
pattern seen in a pretreatment fluoroscopic acquisition.
2.1 Problem Statement
The image based verification process starts with acquisition of a pretreatment
image sequence. Mega voltage or kilo voltage fluoroscopic acquisition synchro
nized with a breathing surrogate is considered (see figures 1b). We assume that
the pretreatment image sequence {Ipt(j)}j∈[0,K−1]has K frames taken at inter
vals of δt covering at least several respiratory cycles (both j and K are integers).
Furthermore, a corresponding synchronized surrogate signal {spt(j)}j∈[0,K−1]is
acquired. In both notations, superscript pt stands for “pretreatment”. We also
assume that we have a planning 4DCT acquisition, which includes L phases of
breathing. We generate a set of Digitally Reconstructed Radiographs (DRRs),
{Ip(i)}i∈[0,L−1], using each phase of the planning 4DCT (e.g., ith) with the exact
Page 4
(a) (b)
Fig.1. (a) depicts the coronal slices of planning 4DCT data sets with visible con
tour lines. (b) demonstrates two frames of a fluoroscopic acquisition overlaid with the
outlines of the projected structures from the planning 4D dataset.
geometry of the pretreatment imaging system and the known patient position
from the initial patient setup step (both i and L are integers). As mentioned in
the previous section, the gating phase interval is delimited by two phases imin
and imaxboth within [0,L−1]. In order to map the gating phase interval, which
is set at the planning step onto the surrogate signal acquired prior to treatment,
we need to establish a mapping function L : [0,K − 1] ?→ [0,L]:
L(j) =
?i ∈ [0,L − 1] there is a correspondence for the frame j
The mapping function needs to determined based on the image information
from the two time series of {Ip(i)}i∈[0,L−1]and {Ipt(j)}j∈[0,K−1]. Once the map
ping function is estimated, the treatment day gating surrogate interval can be
estimated as follows:
L
no correspondence
(1)
smin
pt
=min
jL(j)∈[imin,imax]spt(j)
.
(2)
For computing smax
gating surrogate interval [smin
cycle. The surrogate signal amplitudes of spt(j) for j satisfying the condition
L(j) ∈ [imin,imax] can also be further analyzed to detect the outliers, mainly to
increase the duty cycle as much as possible.
pt
, we switch the min to max in equation 2. The treatment day
pt ,smax
pt
] can also be used to estimate a new duty
2.2Image Similarity Matrix
The first step in establishing an optimal mapping function is to generate a sim
ilarity image matrix between the two image sequences of {Ip(i)}i∈[0,L−1]and
{Ipt(j)}j∈[0,K−1]. We generate a matrix M with size of (L + 1) × K, where
M(i,j) (for i < L) is the normalized mutual information value between the two
images of Ip(i) and Ipt(i). Each column of the matrix M can be thought of as
matching likelihood profile between a frame of the fluoroscopic sequence and
Page 5
various phases of the planning DRRs. A clear peak in that profile could signal
a potential match. However, in the case where there is no match, one would
expect to have consistently low value across the whole column. In order to deal
properly with these cases, we have considered the last column, which is in fact
representing the likelihood of having a “nomatch” situation. For each element
in the last row of the matrix M, we consider a constant value that is the median
of the maximums for various columns minus the range of the similarity values
of the corresponding column.
Finding maximum values for each column of the matrix M could be a po
tential solution for the mapping function L. However, the problem with this
approach is that since each columns are processed independently, no temporal
consistency is considered. For example, images from certain phases of inhale
and exhale could look quite similar if they are observed independently and not
considered in the context of a sequence. Not considering temporal aspect of the
process also makes it difficult to deal with cases, where there is no match. Fur
thermore, robustness to noise and/or to variations of image intensities from the
two sequences could be easily compromised.
2.3Respiratory Motion as A Markov Process
In order to enforce temporal consistency in finding the mapping function L from
the image similarity matrix M, we first need to find a reasonable model for
the breathing process [4]. We assume that the breathing can be modeled as a
Markov process. We consider L + 1 states for the Markov process; L of which
is based on the number of phases in the planning 4DCT and an additional one
that accommodates variations beyond the observed states in the planning 4DCT.
Since the pretreatment acquisition is not triggered, it can start at any breathing
phase, therefore the initial probabilities of the various states are constant(i.e.
P0(i) =
to the process states, therefore the emission probability matrix is the identity.
The state transition probability matrix S, which is of size (L+1)×(L+1) can
be estimated from a set of observation sequences using BaumWelch algorithm
[1]. Observation sequences, in this case, could be derived from various surrogate
signals, where the signal amplitude is separated into L + 1 bins.
We denote the states of the process as Sifor i ∈ [0,L], where SLis the state
not present in the prior observations. S0denotes the start of inhale and SL−1
indicates the end of exhale. The elements of the transition probability matrix
S are S(i,j) = P(sn+1= Sjsn= Si) for i,j ∈ [0,L], where sn indicates the
state at the the time index n. Therefore, the sum of each column or row of
the matrix S is one. Aside from being able to estimate the transition matrix
given a series of sequences, there are many heuristic constraints that one might
consider in determining the elements of the matrix Si. For example, it is more
likely to have an arrangement of states, where the state at k + 1 has a higher
state index than the state at k. The only exception is for SL−1that most likely
transits back to state S0. Furthermore, there is a possibility that the states
repeat themselves. This case covers scenarios, where the breathing process is
1
L+2for i ∈ [0,L + 1]). We consider that the observations are identical
Page 6
(a) (b)
Fig.2. (a) depicts the state transition network (thicker lines denote higher transition
probability). (b) shows various instances of the sequences from the Markov process de
picted at (a). Top most shows a most probable case. Middle row shows three breathing
cycles, where the inhale is elongated in the first cycle, shallow breathing is present in
the second cycle, and the slow breathing is shown in the last cycle. The bottom row
depicts deep breathing in the first cycle, fast exhale and deep inhale in the second
cycle, and fast and deep inhale and slow exhale in the last cycle.
slow. Furthermore, there should be a possibility for having forward transitions
more than one step covering fast breathing scenarios. It is also reasonable to
consider that the unknown state of SLis more likely to happen after the states
closer to the full inhale or full exhale. Considering all these constraints, we
consider the L×L submatrix from S to be a circulant matrix. The first column
of the matrix has a negative quadratic form (i.e., maximum at the first element
and quadratically decreasing as the index increases). The last row of the matrix
represents the probability of the transition to an unknown state. For the last
row, we consider to have a four degree polynomial, which maximizes at the mid
range of the index (i.e., full inhale) and minimizes at the mid range of inhale
and exhale. Finally, a normalization factor is applied to ensure that the sum of
each column is one. The probability of a given sequence having been produced
by a specific Markov process can be computed using the initial state probability
P0and the transition matrix S using a forwardbackward propagation algorithm
[9]. An example of a heuristically determined transition matrix for the breathing
cycle is depicted in figure 2.
2.4 Mapping as a Solution of an Optimization Problem
The mapping function L is in fact a constellation of states Sifor i ∈ [0,L] from
the planning phase. The likelihood of a certain mapping due to the imaging cues
can be computed using the image similarity matrix M. Furthermore, the proba
bility of a certain mapping solely due to the a priori Markov process model can
be computed using P0and S. Therefore, we setup an optimization to maximize
Page 7
the posteriori probability that is expanded to the product of the likelihood and
prior probability using bayesian rule. The optimization formulation can then be
written as:
?L = argmax
where?L is the solution. Equation 3 depicts a combinatorial optimization prob
evaluation of the cost function in this problem is computationally inexpensive,
therefore standard optimization techniques such as simulated annealing could be
used. However, since the problem can be looked at as overlapping subproblems,
we could use Dynamic Programming to find the optimal solution [2]. Similar to
Dijkstra shortest path algorithm, we can define and store the maximum cost of
getting at the time index j in a (L + 1) × K matrix Q:
Once the matrix Q in the equation 4 is computed recursively, the solution of the
optimization problem in the equation 3 is:
L
j<K
?
j=0
M(L(j),j)
×
P0(L(0))
j<K−1
?
j=1
S(L(j),L(j + 1))
likelihoodprior
(3)
lem, since the number of possibilities for state constellations are finite. The
Q(i,j) =
∞
M(i,j) × P0(i)
M(i,j) + max
j < 0
j = 0
i
(Q(i,j − 1) × S(i,j)) otherwise
(4)
?L(j) =˜i = argmax
i
Q(i,j)
forj ∈ [0,K].
(5)
Estimated mapping function?L can be used in equation 2 to compute the updated
3Experimental Results
gating surrogate interval.
To evaluate the performance of the proposed method, we define two error mea
sures. First error metric e1 is the percentage of wrongly labeled fluoroscopic
frames. Second error measure e2is the percentage of wrongly labeled “nomatch”.
Although the wrongly labeled “no match” does not have any affect on the com
puted gating surrogate interval, it has an adverse impact on the duty cycle.
For the first series of tests, we used a breathing phantom that has a moving
target within a hallow cavity. The speed and range of the target motion within
the phantom can be adjusted. A ten phase 4DCT of the breathing phantom was
acquired using Siemens Sensation 64 CT Scanner (Siemens Medical Solutions,
Med CT, Forchheim, Germany). Simulator fluoroscopic sequences of the phan
tom were taken using Siemens Mevasim S Simulator (Siemens Medical Solutions,
Med OCS, Concord CA, USA) from the phantom with three different motion
ranges and speeds. For the second and third series, the 4DCT of patients with
synthetic and real fluoroscopic sequences were used. We acquired two 4DCT
Page 8
Test
Phantom Original (i.e., same as 4DCT)
Phantom Slow Breathing
Phantom Fast Breathing
Phantom Deep Breathing
Phantom Shallow Breathing
Patient1 with Syn. Fluo.
Patient1 with Syn. Fluo. Modified Pattern 1
Patient1 with Syn. Fluo. Modified Pattern 2
Patient2 with Fluo.
Patient3 with Fluo.
Average
e1 (%) e2 (%)
0%
8%
5%
8%
8%
4%
7%
8%
15%
14%
7%
0 %
0 %
0 %
15 %
15 %
10 %
10%
10%
15%
10%
8.5%
Table 1. Results of the two error measures for various datasets.
scans of the same patient at two different time points. We generated DRRs
from one the 4DCT phases and puzzled them together to form synthetic fluo
roscopic sequences. We generated three sequences. For the first, we concatenate
the original 4DCT phases multiple times. For the second and third, we varied
the sequence order and generated two distinct synthetic breathing patterns.
Finally, we acquired two 4DCT scans of the two different patients with the
corresponding simulator acquisitions. In all the cases, correct correspondences
were manually selected. We used splitscreen and a blending display method to
identify the correct correspondences for each fluoroscopic frame out of various
4DCT phases. Sample images and the similarity matrix with the overlaid map
ping function are depicted in Figure 3. The results of the two error measures are
brought in the table 3. The processing time depends on the number of fluoro
scopic sequence frame and it is roughly one second per frame. All the tests were
performed on an Intel Centrino Duo CPU with 2.0 GHz and 2 GB of RAM and
a NVIDIA Quadro FX 2500 display adapter.
4Summary and Conclusion
We have developed an image based mapping/synchronization procedure that au
tomatically labels pretreatment fluoroscopic image frames with the correspond
ing phase from 4DCT. The mapping procedure is formulated as an optimization
process, which finds an optimal mapping maximizing the image similarity be
tween the corresponding pairs, while preserving a temporal coherency to an es
tablished Markov model for breathing. The mapping procedure also detects the
frames with no corresponding phase from the planning 4DCT. The fluoroscopic
image based verification addresses the problem of establishing the correlation
between surrogate signal and internal target prior to treatment. By virtue of
having the labels generated from a mapping process, we can adaptively change
the surrogate gating interval to the uptodate breathing pattern of the patient.
Results on phantom, synthetic, and patient data shows in average 93 percent of
frames are correctly labeled.
Page 9
(a)(b)
Fig.3. (a) and (b) depicts a 4DCT frame with matching fluoroscopic frame with over
laid contours for phantom and patient datasets, respectively. The bottom row shows
the corresponding image similarity matrix for each case.
References
1. Baum, L. E. ,Petrie T., Soules G. , and Weiss N., A maximization technique oc
curring in the statistical analysis of probabilistic functions of Markov chains, Ann.
Math. Statist., 1970, 41(1) : pp. 164171.
2. Bellman, R., Dynamic Programming, Princeton Univ. Press, Princeton, NJ, 1957.
3. Berbeco, R.I., et. al., Towards fluoroscopic respiratory gating for lung tumours with
out radiopaque markers. Phys Med Biol, 2005, 50(19): p. 448190.
4. Huanmei, W., Sharp, G. C., et. al., A finite state model for respiratory image guided
radiation therapy, Phys. Med. Biol. 2004, 49, pp 53575372.
5. –
6. Keall, P.J., V.R. Kini, S.S. Vedam, and R. Mohan, Motion adaptive xray therapy:
a feasibility study. Phys Med Biol, 2001, 46(1): pp 110.
7. Keall P. J., et. al., The management of respiratory motion in radiation oncology
report of AAPM Task Group 76, Med Phys., 2006, 33(10): pp 3874900.
8. Kubo, H.D. and B.C. Hill, Respiration gated radiotherapy treatment: a technical
study. Phys Med Biol, 1996, 41(1): pp 8391.
9. Lawrence R. Rabiner, A tutorial on Hidden Markov Models and selected applications
in speech recognition, Proceedings of the IEEE, 1989, 77 (2): pp 257286.
10. Onimaru, R., et. al., The effect of tumor location and respiratory function on
tumor movement estimated by realtime tracking radiotherapy (RTRT) system. Int
J Radiat Oncol Biol Phys, 2005, 63(1): pp 1649.
11. Schweikard, A., et. al., Robotic motion compensation for respiratory movement
during radiosurgery. Comput Aided Surg, 2000. 5(4): pp. 26377.
12. Shirato, H., et. al., Feasibility of insertion/implantation of 2.0mmdiameter gold
internal fiducial markers for precise setup and realtime tumor tracking in radio
therapy. Int J Radiat Oncol Biol Phys, 2003, 56(1), pp 2407.
13. Rangaraj, D. and L. Papiez, Synchronized delivery of DMLC intensity modulated
radiation therapy for stationary and moving targets. Med Phys, 2005, 32(6): pp
180217.
Page 10
14. Rietzel, E., Chen, G. T. T., Improving retrospective sorting of 4D computed to
mography data, Med Phys, 2006, 33(2): pp 377379.