Real-time positioning and tracking technique for endovascular untethered microrobots propelled by MRI gradients.
ABSTRACT A real-time positioning and tracking technique for untethered devices or robots magnetically propelled by a clinical magnetic resonance imaging (MRI) system is described. The local magnetic field induced by the device, composed of a ferromagnetic material, is used as a signature to localize the device on three one-dimensional projections. A high-precision 3D circular-motion system was used to assess the precision and accuracy of this method. The integration of this technique inside propulsion and imaging MRI sequences was also achieved to demonstrate the feasibility of this tracking scheme in a closed-loop control scheme. Finally, in vivo tracking during automatic navigation of an untethered device in the carotid artery of a living animal is demonstrated.
- SourceAvailable from: Charles C Tremblay[Show abstract] [Hide abstract]
ABSTRACT: Magnetic Resonance Navigation (MRN) aims at navigating artificial or synthetic untethered micro-devices and microrobots using an upgraded clinical Magnetic Resonance Imaging (MRI) system. For larger MRI-based navigated entities, past experiments proved that software-based upgrades only were sufficient. But for microrobots with an overall diameter of only a few tens of micrometers for travelling in narrower blood vessels, hardware upgrades need to be added to the MR scanner, resulting in a MRN system capable of generating 3D magnetic propulsion gradients on the microrobots well above the ones that could be generated by a clinical MRI scanner relying on software-upgrades only. But with the variety of models of clinical scanners coped with many versions of related operating software dedicated to MR imaging, implementing such upgrades that could operate with these scanners becomes a real challenge. As such, a new MRN platform architecture independent of the types of MR scanners is proposed and preliminary experimental data validating the potential of such microrobotic navigation system architecture integrated with a commercially available scanner are reported. The expected steering capabilities of the platform were evaluated initially using a special probe in the form of a magnetic catheter mimicking an anisotropic microrobot. Such special probe also allowed for easier recordings of the gradient steering force that would be induced on such microrobot while validating the technique for catheter steering which is also an important aspect since catheterization is often used for releasing the microrobots in larger arteries. Similarly, MR tracking of the same microrobot was also validated with the new system, confirming that tracking feedback data can be gathered in order to perform closed-loop navigation control.2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2011, San Francisco, CA, USA, September 25-30, 2011; 01/2011
Abstract—A real-time positioning and tracking technique for
untethered devices or robots magnetically propelled by a
clinical magnetic resonance imaging (MRI) system is described.
The local magnetic field induced by the device, composed of a
ferromagnetic material, is used as a signature to localize the
device on three one-dimensional projections. A high-precision
3D circular-motion system was used to assess the precision and
accuracy of this method. The integration of this technique inside
propulsion and imaging MRI sequences was also achieved to
demonstrate the feasibility of this tracking scheme in a closed-
loop control scheme. Finally, in vivo tracking during automatic
navigation of an untethered device in the carotid artery of a
living animal is demonstrated.
AGNETIC Resonance Imaging (MRI) systems are used
to gather non-invasively images of the interior of the
human body. MRI relies on magnetic field gradients to
transform the signal coming from the body into an image.
The use of MRI beyond its imaging capacity was
investigated in - and recently, a clinical MRI system
was successfully used to navigate a 1.5mm chrome steel
bead into the carotid artery of a living swine . The
implementation of a robotic guided system based on MRI
has the tremendous advantage of benefits from the hardware
and software of modern scanner already installed in most
Magnetic manipulation of devices and particles for
medical applications are commonly used in research as well
as in clinical practice. For example, in vivo steering of
magnetic particles by an external magnet although limited to
regions close to the skin while lacking feedback control is
done in order to concentrate drugs in tumor regions, which is
commonly known as Magnetic Drug Targeting (MDT) -
This work was supported in part by the Canada Research Chair (CRC)
in Micro/Nanosystem Development, Fabrication and Validation and grants
from the National Sciences and Engineering Research Council of Canada
(NSERC), the Province of Québec, the Canada Foundation for Innovation
(CFI) and the Fond Québecois de la Recherche sur la Nature et les
S. Martel (corresponding author) is with the NanoRobotics Laboratory,
Department of Computer and Software Engineering, and the Institute of
Biomedical Engineering, École Polytechnique de Montréal (EPM),
Montréal (Québec), P.O. Box 6079 Station Centre-ville, H3C 3A7 Canada
(phone: 514 340-4711 ext. 5098; fax: 514-340-4658; e-mail:
O. Felfoul (e-mail: firstname.lastname@example.org), E. Aboussouan (e-mail:
email@example.com) are with the NanoRobotics Laboratory at EPM.
A. Chanu (e-mail:
. Although not applicable to untethered objects,
magnetic fields are also used to apply a torque on a distal tip
of a catheter where a permanent magnet is placed, thus
helping surgeons to manipulate a surgical instrument .
An automated robotic system intended to operate inside
the human body requires high-precision real-time positioning
techniques. Most often, X-ray fluoroscopy is used to track
traditional medical devices such as catheters and guide wires.
Fluoroscopy is limited however to 2D plane projections and
is known to have ionizing radiation with injurious effect on
cells. On the other hand, optical means can be used to track
microrobots where direct line of sight is possible such as the
human eyes , but have limited or no potential
applications in the remaining regions inside the human body.
We propose in this paper to use MRI to track the position
of an untethered device or robot being controlled in the
human blood circulatory network using magnetic gradients.
We show that MRI is well suited for such application
providing many advantages including but not limited to lack
of radiation and enhanced tissue contrast.
II. PROJECTION BASED TRACKING
The tracking technique described here relies on MRI that
allows the acquisition of high quality images of the inside of
the human body. MRI is based on the Nuclear Magnetic
Resonance (NMR) phenomenon that allows the acquisition
of a weighted signal generated by hydrogen protons
subjected to a magnetic field. However, MRI is known to be
a relatively slow imaging modality compared to CT-scan or
ultrasound imaging. In fact, a high resolution MR image can
take several minutes to be acquired. Even if techniques such
as EPI  allow the acquisition of low resolution images at
a rate of 10 images per second, they are unusable in a closed-
loop real-time control system due to three main reasons.
First, images will suffer from susceptibility artifact, due to
the presence of the ferromagnetic device requiring complex
algorithms to extract the position. Second, fast MR images
involve the application of high gradients amplitude in order
to encode the image which may induce a lot of unwanted
motion on the bead by induction of displacement force on
the ferromagnetic material as described in more details in
. Finally, the spatial resolution is generally lowered in fast
imaging resulting in low quality images.
To overcome these limitations, we used projections to
track the bead taking advantage of the magnetic field it
Real-time Positioning and Tracking Technique for Endovascular
Untethered Microrobots Propelled by MRI Gradients
Ouajdi Felfoul, Member, IEEE, Eric Aboussouan, Member, IEEE, Arnaud Chanu, Member, IEEE, and
Sylvain Martel*, Senior Member, IEEE
b) Amplitude of the z component of the magnetic field for z=0
a) Amplitude of the z component of the magnetic field for y=0 (T)
Fig. 1. Simulation of the magnetic field (T) generated from a 1mm
diameter ferromagnetic bead (Bsat =1.67T) in a 1.5T surrounding field.
a) Sagital or coronal cut with y =0; b) axial cut with z=0. The MRI static
field is along z.
Signal intensity (a.u)
Fig. 2. (First column) Last x, y and z projections ‘P(x)’; (Second column)
first x, y and z projections ‘M(x)’ and; (Last column) their correlation ‘C(x)’
induces. When subject to a strong magnetic field, a
saturated ferromagnetic sphere induces a field B’ (T) given
where µ0 = 4π.10-7 (H.m-1)is the vacuum permeability, r (m)
is the distance from the center of the sphere and m (A.m2) is
the magnetic moment given by:
where MSat (A/m) is the vector magnetization and a (m) is the
radius of the sphere.
Following the application of an RF pulse, only spins
precessing at frequencies within the bandwidth of the pulse
will be excited. If the central frequency of the RF pulse is
offset with respect to the Larmor frequency, and no external
gradient is applied during the excitation, only the gradients
induced by the magnetic particle will specify which spins
will be excited as depicted in Fig. 1. The excited region
constitutes the magnetic signature of the tracked object. We
have opted for a gradient-echo implementation to meet the
real-time constraint of our application even if a spin-echo
sequence is more appropriate for tracking traditional medical
instruments such as catheters and guide wires . We have
added a dephaser gradient in order to partially compensate
the phase coherence loss due to the presence of the bead
induced magnetic field such as explained in .
I. POSITION MEASUREMENT
A. Relative Positioning
The relative positioning of the target can be done by
simply correlating the projection P(x) with a previously
acquired correlation mask M(x) as depicted in Fig. 2 .
Since the same magnetic element is used when M and P are
acquired, they present the same distortion and thus differ
only by a translation corresponding to the movement of the
element between their respective acquisitions.
A computationally efficient way of performing this
correlation is by doing it in the frequency domain (a.k.a. K-
C(x)( ( ).* (PM
The position xmax of the maximum of C(x) is then used to
determine the distance ∆x between the mask and the current
projection: ∆x=xmax-L/2, with L=length(M)=length(P).
Although efficient, this method has the drawback of
introducing an ambiguity in the position.
B. Resolving Positional Ambiguity
Because the discrete Fourier transform assumes a periodic
function, a wrap-around effect will be observed when ∆x≥
L/2. The common solution to this problem would be to
perform the inverse FT of M and P, pad them with L/2 zeros,
perform the FT back to K-space, and then compute their
correlation as seen earlier. This operation needs to be done
only once for M, but has to be done again for each new P,
therefore adding two FT computations to every position
Fig. 3. (a) Photograph of the mechanical setup used to obtain a
displacement along a precise 3D circle. An illustration of the circular-
motion is given in (b).
evaluation. A more efficient alternative is to resolve this
ambiguity heuristically, knowing that the bead or microrobot
cannot leap significantly between acquisitions.
C. Absolute Positioning
The previously described method only yields relative
displacements and cannot give information on the absolute
position of the device with respect to the center of the MRI.
This is due to the fact that the projection signal by itself
doesn’t have a predictable shape, since it depends on the
background signal, which would allow peak finding or other
simple algorithms to determine the center of the device.
By taking two readings of a position, one with positive
readout gradients and one with negative readout gradients,
the position of the object with regard to the center of the
MRI bore can be obtained. Indeed, the shift from the center
of the MRI bore will be reversed when the gradient is
reversed. Convoluting the projections obtained with
opposite gradients gives a maximum at a position related to
the center of the device being navigated. The convolution is
( )( )( )
A xxA xx
A xxA xd
where M+(x) is a projection mask with a magnetic element
arbitrarily placed at x0 acquired using a positive readout
gradient GR, and M-(x) being a mask of the same element, at
the same position x0, taken with the same readout gradient
magnitude but pointing in the opposite direction (-GR). A(x-
x0) is the pattern describing M+(x). The pattern describing M-
(x) will be a reflection of the pattern A around x0, which is
A(x0-x). The superscript *
C(x) will have its global maximum when the two A patterns
overlap, that is, for x=2*x0. Knowing x0, the absolute
position of the bead with respect to the center of the MRI
bore; it is then possible to use the relative positioning
method to get absolute positions by using M+ as the
correlation mask. This is true because the vector (M) has
been flipped as well as the pattern (A) it contains. The
vectors M+ and M- are also zero-padded to allow the
computation in Fourier space without the possible wrap-
around due to the implicit redundancy of the Fourier
represents the complex conjugate.
II. POSITIONING RESOLUTION
A. Experimental Setup
To evaluate the 3D positioning resolution, a mechanical
setup was fabricated. The setup was designed to create a
specific and easily distinguishable 3D motion of the
ferromagnetic bead while minimizing the motion of the
surrounding medium. The setup consists on a thin, rigid pole
holding a 1.5mm diameter ferromagnetic bead as depicted in
Fig. 3a. The resulting motion of the pole is a precise inclined
circle inducing a displacement in the three spatial axes as
shown in Fig. 3b.
The experiments were conducted in a 4 liter phantom
filled with water mixed with 20g/l gelatin, 1.25g/l nickel
sulphate, and 5g/l NaCl, providing a semi-solid medium with
shortened relaxation time for a worst case scenario. The
ferromagnetic core was made of chrome-steel sphere with a
diameter of 1.5mm (Salem Specialty Balls Company,
Canton, CT, USA). Its magnetization at 1.5T which is the
magnetic field inside the MRI system (Siemens Magnetom
Avanto 1.5T, Erlangen, Germany) is M1.5T = 1.35 × 106A/m
as measured with a VSM (Walker Scientific VSM,
Worcester, MA, USA).
The sequence parameters used were the following: RF
excitation with an offset frequency of 1.2kHz and a 30° flip
angle. The time between acquisitions was 50ms leading to a
refresh rate of 20 positions per second. The duration of the
pulse sequence was 22.46ms and since the bead was attached
to a support, no propulsion gradients were applied.
With a 5º delay
Fig. 5. (a) Positions after plane fitting and re-centering. The gray-scaled
dots indicate the distance perpendicular to the plane. (b) Simulation of the
delay between the x and y acquisitions.
Position y vs Time
Position x vs Time
020 40 6080
Position z vs Time
Fig. 4. (a) x, y and z positions versus time obtained following the sketch of
the maximum of the correlation; (b) 3D graph of the positions before
The positions along, x, y and z versus time are presented in
Fig. 4a. The corresponding circle obtained with these
positions before plane fitting is shown in Fig. 4b. We
followed the same conventional MRI axis references where
the z axis corresponds to the bore axis; the y axis is from
down to top; and the x is from left to right. In order to
accurately analyze the precision of the 3D positions obtained
with this mechanical device, a first plane-fitting step was
performed by Principal Component Analysis (PCA). With
this change of coordinates separating the one through-plane
coordinate from the two in-plane coordinates, we can find
the center of rotation more precisely than using a simple
center of mass as shown in Fig. 5a. The distance of the
center to each position was then computed and the mean
radius and standard deviation were used to assess the
precision of this positioning scheme. Note that no filtering or
outlier elimination techniques were used. The mean error
measured was 542µm. Because of the non-negligible
mechanical vibrations occurring when the setup is put in
motion, this precision is considered to constitute a
conservative lower bound to the real precision of the
The delay between the acquisition of the x, y and z
components also accounts for some of the discrepancies in
the data. Indeed, in the right side of Fig. 5a, a slight elliptic
behavior can be observed. We can also see that the points at
the right are more spread than on the left side of the figure
indicating that the manual operator turned the rod slightly
faster in the first region. The delay between the x and y
acquisitions was more significant when the bead was moving
faster. This type of behavior is consistent with delay
simulations in Fig. 5b when keeping in mind that the
coordinates u and v of Fig. 5b have been rotated by the PCA
step described earlier.
III. REAL-TIME NAVIGATION
A. Real-time Navigation Pulse Sequence
In order to be able to guide a ferromagnetic device or
microrobot either inside an MR phantom or in an in-vivo
environment, the tracking sequence must be inserted in a
Fig. 6. User interface showing our custom software projecting the position of
the bead in real-time along the pre-planned path. Numbered boxes refer to
the Siemens user interface. In (1) it is shown an off-resonance image
acquired to tune up the tracking sequence parameters. (2) shows the current
running sequence and (3), the Siemens sequence explorer interface
Fig. 7. X-ray image of the carotid artery of the pig. The position of the
release catheter and the balloon catheter are indicated by arrows. Three
waypoints were assigned to the controller. The path consists of reaching
waypoints 1, 2 and 3 and performing 10 round-trips between waypoint 2 and
real-time pulse sequence that enables real-time feedback.
Fortunately, most recent clinical MRI systems allow real-
time communication between the various modules . Such
a feedback allows the projection data to be sent to a running
process located on another image data analysis computer
containing the controller algorithm. Based on the obtained
projection data and the expected device position, an error is
computed followed by a command, namely a magnetic
gradient amplitude and direction, which is sent back to the
running sequence for execution. The running sequence is
thus only responsible for the propulsion gradient application
based on the last received computed command and for the
tracking sequence execution. A real-time feedback delay
must be considered for the command computation to be
completed and sent back to the running sequence. This
delay has been chosen to last a period of time equal to the
propulsion time plus the tracking time. This feedback delay
is set by the user at the beginning of the procedure. The
propulsion time allowed for the bead to move is then the
chosen feedback delay minus the tracking time.
B. Path Planning
Before the navigation procedure begins, the user executes
a specific vascular image generation called an angiography.
The desired path to be followed by the device is chosen
during this step. Each selected position consists of
waypoints, as depicted by Fig. 6, which the device under
computer navigation control needs to reach before a change
to the next waypoint targeted by the controller is initiated.
Once the waypoint selection is completed, it is sent to the
image analysis computer. During the procedure, the device’s
computed position is compared to the target waypoint
position and a correction command is generated by the
controller to be applied on the next propulsion phase.
C. Real-time Tracking in a Phantom
This tracking technique was successfully integrated into a
propulsion sequence and a PID controller allowing real-time
automatic navigational control of a microrobot in the blood
vessels. A Shelley vascular phantom (Shelley Industrial
Automation Inc.) placed in a container filled with water
doped with a 1.25g/l of NiSO4.6H2O was initially used for
this experiment intended to assess the technique in vitro
prior to asses it in an in vivo environment. The position of
the sphere was visualized in real-time with custom software
receiving the positions from the image calculation program
and displaying them on a 2D grid as shown in Fig. 6. An
overall feedback time of 41.6ms was used allowing 18.5ms
for propulsion with a 24Hz tracking refresh rate.
Since the tracking technique depends on the background
medium, a tuning step is required to find the best SNR .
This step consists on the acquisition of an image based on
the same imaging timing and parameters of the tracking
projections. Here, the projections that we get by running the
tracking sequence are the same than the projection of the
image along a given spatial axis. The optimal parameters are
the ones that give a dark background and a signal coming
only from the medium in the sphere’s neighborhood. The
parameters to be optimized are the offset frequency, the flip
angle and the dephaser gradients used to recover the signal
lost from magnetic field inhomogeneities.
D. Real-time Tracking In Vivo
In vivo experiments were performed with a 25kg domestic
pig. The study was approved by the animal care and use
committee of the Centre Hospitalier de l’Université de
Montréal (CHUM) as well as the committee of École
Polytechnique of Montréal (EPM). During the experiments,
the pig was under general anesthesia.
To release the ferromagnetic bead, a 6-F, 80cm introducer
catheter (Cook, Bloomington, Indiana) was used. The
catheter was inserted from the left femoral artery into the
carotid artery under X-ray standard interventional procedure.
In addition, an angioplasty balloon (5mm×18mm angioplasty
balloon - AV100, Medtronic, Santa Rosa, CA) was placed in
the distal portion of the carotid artery. Its function was to
control the blood flow in the carotid in order to easily
retrieve the sphere. Once in the MRI interventional room, a
3D angiogram with gadoteridol injection was acquired to
plan the path to be traveled. The pre-planned path consists of
reaching waypoints 1, 2 and 3 and performing 10 round-trips
between waypoint 2 and waypoint 3 as illustrated in Fig. 7.
A real-time tracking technique specifically developed to
locate a ferromagnetic core of an untethered microdevice or
microrobot guided by magnetic gradients inside blood
vessels has been described. This technique is important for
the development of platforms and interventional medical
micro/nanorobots operating in the human body including the
cardiovascular system. This paper proved for the first time
that robots with a portion made of ferromagnetic material
can be positioned and tracked by an existing medical
imaging modality allowing targeting and real-time closed-
loop navigational control inside the human body.
We acknowledge the contribution of the Magnetic
Resonance Submarine (MR-Sub) team of the NanoRobotics
Laboratory and in particular, Jean-Baptiste Mathieu, Pierre
Pouponneau, Samer Tamaz and Martin Mankiewicz.
 S. Martel, J. B. Mathieu, Y. L’Hocine, G. Beaudoin, and G. Soulez,
“Method and system for propelling and controlling displacement of a
microrobot in a blood vessel,” U.S. Patent 10/417,475, Apr. 15, 2003.
 J. B. Mathieu, S. Martel, L’H. Yahia, G. Soulez, and G. Beaudoin,
“MRI system as a mean of propulsion for a microdevice in blood
vessels,” in Proc. 25th Ann. Int. Conf. IEEE-EMB, 2003, pp. 3419-
 S. Martel, J. B. Mathieu, O. Felfoul, H. Macicior, G. Beaudoin, G.
Soulez, and Y. L’Hocine, “Adapting MRI systems to propel and guide
microdevices in the human blood circulatory system,” in Proc. 26th
Ann. Int. Conf. IEEE-EMBS, 2004, pp. 1044-1047.
 J. B. Mathieu, G. Beaudoin, and S. Martel, “Method of propulsion of
a ferromagnetic core in cardiovascular system through magnetic
gradients generated by an MRI system,” IEEE Trans. Biomed. Eng.,
vol. 53, n. 2, pp. 292-299, Feb. 2006.
 S. Martel, J.-B. Mathieu, O. Felfoul, A. Chanu, E. Aboussouan, S.
Tamaz, P. Pouponneau, G. Beaudoin, G. Soulez, L’H. Yahia, and M.
Mankiewicz, “Automatic navigation of an untethered device in the
artery of a living animal using a conventional clinical magnetic
resonance imaging system”, Applied Physics Letters, vol. 90, no. 11,
 S.-I. Takeda, Terazono, Bungo; Mishima, Fumihito; Nakagami,
Hironori; Nishijima, Shigehiro; Kaneda, Yasufumi, “Novel drug
delivery system by surface modified magnetic nanoparticles,” Journal
of Nanoscience and Nanotechnology, vol. 6, n. 9-10, pp. 3269-3276,
 F. Mishima, S. Fujimoto, S. Takeda, Y. Izumi, S. Nishijima,
“Development of control system for magnetically targeted drug
delivery,” Journal of Magnetism and Magnetic Materials, vol. 310,
n.2, March 2007, pp. 2883-2885.
 F. Mishima, S. Takeda; Y. Izumi, S. Nishijima, “Three dimensional
motion control system of ferromagnetic particles for magnetically
targeted drug delivery systems,” IEEE Transactions on Applied
Superconductivity, vol. 16, n.2, June 2006, pp. 1539-1542.
 Z. G. Forbes, B. B. Yellen, K. A. Barbee, and G. Friedman, “An
approach to targeted drug delivery based on uniform magnetic fields,”
IEEE Transactions on Magnetics, vol. 39, n. 5, Sept. 2003, pp. 3372-
 E. J. Furlani, E. P. Furlani, “A model for predicting magnetic
targeting of multifunctional particles in the microvasculature,”
Journal of Magnetism and Magnetic Materials, vol. 312, n. 1, May
2007, p 187-193.
 L. E. Udrea, J. C. N. Strachan, V. Badescu, O. Rotariu, “An in vitro
study of magnetic particle targeting in small blood vessels,” Physics
in Medicine and Biology, vol. 51, n. 19, pp. 4869-4881, Oct. 2006.
 E. P. Furlani, K. C. Ng, “Analytical model of magnetic nanoparticle
transport and capture in the microvasculature,” in Physical Review E,
v 73, n 6, June 2006, p 61919-1-10
 B. Gleich, N. Hellwig, H. Bridell, R. Jurgons, C. Seliger, C. Alexiou,
B. Wolf, T. Weyh, “Design and evaluation of magnetic fields for
nanoparticle drug targeting in cancer,” in IEEE Transactions on
Nanotechnology, vol. 6, n. 2, March 2007, p 164-70
 K. B. Yesin, K. Vollmers, J. B. Nelson, “Guidance of magnetic
intraocular microrobots by active defocused tracking,” in IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS),
vol. 4, pp. 3309-3314, 2004.
 O. Felfoul, J.B Mathieu, G. Beaudoin and S. Martel, “In Vivo MR
Tracking Based on Magnetic Signature Selective Excitation”, IEEE
Trans. Med. Imag. vol. 27, no. 1, pp28-35, Jan. 2008.
 J-H. Seppenwoolde, M. A. Viergever, and C. J. G. Bakker, “Passive
tracking exploiting local signal conservation: The white marker
phenomenon,” Magnetic Resonance in Medicine, vol. 50, no. 4,
pp784-790, Oct. 2003.
 E. Aboussouan, and S. Martel, “High-precision absolute positioning
of medical instruments in MRI systems,” in Proc. 28th Ann. Int. Conf.
IEEE-EMB, pp. 743-746. 2006.
 A. Chanu, O. Felfoul, G. Beaudoin, and S. Martel, “Adapting clinical
MRI software environment for the real-time navigation of
endovascular untethered ferromagnetic devices,” in Magnetic
Resonance in medicine, vol. 59, no. 6, pp1287-97, June. 2008.