User Centric Device Registration for Streamlined Workflows in Surgical Navigation Systems

Abstract

Alongside sweeping transformations in healthcare, a timeless drive to make surgical interventions less invasive and more effective has led to the integration of disparate technologies into surgical navigation systems. Fusions of device tracking and medical imaging modalities have been comprehensively investigated for opportunities to improve care. Such composite systems provide more and better information, enabling clinicians to operate less invasively and more effectively. Because of these merits, the preoperative ritual of harmonizing multiple information sources has been tacitly adopted.

In this paper, we challenge the paradigm of preoperative registration. Proposed herein is a technique in which a clinician registers an interventional device to a navigation system simply by gesturing the device through a strategically designed fixture. In the background, the system continuously monitors the device path for this registration gesture. We demonstrate generality by applying the method to both robotic and electromagnetically tracked devices, and exhibit versatility by repeating the registration at multiple device base locations. Experiments indicate sub-millimeter accuracy versus conventional approaches on the same setup. Consequently, clinicians can register devices on the fly, increasing flexibility in setup and redefining workflow possibilities in surgery.

Full text

User Centric Device Registration for Streamlined Workflows in
Surgical Navigation Systems
Paul Thienphrapa, Prasad Vagdargi, Alvin Chen, and Douglas Stanton
Abstract— Alongside sweeping transformations in healthcare,
a timeless drive to make surgical interventions less invasive and
more effective has led to the integration of disparate technolo-
gies into surgical navigation systems. Fusions of device tracking
and medical imaging modalities have been comprehensively
investigated for opportunities to improve care. Such composite
systems provide more and better information, enabling clini-
cians to operate less invasively and more effectively. Because of
these merits, the preoperative ritual of harmonizing multiple
information sources has been tacitly adopted.
In this paper, we challenge the paradigm of preoperative
registration. Proposed herein is a technique in which a clinician
registers an interventional device to a navigation system simply
by gesturing the device through a strategically designed fixture.
In the background, the system continuously monitors the device
path for this registration gesture. We demonstrate generality
by applying the method to both robotic and electromagnetically
tracked devices, and exhibit versatility by repeating the regis-
tration at multiple device base locations. Experiments indicate
sub-millimeter accuracy versus conventional approaches on the
same setup. Consequently, clinicians can register devices on
the fly, increasing flexibility in setup and redefining workflow
possibilities in surgery.
I. INTRODUCTION
Surgical and interventional systems are in the midst of
a revolution. Healthcare is witnessing a gradual transition
from traditional freehand procedures to smart guidance and
robotics. This progression can be found in the history of
bronchoscopy [1], which was invented in its modern form in
1964 [2] to enable inspection and intervention in the airways.
In 1992, endobronchial ultrasound (EBUS) was incorporated
to visualize tissue beyond the airways [3]. Electromagnetic
(EM) navigation was subsequently integrated to improve
localization of targets with respect to preoperative CT; the
year 2006 played host to the first human trials [4]. By 2018,
robotic bronchoscopy had been performed in a human [5]
using the Monarch Platform [6].
Increasing capability begets complexity. In the example
above, clinicians learned how to interpret ultrasound images
in the EBUS era, register devices to CT in the EM navigation
era, and register robots to anatomical roadmaps in contem-
porary times. These burgeoning technical skills rest atop
specialized medical training, so growing complexity, while
well intentioned, can induce workflow inefficiencies and even
This work was funded internally by Philips Research North America.
Paul Thienphrapa, Alvin Chen, Douglas Stanton are with Philips Research
North America, Cambridge, MA 02141, USA
Prasad Vagdargi is with the Laboratory for Computational Sensing and
Robotics, Johns Hopkins University, Baltimore, MD 21218, USA
occasional errors. For the remainder of this section, we take
a step back in order to understand the broader influences
that, despite these challenges, tend to increase complexity in
surgical navigation.
A. Clinical Trends
Value-based healthcare [7] persists despite challenges in its
implementation, fueled by economic pressure on patients and
providers. Improved outcomes, patient satisfaction, and oper-
ational efficiency have thus become focal points. A byproduct
of these movements is the growth of minimally invasive
approaches to surgical interventions; increasingly skilled
and complex procedures have been enabled and improved
by modernization. Efforts have thus been directed towards
incorporating robotics, optical tracking, EM tracking, and
image fusion, among others, into integrated solutions. GE
Innova IGS 530, Philips EchoNavigator, and Siemens Artis
zeego comprise a small cross section of solutions emerging in
response to needs in interventional cardiology. Meanwhile,
initiatives such as the US Food and Drug Administration
(FDA) workshop on surgical robotics [8], standardization of
medical robot autonomy [9], and FDA approval of artificial
intelligence to detect eye disease [10] are refining the regu-
latory pathway for continued innovation.
B. Technological Trends
EM and optical tracking systems are mature technologies
in ubiquitous use, culminating in their integration into com-
mercial navigation systems such as (respectively) Medtronic
superDimension for bronchoscopy and Stryker NAV3 for
orthopedics. Universal Robots, KUKA, and a host of other
robotics companies now offer platforms for non-industrial
and, increasingly, medical integration. Auris Health exem-
plifies this development—its Monarch Platform comprises
two customized manipulators from Kinova [11]. A growing
collection of open source toolkits allows one to draw from
an extensive selection of technologies and compose sophis-
ticated prototypes. The prospects of surgical navigation are
further enhanced by artificial intelligence, which for surgery
is a nascent specialty promising to augment perception and
cognition [12].
C. Commercial Trends
Activity in the private sector has had an indirect yet pal-
pable impact on the evolution of surgical navigation. For ex-
ample, in the context of its medical imaging systems, Philips
acquired Volcano and its intravascular ultrasound catheters,
reflecting a broader phenomenon of combining information

and treatment capabilities. Later in 2015, Ethicon and Verily
formed Verb Surgical to collaborate on digital innovations
in surgery; Ethicon would solidify its robotics presence by
acquiring Auris Health in early 2019. In 2016, Medtronic
partnered with (and later acquired) robotics firm Mazor to
forge a new solution for spinal implants. Abbot acquired
like-entity St. Jude Medical in a $25B USD transaction, a
fraction of the $332B USD in healthcare mergers of 2017
[13], populated mainly by shifts in hospital and insurance
enterprises. In 2018, Amazon, J.P. Morgan, and Berkshire
Hathaway introduced themselves into the industry through a
joint healthcare venture, headlining a set of non-traditional,
high profile alliances that would follow. While the circum-
stances surrounding these events vary, the tendency towards
consolidation foretells increasing integration of capabilities
that were previously decoupled.
D. Broader Trends
The commercial success of Intuitive Surgical’s da Vinci
Surgical System has had a notable influence on the adoption
of advanced surgical technologies. More recently, consumer
products featuring smart voice control and connectivity have
spurred breakthroughs in overall technological engagement.
With social momentum stemming from digitization and per-
sonalization, what was once a willingness to adopt innovation
is accelerating into an expectation.
E. Integration
Evolution in surgical systems can be understood as a
confluence of these trends, and a theme that threads them
all is integration. Advantageous integration would be those
that support key clinical needs: improved outcomes, pa-
tient satisfaction, and operational efficiency. In the context
of surgical navigation, systems should be configurable to
accommodate variances in patient presentation, yet simple
to configure for any of these variances. Registration for
surgical navigation has been thoroughly investigated, based
on practices of times past. Given emerging trends, however,
the convention warrants reexamination of what was a known
art a generation ago. In the following section, we provide
some background on existing registration approaches before
detailing our proposed method.
II. BACKGROU ND
Clinicians A and B, users of a navigation system, agree
to perform a registration task. Clinician A activates a special
mode in the software. Clinician B, using the tracked device,
locates the first landmark in the surgical space. She touches
the device to the landmark, and signals as much to Clinician
A. Manning the computer console, Clinician A enters this
event into the software before notifying Clinician B of the
same. The process repeats itself in this manner until all three
or more ordered landmarks have been visited. The software
computes the spatial transformation between the device and
a virtual workspace. The intervention can proceed unless—
or until—the calibration is perturbed, whereupon Clinicians
A and B agree to start over.
This canonical registration scenario has recycled itself
through many research labs, where the ample technical ex-
pertise can help mitigate the minutiae. After all, the benefits
of navigation should justify these cognitive and operational
overheads. In clinical settings, however, the sequence may be
experienced as arcane. Registration is indeed widely regarded
as cumbersome, particularly with artificial fiducials [14].
The rationalization of benefits continues to falter amidst
trade terms such as Flash Registration from 7D Surgical
and Universal Automatic Image Registration from Brainlab
promising to alleviate the technical pain. The latter also
offers Z-touch [15], comparable to the Medtronic Fazer [16]:
the clinician scans the anatomy with a laser, and the digitized
reflections are matched to a skull model.
Early stage investigations have likewise acknowledged
the prohibitive nature of registration. Automatic registration
was proposed for neurosurgery in [17]. Bimodal fiducials
were attached to the patient’s skull prior to cone beam CT
imaging. Intraoperatively, the fiducials were localized in both
the image and with an optical tracker, allowing the image
to maintain alignment with the moving anatomy. Ref. [18]
presented an automatic method for bronchoscopy. Airways
were segmented from a lung CT image and abstracted into a
tree representation. An EM tracked bronchoscope was then
navigated blindly down the trachea. Registration between
the bronchoscope position and CT image was refined based
on its likeliest position on the bronchial map. Historically,
calibration activities in robotic surgery have centered around
improving accuracy in neurosurgery [19]–[21], orthopedics
[22]–[24], and laparoscopy [25]–[27]; in the latter case, [28],
[29] progress towards automated techniques.
III. MET HODS
A. Contributions
Technologies emerge, circumstances evolve, and research
efforts advance, yet one element continues to elude progress
in navigation systems: registration of tracked devices. We
herein propose a streamlined strategy for registering devices
in surgical navigation systems. An operator passes the device
through a special fixture. A background monitor detects this
event, and registration is complete.
We submit, universally, that clinical users can be liberated
from the technicalities of registration. Given explicit initia-
tion of the task through a single simple gesture, clinicians
maintain cognizance over the workflow. Our approach is
embedded into the clinical setup and is carried out determin-
istically, as compared to random scanning using a separate
laser [15], [16]. We work towards a solution that is free of
patient-mounted fiducials, such as the bimodal fiducials [17]
described above whose placement is a substantial workflow
disruption. Foregoing the uncertainties of an iterative reg-
istration [18], our approach overcomes the aforementioned
drawbacks of prevailing registration paradigms. Detailed
below is a new blueprint for registration, an intuitive interface
to complex systems operating in clinical environments.

Fig. 1. Example registration fixture containing a special path for the device
to traverse, suitable for a flexible EM tracked catheter.
Device
Fixture
Workspace
w
T
f
(known)
Fig. 2. Coordinate transforms in the proposed technique. The workspace
may refer to the patient (not pictured), provided the fixture is anatomically
attached. Alternatively, the transform between the workspace and the patient
can be obtained intraoperatively using established methods.
B. Overview
To the clinician, registration using the proposed technique
is achieved by inserting the tracked device through a fixture
(Fig. 1) mounted in the workspace (e.g., patient table). Fig. 2
illustrates the underlying coordinate transforms:
•wTf: Fixture frame fin workspace wis by design;
•fTd: Device frame drelative to the fixture emerges once
the monitor detects the known path in the fixture; and
•wTd=wTf×fTdis the device-to-workspace registration,
immediately computed upon fixture traversal.
The supporting system process is captured in Fig. 3. As the
device traverses the fixture, a background monitor accumu-
lates tracked positions, searching for a traversed path that
matches the expected fixture path. By crafting a fixture and
mounting it in the workspace in a predetermined manner, we
embed the technical burden of registration into the system
design and away from end users.
Recall fixture path in
workspace coordinates
(
w
T
f
)
Compute
w
T
d
=
w
T
f
×
f
T
d
Registration complete
Tracked device guided
through fixture
Fixture path
detected?
No
Yes
Determine sensed
device path in fixture
coordinates (
f
T
d
)
Fig. 3. Underlying process for the streamlined registration technique.
Device position is continuously monitored, and registration is triggered only
when the known path is detected.
C. Tracking the Patient Anatomy
A navigation system may accommodate incidental patient
motion by registering and tracking the pertinent anatomy. An
example is in a cardiac catheterization lab, where the patient
table is encoded with respect to the C-arm. Fluoroscopy,
rather than fiducials, can then be used to track the patient
[30], [31] so that the transform pTwfrom the workspace (ta-
ble) to patient frame pis known intraoperatively. Attaching
the fixture of Fig. 1 to the table then permits the device to
be registered to the patient as pTd=pTw×wTd, where wTd
is computed as above.
This represents a clinical scenario in which the assumption
of a known fixture-to-workspace relationship wTf(see Fig. 2)
is adequate, so we defer consideration of the patient frame.
Less preferably, mounting the fixture onto the patient equates
the patient with the workspace (pTw=I, so pTd=wTd). In
yet other setups, optical tracking can substitute for the role
that fluoroscopy plays above. In other words, we can account
for patient motion through established means; so for the sake
of clarity, we generalize our definition of workspace—it may
equivalently refer to the table or patient frame.
D. Path Monitoring
The utility of our registration method thus hinges upon the
device-to-fixture transformation fTd. As the clinician manip-
ulates the tracked device, a background process (Fig. 3) scans
the trajectories for the fixture path. Once detected, the path is
transformed into the fixture frame. This workflow allows the
clinician to perform registration on demand and away from
the console. The path should be distinct from incidental use
to avoid inadvertent triggering; the system may also offer
convenient means of confirming user intent.
In practice, the fixture path search space can be limited to
the most recently acquired N=fs×tf+∆tf+tdposition
samples, where fsis the sampling rate, tfis the time for the
device to transit the registration fixture, ∆tfaccommodates
variabilities in that estimate, and tdis an extra delay between
the travel and the result. These quantities can be tuned to the
use case. For our fixture (Fig. 4, top), we provision tf=2,
∆tf=1, and td=1 seconds. A sampling rate fs=30 Hz
then yields N=120 samples.
Given a set of device positions P=pi∈R3|1≤i≤N,
the problem is to determine whether some Q⊆Pmatches a
model, and if so, find the rigid transformation fTdmapping
Qin device space to fixture model space. For this problem, a
multitude of solutions is available in the literature. Should the
model also be represented as a set of points, methods such
as iterative closest point (ICP) [32], its advancements (e.g.,
[33]–[35]), coherent point drift [36], and cross correlation
may be used. Alternatively, the model may be designed as a
function to which Qis fit using a polynomial, B-splines [37],
NURBS [38], Levenberg-Marquardt [39], etc. The present
framework is agnostic to the algorithms used; such decisions
would ultimately be driven by requirements. In that regard,
we exploit properties of our design to reduce the problem to
a sequence of intermediate calculations:

-50 50
x
y
Fixture Model: Frame fT
-40 -20 0 20 40 60 80
x (mm)
-40
-30
-20
-10
0
10
20
30
40
z (mm)
Device Path through Fixture: Frame
dT
Device Samples
Spline
20 40 60 80 100 120 140 160
Sample #
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
Curvature (mm-1)
Cross Correlation - Fixture vs. Device Path Curvature
Fixture
Device
-360
-340
-320
-160
Device Path Registered to Fixture Model: fTd
80
z
-140 100
-120 x
y120
-100 140
-80 160
-60
Model
Device Samples
Fig. 4. Path monitoring approach for computing device-to-fixture trans-
formation fTd. (Top) Fixture path model, an asymmetric, planar curve with
coordinate frame fTdefined using PCA. (Mid-left) Position samples through
the fixture with spline smoothing. (Mid-right) Cross correlation is performed
between the curvature of the model and device spline to find Q, upon which
PCA is used to compute device path frame dT. (Bottom) Result of applying
fTd, approximated from fT×dT−1and refined using ICP.
1) Generate a sequence Rby spline smoothing Pand
resampling with equidistant spacing (Fig. 4, mid-left)
2) Compute curvature κi=kri+1−2ri+ri−1k∀ri∈R
3) Perform cross correlation between the κiand its model
counterpart to identify pertinent path points Q⊆P
(Fig. 4, mid-right)
Reducing Pto a 1D signal potentially introduces ambiguities,
but these can be overcome (see Sec. IV-D) to enable immedi-
ate computation on modest hardware. Provided a correlation
is found, the algorithm continues:
4) a. Retrieve fT, the local frame of the fixture model
b. Compute dTfrom the principal components of Q
5) Approximate the transform as fTd=fT×dT−1and
then refine using ICP (Fig. 4, bottom)
While device position is continuously monitored, the result
is applied only if the fiducial registration error (FRE) is low
enough to suggest that the clinician has indeed executed the
registration task. Otherwise, the result is passively ignored.
An alternative to the local fixture frames of Step 4 could
be predefined curve segments detected as discrete features
for landmark registration. Indeed, the design possibilities
are innumerable, as discussed in Sec. IV-D. For the present
study, we favored a design that reduced complexity in both
mechanical and computational terms, facilitating ease of use
and real-time processing.
E. Impact
The simplicity of the proposed method belies a broader
shift from a system centric to a clinician focused mindset,
highlighting opportunities for intuitive, non-disruptive inter-
faces for workflow simplicity amidst increasing complexity
overall. Within a minimal footprint, it can help clinicians
register devices more consistently and re-register more con-
veniently (e.g., when an EM field generator is displaced).
Enabling greater versatility in procedure setup, it can enhance
the realized efficacy, and eventual permeation, of integrated
solutions.
The straightforward principles of our approach make it
broadly applicable to a variety of localization systems and
robotics, while being conducive to product development and
maintenance. The concepts can be engineered to specific
clinical procedures, workflows, and constraints. In the exam-
ple above, the fixture path is designed as a smooth channel
that accepts passage of flexible, low profile devices such as
catheters used in electrophysiology, transcatheter structural
heart procedures, vascular interventions, and lung biopsies.
For rigid devices such as robotic end effectors, the path may
be a groove instead of a blind channel, and angular rather
than smooth. Notably, the method can be used without any
device alterations; it simplifies registration without disrupting
other aspects of the workflow.
IV. EXP ERI MEN TS
A. Registering a Robotic Device
The principle is first demonstrated on registration of a
robotic manipulator to a workspace, a scenario applicable to
bedside laparoscopic assistants such as [40]. For quantitative
evaluation, we construct a phantom workspace with five post
targets spanning a 60-mm cube, as shown in Fig. 5 (top-
left). This volume approximates surgical regions of interest
such as the heart, prostate, and local anatomy peripheral to
a lung tumor. The workspace coordinate system is defined
per Fig. 5 (bottom-left). A 3D printout of the fixture path of
Fig. 4 (top) is attached in a known configuration, 100 mm
away from the posts. A Universal Robots UR5 arm is secured
at an arbitrary position, and its end effector, a rigid needle
used in orthopedic surgery, is backdriven through the fixture.
Once completed, the path becomes known in both robot and
workspace frames, thus registering them together.
Simulating versatility in making intraoperative setup ad-
justments, the registration is repeated with the robot base at
three different locations with respect to the workspace, as
labeled in Fig. 5 (right). In each trial, the robot visits each
target and corresponding measurements are compared against
the ground truth. The FRE using the path is found to be
1.7 mm, while the target registration error (TRE) is 0.3 mm.
For reference, conventional landmark registration using these
targets as fiducials leads to an FRE of 0.1 mm. The results are
plotted together in Fig. 5 (bottom-left). Interestingly, the FRE
in the proposed approach is higher than with the landmark
method, yet the TRE remains nearly perfect. Beyond yielding
accurate performance with a simplified workflow, this result
demonstrates the robustness of path registration to errant or
noisy data points—the continuum of samples along the path
effectively serves as a series of fiducials.

tŽƌŬƐƉĂĐĞ dĂƌŐĞƚƐ
&ŝdžƚƵƌĞ
WĂƚŚ
^ƵƌŐŝĐĂů
dŽŽů
hZϱ
WŽƐ͘ϭ
WŽƐ͘ϯ
Errors (mm RMS) Pos.1 Pos.2 Pos.3 Avg.
FRE-Reference 0.1 0.1 0.2 0.1
FRE 1.7 1.8 1.6 1.7
TRE 0.2 0.3 0.3 0.3
Fig. 5. Registering a robotic device to a workspace. (Top-left) CAD model
of a phantom workspace showing fixture path and ground truth targets.
(Right) Experimental setup with a UR5 robot. Intraoperative adjustments
to robot base are simulated by re-registering at the positions labeled in
blue. Measured RMS errors are also tabulated. (Bottom-left) Plot of the
registration result; robot path (hollow blue circles) overlays the model fixture
(yellow line), while robot target positions (filled orange circles) overlay
ground truth targets (blue posts). Workspace axes are also labeled.
B. Registering an EM Tracked Device
We next demonstrate the ability of the streamlined method
to register a flexible EM tracked device such as a catheter,
an instrument used in a variety of clinical interventions as
previously noted. An EM field generator is stationed at an
arbitrary position with respect to the workspace phantom
described above, and the device is guided through the fix-
ture. Once traversal is complete, the EM tracking system
becomes registered to the workspace because the path is
simultaneously known in both coordinate frames. Simulating
both inter- and intraoperative adjustments of the EM tracking
system, the procedure is repeated with the field generator
at four different locations, as depicted in Fig. 6. The FRE
using the path is found to be 1.8 mm, and the TRE ensuing
from touching the device tip to the targets is 2.9 mm. For
comparison, conventional landmark registration using the
targets as fiducials yields an FRE of 2.0 mm.
The FRE is comparable between methods, but the TRE
will likely be lower in the conventional case. Whereas fidu-
cials and targets are typically conceived within each other’s
proximity, the path fiducial in the present setup is located
remotely from the targets. Placing the target anatomy closer
to the path may mitigate the influence of EM field distortions
that otherwise plague larger workspaces. This provision may
then be relaxed under navigation systems with less restrictive
configurations, as evidenced by the robotic results above.
Despite stringent EM working volumes, the proposed ap-
proach achieves sufficient accuracy for most cases (within a
millimeter of conventional methods) while providing clinical
users with benefits of simplicity and versatility.
C. Discussion
A tantalizing exercise left for future work is the coordina-
tion of both the robot and catheter—that is, multiple tracked
tŽƌŬƐƉĂĐĞ
dĂƌŐĞƚƐ
&ŝdžƚƵƌĞWĂƚŚ
D^ĞŶƐŽƌ
D&ŝĞůĚ
'ĞŶĞƌĂƚŽƌ
WŽƐ͘Ϯ
Errors (mm RMS) Pos.1 Pos.2 Pos.3 Pos.4 Avg.
FRE-Reference 2.4 2.1 1.8 1.7 2.0
FRE 2.0 1.5 2.4 1.1 1.8
TRE 2.8 3.0 2.2 3.6 2.9
Fig. 6. Setup to register an EM tracked device (e.g., a catheter) with a field
generator next to a workspace. Intraoperative field generator displacement
is simulated by re-registering the device at multiple positions indicated in
blue. Measured RMS errors are also tabulated.
devices—into the same system. Not only would the benefit
of an efficient workflow scale with complexity, the nature of
the workflow may be redefined altogether. Device registration
can be deferred to the moment of use, reducing the burden
of registering beforehand. Registration of navigated devices
to only each other can be obtained using a fixture that is
decoupled from any particular workspace. Static preoperative
arrangements give way to dynamic intraoperative adjust-
ments: the repeatability of our approach is suggested by the
standard deviations of 0.1 mm and 0.6 mm in the accuracy
of the robotic and EM results respectively; direct validation
of repeatability is slated for future work.
While the S-shaped path used in this study suffices for
proof of feasibility, we note that the choice is not necessarily
optimal. For example, the user may favor driving the robot
through piecewise straight segments rather than curved ones,
and in the case of catheters and guidewires, device stiffness
may warrant retooling of curves. Meanwhile, intricate paths
may be favored for computational robustness. Indeed, the
concept of optimality may be intractable due to an expansive
design space; the following section initiates a discussion on
these considerations. Nevertheless, thematically consistent
considerations including usability, implementation simplicity,
and the psychophysics of perception, cognition, and judg-
ment may serve as valuable guiding principles.
D. Design Considerations
Given the breadth of alternative fixture embodiments, a
selection thereof is noted here for future reference. In terms
of path complexity, usability may favor simpler paths that
are shorter and straighter, but this may result in computa-
tional ambiguities that compromise the reliability of gesture
detection (to be examined explicitly should any issues arise).
So while complexity may be increased in response, residual
inaccuracies can be resolved by tiering other capabilities in-
stead. For example, intraoperative fluoroscopy or ultrasound
can be used for image-based corrections; registration can thus
remain simultaneously effective and user friendly even when
individual methods are imperfect.

Path traversal is presently described as a manual process,
but other approaches may better fit different scenarios. For
instance, the fixture may include active elements that engage
and assist the device through the path. For flexible devices, a
technique for pulling rather than pushing may be preferred,
while a robot may guide itself under vision or force control
when it is otherwise lost; this may hold particular promise for
surgical microrobots [41]–[43]. The path itself may consist
of discontinuous segments. For any given configuration, the
computational problem can be constructed for efficiency and
robustness, as mentioned in Sec. III-D.
Beyond the intrinsic design of the registration fixture itself,
decisions on where and how it is mounted in the workspace
can have profound effects on usability and functionality. For
example, application particulars may determine the suitabil-
ity of different mounting options, which may be the patient
table, table rail, or patient anatomy. Specific requirements
would further determine the feasibility of fixture sterilization,
reuse, and integration as aftermarket addenda to existing
systems. For facilities needing large variances in workspace
coverage, multiple fixtures may be installed for improved
accuracy and workflow.
V. CONCLUSIONS
Relating to a common experience, automotive GPS works
without a pre-drive calibration—even though it would ar-
guably be a worthwhile exchange for the benefits of vehicular
navigation. Intuition suggests, however, that this hypothetical
prerequisite would be distracting. Workflow disruption is
inefficient: at best a nuisance, at worst a cognitive tax. “In the
economy of action, effort is a cost” [44] leading to critical
errors in driving [45], a fairly common task. The analogous
phenomenon in surgical interventions—mentally demanding
missions executed under pressure—is that motor skills and
judgment are both compromised [44], [46].
In building the technicalities of registration into the sys-
tem design upfront, we have developed a scheme that is
streamlined for clinical users. We reason that user experi-
ence will grow in importance as discrete technologies are
increasingly integrated to address clinical needs. This paper
illustrates the approach with a robot and a flexible manual
instrument, two instances of an overarching effort to make
complex systems intuitive. Until devices can auto-register,
we advocate for the elevation of users and workflows to be-
yond an afterthought, whether through standard platforms or
protocols. The psychophysics of perception, cognition, and
judgment can inform the design of navigation interfaces and
workflows which may, as a side effect, alleviate subconscious
reservations to innovation as well.
ACKNOWLEDGMENT
The authors thank Bharat Ramachandran, Aryeh Reinstein,
and Frank Keegan for critical feedback on concepts, Jochen
Kruecker for assistance with the EM tracking system, and
Sean Kyne for integration of the robotic instruments. We also
thank our colleagues at Philips Research for their support.
REFERENCES
[1] T. S. Panchabhai and A. C. Mehta, “Historical perspectives of bron-
choscopy: Connecting the dots,” Annals of the American Thoracic
Society, vol. 12, no. 5, pp. 631–641, 2015.
[2] T. Miyazawa, “History of the flexible bronchoscope,” in Interventional
Bronchoscopy, ser. Progress in Respiratory Research, C. Bolliger and
P. Mathur, Eds. S. Karger AG, Oct. 2000, vol. 30, pp. 16–21.
[3] T. H¨
urter and P. Hanrath, “Endobronchial sonography: Feasibility and
preliminary results,” Thorax, vol. 47, no. 7, pp. 565–7, July 1992.
[4] Y. Schwarz, J. Greif, H. D. Becker, A. Ernst, and A. Mehta, “Real-time
electromagnetic navigation bronchoscopy to peripheral lung lesions
using overlaid CT images: The first human study,” Chest, vol. 129,
no. 4, pp. 988–994, Apr. 2006.
[5] “El Camino Hospital pioneers first US use of robotic platform for lung
cancer diagnosis,” Mountain View, CA, Apr. 2018. [Online]. Available:
https://www.businesswire.com/news/home/20180412005983/en/El-
Camino-Hospital-Pioneers-U.S.-Robotic-Platform
[6] J. B. Alvarez, G. J. Kintz, D. S. Mintz, E. Romo, J. Zhang, S. Bu-
rion, S. Wong, J. D. Bogusky, K. A. Bender, and A. Yu, “System
for robotic-assisted endolumenal surgery and related methods,” U.S.
Patent 9,763,741, Sept. 19, 2017.
[7] M. E. Porter and E. O. Teisberg, Redefining Health Care: Creating
Value-based Competition on Results. Harvard Business Press, 2006.
[8] “Robotically-assisted surgical devices: Challenges and opportunities;
public workshop; request for comments,” Federal Register, vol. 80,
no. 37, pp. 10 124–10 125, 2015.
[9] G.-Z. Yang, J. Cambias, K. Cleary, E. Daimler, J. Drake, P. E.
Dupont, N. Hata, P. Kazanzides, S. Martel, R. V. Patel, V. J. Santos,
and R. H. Taylor, “Medical robotics—regulatory, ethical, and legal
considerations for increasing levels of autonomy,” Science Robotics,
vol. 2, no. 4, p. eaam8638, Mar. 2017.
[10] “FDA permits marketing of artificial intelligence-based device
to detect certain diabetes-related eye problems,” Silver Spring,
MD, Apr. 2018. [Online]. Available: https://www.fda.gov/newsevents/
newsroom/pressannouncements/ucm604357.htm
[11] Monarch Platform User Manual. Auris Health, Inc., 2018.
[12] D. A. Hashimoto, G. Rosman, D. Rus, and O. R. Meireles, “Artificial
intelligence in surgery: Promises and perils,” Annals of Surgery, vol.
268, no. 1, pp. 70–76, Jan. 2018.
[13] Global Healthcare Private Equity and Corporate M&A Report 2018.
Bain & Company, 2018.
[14] D. P. Perrin, N. V. Vasilyev, P. Novotny, J. Stoll, R. D. Howe, P. E.
Dupont, I. S. Salgo, and P. J. del Nido, “Image guided surgical
interventions,” Current Problems in Surgery, vol. 46, no. 9, pp. 730–
766, Sept. 2009.
[15] A. Raabe, R. Krishnan, R. Wolff, E. Hermann, M. Zimmermann,
and V. Seifert, “Laser surface scanning for patient registration in
intracranial image-guided surgery,” Neurosurgery, vol. 50, no. 4, pp.
797–801; discussion 802–3, Apr. 2002.
[16] J. M. Henderson, K. R. Smith, and R. D. Bucholz, “An accurate
and ergonomic method of registration for image-guided neurosurgery,”
Computerized Medical Imaging and Graphics, vol. 18, no. 4, pp. 273–
277, July 1994.
[17] G. J. Bootsma, J. H. Siewerdsen, M. J. Daly, and D. A. Jaffray,
“Initial investigation of an automatic registration algorithm for surgical
navigation,” in IEEE Int. Conf. of the Engineering in Medicine and
Biology Society (EMBS). IEEE, Aug. 2008, pp. 3638–3642.
[18] X. Luo, “A bronchoscopic navigation system using bronchoscope
center calibration for accurate registration of electromagnetic tracker
and CT volume without markers,” Medical Physics, vol. 41, no. 6Part1,
p. 061913, May 2014.
[19] T. Haidegger, T. Xia, and P. Kazanzides, “Accuracy improvement
of a neurosurgical robot system,” in IEEE RAS/EMBS Int. Conf. on
Biomedical Robotics and Biomechatronics (BioRob). IEEE, Oct.
2008, pp. 836–841.
[20] M. Heinig, U. G. Hofmann, and A. Schlaefer, “Calibration of the
motor-assisted robotic stereotaxy system: MARS,” Int. J. Computer
Assisted Radiology and Surgery, vol. 7, no. 6, pp. 911–920, Nov. 2012.
[21] F. Vicentini, P. Magnoni, M. Giussani, and L. M. Tosatti, “Analysis and
compensation of calibration errors in a multi-robot surgical platform,”
in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS).
IEEE, Sept. 2015, pp. 3633–3640.

[22] B. Guo, H. Jin, P. Zhang, J. Zhang, Y. Hu, and H. Zhang, “Accuracy
analysis and calibration of a parallel guidance device for minimal inva-
sive spinal surgery,” in IEEE Int. Conf. on Robotics and Biomimetics
(ROBIO). IEEE, Dec. 2013, pp. 1468–1473.
[23] G. Liu, X. Yu, C. Li, G. Li, X. Zhang, and L. Li, “Space calibration
of the cranial and maxillofacial robotic system in surgery,” Computer
Assisted Surgery, vol. 21, no. sup1, pp. 54–60, Dec. 2016.
[24] Y. Ning, P. Gao, Y. Sun, X. Qi, and Y. Hu, “A practical calibration
method for spinal surgery robot,” in Int. Conf. on Advanced Robotics
(ICAR). IEEE, July 2017, pp. 137–142.
[25] F. Mourgues, ´
E. Coste-Mani´
ere, and C. Team, “Flexible calibration
of actuated stereoscopic endoscope for overlay in robot assisted
surgery,” in Int. Conf. on Medical Image Computing and Computer-
Assisted Intervention (MICCAI). Berlin, Heidelberg: Springer, Berlin,
Heidelberg, 2002, pp. 25–34.
[26] R. Shahidi and M. Epitaux, “Apparatus and method for calibrating an
endoscope,” U.S. Patent 6,511,418, Jan. 28, 2003.
[27] S. Lee, H. Lee, H. Choi, S. Jeon, and J. Hong, “Effective calibration
of an endoscope to an optical tracking system for medical augmented
reality,” Cogent Engineering, vol. 4, no. 1, July 2017.
[28] D. Stoyanov, A. Darzi, and G.-Z. Yang, “Laparoscope self-calibration
for robotic Assisted minimally invasive surgery,” in Int. Conf. on Med-
ical Image Computing and Computer-Assisted Intervention (MICCAI).
Springer, Berlin, Heidelberg, Oct. 2005, pp. 114–121.
[29] Z. Wang, Z. Liu, Q. Ma, A. Cheng, Y.-h. Liu, S. Kim, A. Deguet,
A. Reiter, P. Kazanzides, and R. H. Taylor, “Vision-based calibration of
dual RCM-based robot arms in human-robot collaborative minimally
invasive surgery,” IEEE Robotics and Automation Letters, vol. 3, no. 2,
pp. 672–679, Apr. 2018.
[30] H. Shirato, S. Shimizu, K. Kitamura, T. Nishioka, K. Kagei,
S. Hashimoto, H. Aoyama, T. Kunieda, N. Shinohara, H. Dosaka-
Akita, and K. Miyasaka, “Four-dimensional treatment planning and
fluoroscopic real-time tumor tracking radiotherapy for moving tumor,”
Int. J. Radiation Oncology*Biology*Physics, vol. 48, no. 2, pp. 435–
442, Sept. 2000.
[31] H. Shirato, S. Shimizu, T. Kunieda, K. Kitamura, M. van Herk,
K. Kagei, T. Nishioka, S. Hashimoto, K. Fujita, H. Aoyama,
K. Tsuchiya, K. Kudo, and K. Miyasaka, “Physical aspects of a real-
time tumor-tracking system for gated radiotherapy,” Int. J. Radiation
Oncology*Biology*Physics, vol. 48, no. 4, pp. 1187–1195, Nov. 2000.
[32] P. Besl and N. D. McKay, “A method for registration of 3-D shapes,”
IEEE Transactions on Pattern Analysis and Machine Intelligence,
vol. 14, no. 2, pp. 239–256, Feb. 1992.
[33] Z. Zhang, “Iterative point matching for registration of free-form curves
and surfaces,” Int. J. Computer Vision, vol. 13, no. 2, pp. 119–152,
Oct. 1994.
[34] A. Segal, D. Haehnel, and S. Thrun, “Generalized-ICP,” in Robotics:
Science and Systems (RSS), June 2009.
[35] S. D. Billings, E. M. Boctor, and R. H. Taylor, “Iterative Most-Likely
Point Registration (IMLP): A robust algorithm for computing optimal
shape alignment,” PLOS ONE, vol. 10, no. 3, p. e0117688, Mar. 2015.
[36] A. Myronenko and X. Song, “Point set registration: Coherent Point
Drift,” IEEE Transactions on Pattern Analysis and Machine Intelli-
gence, vol. 32, no. 12, pp. 2262–2275, Dec. 2010.
[37] I. J. Schoenberg, “Contributions to the problem of approximation
of equidistant data by analytic functions: Part A.—On the problem
of smoothing or graduation. A first class of analytic approximation
formulae,” Quarterly of Applied Mathematics, vol. 4, no. 1, pp. 45–
99, 1946.
[38] K. J. Versprille, “Computer-aided design applications of the rational b-
spline approximation form,” Ph.D. dissertation, Syracuse Univ., 1975.
[39] K. Levenberg, “A method for the solution of certain non-linear
problems in least squares,” Quarterly of Applied Mathematics, vol. 2,
no. 2, pp. 164–168, 1944.
[40] R. H. Taylor, J. Funda, B. Eldridge, S. Gomory, K. Gruben, D. LaRose,
M. Talamini, L. Kavoussi, and J. Anderson, “A telerobotic assistant
for laparoscopic surgery,” IEEE Engineering in Medicine and Biology
Magazine, vol. 14, no. 3, pp. 279–288, 1995.
[41] V. Magdanz, S. Sanchez, and O. G. Schmidt, “Development of a
sperm-flagella driven micro-bio-robot,” Advanced Materials, vol. 25,
no. 45, pp. 6581–6588, Dec. 2013.
[42] I. S. M. Khalil, V. Magdanz, S. Sanchez, O. G. Schmidt, and S. Misra,
“The control of self-propelled microjets inside a microchannel with
time-varying flow rates,” IEEE Transactions on Robotics, vol. 30,
no. 1, pp. 49–58, Feb. 2014.
[43] M. Medina-S´
anchez and O. G. Schmidt, “Medical microbots need
better imaging and control,” Nature, vol. 545, no. 7655, pp. 406–408,
May 2017.
[44] D. Kahneman, Thinking, fast and slow. New York: Farrar, Straus and
Giroux, 2011.
[45] K. Rumar, “The basic driver error: late detection,” Ergonomics, vol. 33,
no. 10-11, pp. 1281–1290, Oct. 1990.
[46] C. M. Carswell, D. Clarke, and W. B. Seales, “Assessing mental
workload during laparoscopic surgery,” Surgical Innovation, vol. 12,
no. 1, pp. 80–90, Mar. 2005.