Conference PaperPDF Available

Designing a 3D Input Device For Interventional Radiology

Authors:

Abstract and Figures

We describe here the design of a computer input device and its software that allows the interventional radiologist to handle the 3D images from the surgical table to perform easier and safer treatments under X-Ray guidance (the “InRoom3D” project). This table-side box with its 6 Degree-Of-Freedom (DOF) handle and its mini-keyboard allow the radiologists to perform the 3D image manipulation and the other related tasks such as distance measurement and function selection. The originality of the device is in the adaptation to the surgical/interventional environment and the integration of the classic point-andclick mouse function. Experiments carried out thus far have validated the technology for precise 3D orientation and usability evaluations have shown good user acceptance of the product. Besides interventional radiology, the user interface answers to the generic need for a 3D/2D computer input device compatible with the surgical environment.
Content may be subject to copyright.
1
Designing a 3D Input Device For Interventional Radiology
Pascal Salazar
pascal.salazar@med.ge.com Jean-Michel Marteau
jean-michel.marteau@med.ge.com
GE Healthcare – Global Industrial Design Department
283, rue de la Minière – 78533 Buc cedex - France
ABSTRACT
We describe here the design of a computer input device and
its software that allows the interventional radiologist to
handle the 3D images from the surgical table to perform
easier and safer treatments under X-Ray guidance (the
“InRoom3D” project). This table-side box with its 6
Degree-Of-Freedom (DOF) handle and its mini-keyboard
allow the radiologists to perform the 3D image
manipulation and the other related tasks such as distance
measurement and function selection. The originality of the
device is in the adaptation to the surgical/interventional
environment and the integration of the classic point-and-
click mouse function. Experiments carried out thus far have
validated the technology for precise 3D orientation and
usability evaluations have shown good user acceptance of
the product. Besides interventional radiology, the user
interface answers to the generic need for a 3D/2D computer
input device compatible with the surgical environment.
Author Keywords
6 DOF control, 3D interface, Virtual environment,
Rotation, Medical, Radiology, Usability study.
ACM Classification Keywords
H5.2 [Information Interfaces And Management]: User
Interfaces---Input devices and strategies, Ergonomics,
Prototyping. J.3 [Life and medical sciences]: Medical
information systems.
INTRODUCTION
The users and the working environment
Interventional radiology is a set of minimally invasive
diagnostic and therapeutic techniques in which the
radiologists guide small devices such as catheters through
the blood vessels to treat vascular diseases. While live 2D
X-Ray (fluoroscopy) has been the main way to visualize the
blood vessels during the clinical procedure, the 3D
visualization of the vascular anatomy is widely recognized
as an invaluable tool in interventional radiology. It helps the
physicians to determine both the optimal course of patient
therapy and the proper implementation of the chosen course
of action. The interventional radiologists can visualize the
3D anatomy on a desktop review workstation in a control
room adjacent to the catheterization lab (also called cath.
lab). However, using the 3D visualization would be much
more efficient if the radiologists could directly display and
manipulate the 3D images without leaving the cath. lab and
interrupting the clinical procedure. This is precisely the
main goal of the “InRoom3D” project as it integrates for
this purpose both the input device and its application.
The design challenges
The first challenge of the project is to design a generic input
device allowing the user to perform smooth, precise and
quick 3D orientations of the anatomy visible in a in-room
monitor (see figure 1 – top right). In addition to 3D
rotations, other movements such as integrated panning and
zooming are needed. In using a fast switch, the radiologist
will be able to move a 2D cursor and to click on on-screen
buttons and menus.
Figure 1. Interventional Radiologist reviewing 3D images (top
right) in the cath. lab. The user handles the 3D Box input
device under the sterile drape (bottom left).
The second challenge is to adapt the input device and the
application to the difficult conditions of the cath. Lab, i.e.
2
the user works in a stand-up position, wearing gloves and
operating with a thick sterile drape that may be fully opaque
(See figure 1 above). Also, the device on the side of the
operating table must resist to fluids and must be compatible
with a sterile environment.
In addition to the constraints linked to the surgical
environment and to ensure the compatibility with existing
table rails, the top part of our new input device must fit into
the lower part of the existing table-side box already used in
the radiology lab to control the X-Ray acquisition system.
However, this backward compatibility strategy brings other
constraints regarding the clearance room for the 3D handle
(knob) manipulation.
Existing solutions in the medical world
Current solutions for 3D image manipulation in a sterile
environment usually integrate joysticks [1] [6]. However,
most of the joysticks are designed to effect translations
rather than integrated multi-axes smooth 3D rotations.
Free moving hand-held 6 DOF “flying” devices such as the
“3D ball” have generated considerable interest in the user
interface community and their potential use in neurosurgery
have been considered [4]. For our application, however, we
excluded them because of their intrinsic limitations. The
3D handling of these devices is tiring and the lack of
persistence in position requires an additional pedal or
button to freeze the 3D movement [7].
The CAD desktop 6 DOF input devices
The Computer Aided Design (CAD) workstations
extensively use solutions based on an opto-electronic 3D
input device such as the SpaceMouse® [3] to perform quick
and precise 3D orientations. These isometric devices
provide the computer simultaneous information with about
6 degrees of freedom, 3 in rotation and 3 in translation.
Because of its available number of DOF and its reliability,
i.e. calibration free and high duration, this technology is
attractive to the user interface designer considering the
similar need of 3D image manipulation for the
interventional radiologists. However, these 6 DOF devices
in CAD workstations appear mostly as bi-manual desktop
solutions. The normal mouse is still necessary for the point-
and-click selection tasks.
METHODOLOGY AND DESIGN PROCESS
General framework
The design process of the “InRoom3D” project is based on
the GE Healthcare Six Sigma methodology that integrates
the user centered design approach with:
The identification of the design goals, based on the
customers needs analysis
A process of goal refinement toward quantifiable
technical requirements, including usability
requirements
A data driven selection of design options based on
internal experiments and usability studies
Among the many Six Sigma tools used in our project, the
Quality Function Deployment is the most rigorous way to
list users expectations, to prioritize them according to their
user importance and to compute the prioritized technical
requirements.
Main design goals
Users interviews and users workshops during meetings with
radiologists and video analysis of the interventional
radiologists at work provided raw materials for precise
functional and usability requirements. The table below
shows the five first main requirements for the “InRoom3D”
user interface, including both hardware and software, and
the corresponding main evaluation techniques.
Main Design Goals Evaluation methods
Fast & Easy 3D orientation of a
3D object
Experiment,
User testing &
questionnaire
Adapted to glove & sterile drape User testing &
questionnaire
Experiment,
Comfortable in stand-up position User testing &
questionnaire
Easy access to main 3D
visualization functions Design, User testing
& questionnaire
Usable in blind conditions Design, User testing
& questionnaire
Main design goals and their evaluation methods
Validation methods: iterative design and usability
evaluation
The iterative design process followed in the project required
both virtual and real prototypes. The virtual study with
anthropometric mannequins and the 2D or 3D models was
the preferred design method in the early phase of the project
(See figure 2 below).
The following usability evaluation methods have been
extensively used for the “InRoom3D” project: (see: Design
Validation, below):
Iterative design with specific user testing
Validation by design
Experimental usability evaluation
Integrated usability evaluation
3
Figure 2. Two steps in virtual prototyping of the 3D box
handle and palm rest (side view). Left: early study. Right: first
functional prototype
SOLUTION
The design of the table-side3D box
The input device (the “3D box”) integrates the 6-DOF
technology for a stand up sterile environment with the
integration of 3D movements and the standard mouse point-
and-click selection function. See figure 3 below.
The main design features are:
The 3D 6-DOF opto-electronic sensor housed in
the handle (called here “3D handle” from its main
function)
The lateral main buttons for switching between 3D
movements and 2D cursor control
The additional keyboard with its nine shortcut
keys
The palm rest.
Figure 3. 3D box for interventional radiology. 1: mini
keyboard, 2: main lateral buttons. 3: 3D handle. 4: Palm rest.
The 3D handle (knob)
The design of the 3D handle follows the main usability
requirement, i.e. to provide the user a comfortable and
precise tool for continuous integrated 2D/3D movements in
the conditions of the cath. lab (See figure 3 - 3). The user
can perform multi-axes rotations and translations on the
handle to achieve the 3D rotations or the integrated
zooming and panning of the 3D object. The dimensions, the
front/back orientation of the 3D handle and the clearance
room for fingers around the handle follow the
anthropometric constraints for handle grasping in the stand
up working position, with the gloves and the sterile drape
[2] [5].
Based on the usability evaluation of the first functional
prototype, the 3D handle was modified with deeper concave
finger rails on the side for a better grasp with gloves and
sterile drape (See figure 4 below). In addition, straighter
front/back lines appear on the side of 3D handle, instead of
a round shape, to give tactile cues for easier control during
the front/back and left/right image translation.
Figure 4. Close up on the 3D handle. The lateral finger rails
and the front edges on the handle offer a better grasping with
gloves and sterile drape cover.
The main buttons
The main buttons are a key part of the 3D box to quickly
switch between a 3D rotation and the control of a 2D on-
screen cursor for function selection (See figure 3 - 2). In
addition to that, inside the 2D cursor mode, the user can
press the main buttons to perform any point-and-click
action such as the button or menu selection. Therefore, the
access of the main buttons should be fast and easy, without
physical strain for a frequent use.
Specific mechanical part prototypes and iterative design
validated the main dimensions, orientation and switches
characteristics for the main buttons, taking into account the
integration constraints of the switches in the mechanical
design.
Each of the two main buttons on both sides of the 3D
handle has the same function. This design provides an
easier access, especially for the left-handed users, and more
flexibility in the user’s movements.
The mini keyboard
The radiologist can perform any action in using the 3D
handle, the main buttons and the on-screen buttons or menu
items. However, a mini keyboard, with 9 key functions,
allows the user to quickly perform frequent or important
4
actions such as storing a given orientation for 3D
visualization or switching mode between 3D rotations and
integrated zooming & panning (See figure 3 - 1 above). .
Criteria for placement of functions on the keyboard are the
expected frequencies of use of the functions to minimize
the reaching distance from the 3D handle, the similarity
between functions (similar functions are close to each
other) and their order in the workflow.
Two types of button shapes and tactile cues on the keyboard
such as a smooth spike and two edges on the top part of the
box help the user to find the keys under the sterile drape
(See figure 3 and figure 4 above).
The palm rest
The palm rest is one of the key features for the usability of
an input device in a stand up position (See Figure 3 - 4). It
offers the stability and the comfort needed for a potentially
long 3D image review. The anthropometry provided the
main requirements for the palm rest [2][5]. A non-
functional prototype validated the overall size of the user
part (the relative position of the palm rest cushion to the 3D
handle). The functional prototype evaluation provided
additional data for the optimization of the palm rest cushion
(thickness, material, shape). Requirements for an easy
cleaning and aesthetic consideration guided the design
optimization on the last prototype of the palm rest (See
figure 4 above and figure 6 -5 below).
The “InRoom3D” application
The “InRoom3D” application offers a series of functions to
visualize the 3D images, to store and to retrieve useful 3D
points of views, to synchronize the orientation of 3D image
with live X-Ray 2D view, and to measure anatomical
structures. The software user interface on the monitor offers
a user support for better control of the 3D movement,
function selection and readability in the
surgical/interventional working environment: bigger on-
screen buttons and menus, roll over changes for easy button
and menus selection, assistance to the 3D movements,
multiple ways to select functions with virtual buttons and
hard keys on the mini keyboard.
Figure 5 illustrates the synergy between the hardware and
the software user interface design. During the procedure,
the radiologist can store and retrieve up to 9 points of view
of the 3D anatomy. To retrieve a stored point of view, the
user clicks either on a specific on-screen button or presses a
specific hard key on the mini keyboard. A catalog of
images appears (See figure 5 below). To select the point of
view, the user clicks on the image of the needed point of
view or presses the hard key on the mini-keyboard
corresponding to the desired point of view. Once selected,
the pop up catalog disappears and the full screen 3D image
is updated following the selected point of view.
Figure 5. “InRoom3D” screen. The pop up catalog of images
allows the user to retrieve a stored point of view.
DESIGN VALIDATION
Iterative design with prototype and usability evaluation
Usability evaluations on prototypes and physical mockups
provided feedback for the iterative design of the table-side
3D box device. Among others, the anthropometric method
of limits [5] was useful to find the optimal values of the
height, the orientation, the dimensions of the 3D mouse
handle, the keyboard, the buttons, and the palm rest. Figure
6 illustrates a step in the design update of the table-side 3D
box after usability evaluation with the first functional
prototype (top view). The new design shows an updated
keyboard layout based on updated frequencies of use of the
buttons (1), a second main button on the left of the 3D
handle (2), more clearance room for the fingers around the
3D handle (3), a modified 3D handle shape (4) and a
modified palm rest (5).
User tests on non-functional and functional prototypes
validated the parameters of the mini keyboard: the test
included the access to the keys (keyboard layout), the
forearm-wrist angle (front/back angle of the keyboard), the
clearance for fingers (Key pitch), and the key usability
(switches operating travel, operating force and snap ratio).
The final update of the mini keyboard layout was made
based on the usability evaluation with the functional
prototype. The most frequently used functions were placed
near the 3D handle to keep the user’s hand on the 3D
handle or near to it. See figure 6 - 1 (the double arrows
show the permutations of functions).
5
Figure 6. Design update of the 3D box following the usability
testing on the first functional prototype. 1: switch between
functions in the mini keyboard, 2: New left main lateral
button. 3: More room for fingers. 4: Modified shape of the 3D
handle. 5: Modified palm rest.
Validation by design
With the first functional prototype, the design team
conducted straightforward usability evaluation in checking
all the objective usability requirements. They are mostly
efficiency requirements expressed in number of clicks to
complete some task such as the quick access to the
“Zooming & panning” mode (no more than a single click is
needed).
Experimental evaluation
To assess the usability improvement over the state of the
art, two experiments were designed to measure the
objective performance of the “InRoom3D” application and
the 3D box for a precise 3D orientation task.
First experiment
The goal of the first experiment was to validate the main
design option for the 3D orientation: the use of the 3D opto-
electronic 6-DOF sensor and the software for precise full
3D orientation of medical images. The input device,
including our 6-DOF sensor, was a desktop version of our
input device: the SpaceMouse® from 3dconnexion
company. The benchmarked product was the existing
desktop workstation for 3D image review with a classic
mouse (GE Healthcare Advantage Workstation). We
compared both technologies in desktop conditions. Our
requirement to validate the technology was a better or a
similar performance for the SpaceMouse® input device.
The task consisted in performing in a minimum time a
precise 3D rotation from 8 possible starting positions to
reach a target 3D orientation. To find the target orientation,
the subject performed a precise (+/-1°) 3D alignment of a
ball in a 3D ring. After a training session, each of the 7
users performed in a randomized order 40 trials with the 2
tested devices (20 trials/device).
The results showed significant better performances (Median
Mood test - p = < 0.01) for each of the 7 users when they
use the 6-DOF sensor technology compared to the classic
mouse. To achieve the 3D orientation is almost 50% faster
with the space mouse 6-DOF input device than with the
classic mouse. See figure 7 below.
Figure 7. Compared completion time to perform a precise 3D
rotation in desktop situation. Classic mouse (white) versus
Space mouse 3D input device (dark).
Following our requirement, this first experiment validated
the selected technology for 3D orientation of medical
images in a desktop condition. A usability study based on
user testing with radiologists represents the second part of
this validation (see below – Integrated usability validation).
Second experiment
The second experiment was designed to validate for the
speed and precision of a 3D orientation the 6-DOF input
device in the interventional radiology conditions: (stand up
position, gloves, sterile drape). We compared it with the
classic mouse in the conditions for the desktop review
(sitting position, no gloves, no drape). Our requirement was
that our table-side 3D box prototype should offer at least
equal performances for 3D orientation compared to the
classic mouse in the desktop conditions.
The results from 6 subjects demonstrated that in spite of the
difficult conditions of the surgical-like environment, the
new 3D input device prototype has equal or better
performance for the precise 3D orientation task than the
classic mouse in the desktop conditions (No significant
better performance for the desktop conditions).
6
Before further integrated usability evaluation, this second
experiment validated our 3D box prototype for precise 3D
orientation in the cath. lab conditions.
Integrated usability validation
Besides the iterative design with prototyping and usability
studies, integrated usability evaluations were conducted
with the participation of 12 radiologists who were trained
and asked to perform a list of 6 prescribed main tasks to
validate the desktop version of our application with a
desktop 6-DOF 3D input device. A subjective evaluation of
10 critical usability features and the detailed users
comments validated the main user interface choices and
guided the iterative design change of the product.
The design teams with the support of radiologists
completed several integrated usability evaluations of this
type during the project life on the functional prototypes to
validate and to optimize the design choices. Figures 5 and 6
show some main design changes following the integrated
evaluation of the functional prototype. In progress clinical
evaluation of the InRoom3D application is currently
validating the product in the full clinical context.
CONCLUSION
This paper shows that despite the traditional constraints in
the surgical/interventional environment, an adapted single
handed integrated input device allows the interventional
radiologist to freely manipulate the 3D images and to
perform a variety of actions that were previously possible
only in the desktop conditions. Usability studies in a
clinical environment also show that with an in-room table-
side control the radiologist can concentrate on the
intervention and on the 3D images review without leaving
the radiology lab, thus reducing the time for the
intervention. The table-side 3D box doesn’t need a specific
visual attention and the user can perform all the needed
actions even with a fully opaque sterile drape covering the
input device.
Since performing easy 3D image manipulation and mouse-
like point-and-click functions in a sterile environment is a
common need in surgery, the user interface design choices
presented here may be a useful starting point for other
surgery applications beyond the field of the interventional
vascular radiology. A possible extension of the mouse-like
“point-and-click” function implemented here would be to
separate the functions of the 2 main buttons in linking the
right button to a pop up menu capability similar to the
classic mouse right button.
Given the constraint due to the current thick sterile drapes,
future directions for the improvement of
surgical/interventional input device usability should either
includes more flexible sterile drapes or other interaction
techniques such as voice control and gesture recognition to
minimize the user actions though the sterile drape.
DESIGN TEAM
The “InRoom3D” project team consists of a user interface
designer, an industrial designer and a software developer
with the support of application specialists, marketing staff,
and the users: the interventional radiologists.
ACKNOWLEDGMENTS
We especially thank Pr. Lasjaunias. Neuroradiology Dpt. at
Kremlin Bicetre Hospital. France, for his support for the
InRoom3D project and all the interventional radiologists
involved in the various evaluations of the “InRoom3D”
product.
REFERENCES
1. 3D liver aids surgery. Virtual organ image beamed into
OR. Nature Science Update, 11 March 2002
2. Cacha, C.: Ergonomics and Safety in Hand Tool Design,
Lewis, 1999.
3. Dietrich, J. and Plank, G.: Optoelectronic system housed
in a plastic sphere. US patent 4,785,180. Nov. 15 1988.
4. Hinckley, K., Tullio, J., Pausch, R.,Proffitt, D., and
Kassel, N.: Usability analysis of 3D rotation techniques,
Proc. UIST’97, 1-10.
5. Pheasant, S.: Bodyspace. Anthropometry, ergonomics
and the Design of Work. 2nd Ed., Taylor & Francis.
London, UK, 2001
6. Visual Networking Area at the University of
Manchester. SGI Document 3294pdf. SGI Corp.,
http://www.sgi.com/industries/sciences/medical/success/
7. Zhai, S. User Performance in Relation to 3D Input
Device Design, Computer Graphics 32,4 (1998), 50-54.
http://www.dgp.toronto.edu/people/BillBuxton/SIGGR
APHQ/shumin.html
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Based mainly on a series of studies the author conducted at the University of Toronto, this article reviews the usability of various six degrees of freedom (6 DOF) input devices for 3D user interfaces. The following issues are covered in the article: the multiple aspects of input device usability (performance measures), mouse based 6 DOF interaction, mouse modi-fications for 3D interfaces, free-moving isotonic 6 DOF devices, desktop isometric and elastic 6 DOF devices, armature-based 6 DOF devices, position vs. rate control and the form factors of 6 DOF control handle.These issues are treated at an introductory and practical level, with pointers to more technical and theoretical references.
Article
Full-text available
We report results from a formal user study of interactive 3D rotation using the mouse-driven Virtual Sphere and Arcball techniques, as well as multidimensional input techniques based on magnetic orientation sensors. Multidimensional input is often assumed to allow users to work quickly, but at the cost of precision, due to the instability of the hand moving in the open air. We show that, at least for the orientation matching task used in this experiment, users can take advantage of the integrated degrees of freedom provided by multidimensional input without necessarily sacrificing precision: using multidimensional input, users completed the experimental task up to 36% faster without any statistically detectable loss of accuracy. We also report detailed observations of common usability problems when first encountering the techniques. Our observations suggest some design issues for 3D input devices. For example, the physical form-factors of the 3D input device significantly influenced u...
Optoelectronic system housed in a plastic sphere. US patent 4,785,180
  • J Dietrich
  • G Plank
Dietrich, J. and Plank, G.: Optoelectronic system housed in a plastic sphere. US patent 4,785,180. Nov. 15 1988.
3D liver aids surgery. Virtual organ image beamed into OR
3D liver aids surgery. Virtual organ image beamed into OR. Nature Science Update, 11 March 2002