Conference PaperPDF Available

Patient Specific Virtual and Physical Simulation Platform for Surgical Robot Movability Evaluation in Single-Access Robot-Assisted Minimally-Invasive Cardiothoracic Surgery

Abstract and Figures

Recently, minimally invasive cardiothoracic surgery (MICS) has grown in popularity thanks to its advantages over conventional surgery and advancements in surgical robotics. This paper presents a patient-specific virtual surgical simulator for the movability evaluation of single-port MICS robots. This simulator can be used for both the pre-operative planning to rehearse the case before the surgery, and to test the robot in the early stage of development before physical prototypes are built. A physical simulator is also proposed to test the robot prototype in a tangible environment. Synthetic replicas of the patient organs are able to replicate the mechanical behaviors of biological tissues, allowing the simulation of the physical interactions robot-anatomy. The preliminary tests of the virtual simulator showed good performance for both the visual and physics processes. After reviewing the physical simulator, a surgeon provided a positive evaluation of the organ replicas in terms of geometry and mechanical behaviors.
Content may be subject to copyright.
Patient Specific Virtual and Physical Simulation
Platform for Surgical Robot Movability
Evaluation in Single-Access Robot-Assisted
Minimally-Invasive Cardiothoracic Surgery
Giuseppe Turini1,2(B
), Sara Condino2, Sara Sinceri2, Izadyar Tamadon3,
Simona Celi4, Claudio Quaglia3, Michele Murzi4, Giorgio Soldani5,
Arianna Menciassi3, Vincenzo Ferrari2,6, and Mauro Ferrari2
1Computer Science Department, Kettering University, Flint, MI, USA
2Department of Translational Research on New Technologies in Medicine
and Surgery, EndoCAS Center, University of Pisa, Pisa, Italy
3The Biorobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
4Fondazione Toscana Gabriele Monasterio, Pisa, Italy
5Consiglio Nazionale delle Ricerche, Istituto di Fisiologia Clinica, Pisa, Italy
6Information Engineering Department, University of Pisa, Pisa, Italy
Abstract. Recently, minimally invasive cardiothoracic surgery (MICS)
has grown in popularity thanks to its advantages over conventional
surgery and advancements in surgical robotics.
This paper presents a patient-specific virtual surgical simulator for the
movability evaluation of single-port MICS robots. This simulator can be
used for both the pre-operative planning to rehearse the case before the
surgery, and to test the robot in the early stage of development before
physical prototypes are built.
A physical simulator is also proposed to test the robot prototype in
a tangible environment. Synthetic replicas of the patient organs are able
to replicate the mechanical behaviors of biological tissues, allowing the
simulation of the physical interactions robot-anatomy.
The preliminary tests of the virtual simulator showed good perfor-
mance for both the visual and physics processes.
After reviewing the physical simulator, a surgeon provided a positive
evaluation of the organ replicas in terms of geometry and mechanical
Keywords: Virtual reality ·Unity game engine ·Computer-assisted
surgery ·Minimally-invasive surgery ·Cardiothoracic surgery ·Robotic
surgery ·Surgical simulation
Springer International Publishing AG 2017
L.T. De Paolis et al. (Eds.): AVR 2017, Part II, LNCS 10325, pp. 211–220, 2017.
DOI: 10.1007/978-3-319-60928-7 18
212 G. Turini et al.
1 Introduction
Minimally invasive cardiothoracic surgery (MICS) has grown rapidly over the
past decade, thanks to continuous innovations in surgical techniques, advances
in surgical instruments, and the adoption of robotic technologies. According to
literature evidence, major cardiac operations traditionally performed through a
median sternotomy can be accomplished less invasively through small incisions,
with equivalent safety and durability [12].
Robotic systems have been utilized successfully to perform complex surgi-
cal procedures such as mitral valve and tricuspid valve repairs [3,9], single and
multiple vessel coronary artery bypass surgeries [9,11], atrial fibrillation abla-
tions [20], intracardiac tumor resections, atrial septal defect closures, and left
ventricular leads [2,18]. Moreover recent studies demonstrated the feasibility of
performing aortic valve replacement in adults using surgical robots [10].
Single port access (SPA) surgery, which uses one skin incision for interven-
tions, caught the attention of the surgical community in the past a few years
because of its potential to further reduce the invasiveness of surgical procedures
and post-operative complications. Looking at this potentiality, researchers have
proposed various robotic systems to assist SPA surgery [21]. For example, an
articulated robotic probe has been designed for cardiac surgery applications [8],
and a flexible “snakelike” robotic systems have been developed to allow physi-
cians to view, access, and perform complex procedures on the beating heart
through a single-access port [14].
Despite the reduced invasiveness, SPA robotic surgery has its own limitations.
One of the main issues is that the anatomical region reached from an incision site
is restrained. So the access port placement has to be carefully chosen depending
on: patient anatomy, steps of the surgical procedure, and robot workspace [19].
Patient-specific surgical simulators could overcome this limitation by allowing
the surgeon to plan the intervention in order to evaluate the robot workspace
and the optimal access port placement [13,19]. Moreover they can also be used
by robot designers to evaluate the robot dimensioning and distal dexterity in a
realistic scenario, thus improving the quality and shortening the design cycle.
In this paper we present a patient-specific virtual and physical simulation
platform (Figs. 1and 8b) for surgical robot evaluation in single-access robot-
assisted minimally-invasive cardiothoracic surgery.
2 Modeling of the Patient Anatomy
A computed tomography dataset with contrast medium was used to generate the
3D model. The stack of medical images in DICOM format was processed using a
specific segmentation pipeline, developed in VMTK, a software for the generation
of 3D virtual models by integrating custom Python scripts. The segmentation
algorithm is based on a hierarchical approach as previously described in [5].
Basically: once the most simple feature to be identified is reconstructed, the
associated pixels are excluded from the subsequent segmentation phase. Finally,
Virtual-Physical Simulator for Surgical Robot Movability Evaluation 213
Fig. 1. Overview of the virtual simulator: the surgical robot initially positioned above
the chest, 2 rib parts (in green) can be expanded to facilitate the robot insertion, the
insertion point highlighted by a red dot on the skin, the 3 views from the robot end-
effector cameras (right), and the GUI to control the virtual simulator (bottom). (Color
figure online)
mesh reconstruction, artefacts removal, and holes filling stages were performed
to generate the 3D models of the patient anatomy necessary for the surgical
simulation, including: rib cage, aortic arch, ascending aorta, and aortic valve.
3 Development of the Virtual Simulator
The virtual simulator was designed to be a standalone desktop application for
the Microsoft Windows platform (Fig. 1). We used Blender and Unity as the
main tools for the 3D content creation and the software development respectively
(Fig. 2). Both tools were chosen because they are cost-effective and technically
suitable to implement virtual surgical simulators [7,15].
3.1 Modeling of the Virtual Surgical Robot
The virtual surgical robot was modeled accordingly to the current prototype as
designed by the mechanical engineering team. This single-access surgical robot
consists in: a mechanical flexible trunk with a user-controlled torsion, 3 distal
blades allowing its insertion and anchoring into the aorta, and 3 distal cameras
for the inspection of the aorta inner part once the robot is inserted (Fig. 4a).
In Unity, the entire behavior of the virtual robot was controlled by a C#
script component, implementing the following mechanisms:
214 G. Turini et al.
Fig. 2. The virtual simulator project in the Unity game engine editor.
control the main body torsion using the keyboard arrow keys;
robot insertion/extraction pressing “page down/“page up” respectively;
opening/closing of the robot end-effector blades with keys “e” and “q”;
robot rotation around its axis using “comma” and “period” keys.
In order to enable the interactions between the virtual surgical robot and
the virtual anatomy, we configured each robot part with a Collider component,
and a Rigidbody component (Fig. 3). These Unity components enable collision-
detection capabilities, and physics properties respectively.
All the robot Collider components have been configured using the proper
type (i.e. shape) to approximate the 3D geometry of the respective part, and
setting them to be trigger Colliders. In this way, Unity will be able to detect the
collisions robot-anatomy, but these collisions will not affect the robot positioning.
(a) (b) (c) (d)
Fig. 3. The structure of the Collider components of the virtual robot in Unity: (a),
(b), (c), and (d) show the torsion boundaries of the surgical robot.
Virtual-Physical Simulator for Surgical Robot Movability Evaluation 215
(a) (b)
Fig. 4. Detailed view of: (a) the virtual robot end-effector (highlighted in orange),
including 3 blades to open/close the tip and 3 cameras for endovascular inspection;
and (b) the navigation mesh (in blue) baked on the chest of the patient, and used to
implement the interactive navigation of the insertion point on the skin (Color figure
All the robot Rigidbody components have been configured to be kinematic.
In this way, we can assign each part its own physics properties, but we disable
any update of its position/orientation performed by the Unity physics engine.
This configuration provides the maximum control to the user, and preserves
the capability to detect all the interactions between the robot and the anatomy.
3.2 Interactive Simulation of the Surgical Robot Movability
The virtual surgical simulator was designed to provide a user-friendly interface
to enable the surgeon to rehearse the robot placement using only: the mouse, its
3 buttons, and a minimal set of keyboard keys (Fig. 1).
The complete interface includes some keyboard controls, a GUI panel (Fig.1),
and some interactions available directly on the 3D virtual anatomy:
an invisible trackball allows the user to rotate, zoom in, and zoom out the
main view (i.e. the point-of-view of the surgeon) using only mouse buttons
and drag-and-drop (see Figs. 5aand5b);
the virtual chest is interactive, allowing the user to click on the skin to directly
place the insertion point (see red dot in Fig. 1);
the insertion point can also be precisely positioned by moving it on the skin
using the “WASD” keys (Fig. 1);
five buttons on the GUI panel (Fig. 1) allow the user to rotate the robot in
respect to the access point pivot axis, orthogonal to the skin surface (Fig. 5a);
an error message is shown on the GUI panel (and an audio signal is played)
whenever a collision between the robot and the rib cage is detected (Fig. 5b).
216 G. Turini et al.
(a) (b)
Fig. 5. Two different placements of the single-access surgical robot using the virtual
simulator: (a) frontal view of the virtual anatomy and the robot tilted in respect to
the pivot axis (white), and (b) side view of the virtual anatomy with an error message
(red) signaling a collision between the robot and the rib cage.
The trackball was implemented in a C#script component attached to the
main camera, and allows: the rotation of the main view around the virtual
anatomy, with angular limits to avoid uncomfortable points of view; the zooming
in and out implemented modifying the main camera field-of-view angle.
The interaction with the virtual chest was implemented through ray-casting,
in a C#script component attached to it. Every time a mouse right click event
was raised, the script converted it into a 3D ray using the Camera class un-
projection capabilities. Then, the script used the Unity physics engine to cast
the 3D ray, detecting its collision with the virtual chest thanks to a Mesh Collider
component added to the skin.
The fine positioning of the insertion point is performed moving it on the
skin using the keyboard. This implementation exploited a Unity NavMesh:a
navigation mesh approximating the “walkable” surface and enabling artificial
intelligence path-planning capabilities (Fig. 4b). In our project, a NavMesh has
been baked on the chest, and it has been properly configured to allow movements
only on the almost-flat part of the skin. A NavMeshAgent component attached
to the insertion point enables the movement on the NavMesh. Finally, a C#
script component attached to the chest controls the NavMeshAgent to perform
the proper movement accordingly to the user inputs.
The GUI panel buttons allow the tilting of the virtual robot in respect to
the access point pivot axis (Fig. 5a). This feature has been implemented simply
exploiting the parenting between Unity GameObjects, in this case: the insertion
point (the parent GameObject), and the virtual robot (the child GameObject).
Finally, collisions between the virtual robot and the rib cage are identified
using Unity collision-detection capabilities. In fact, all the robot parts have their
respective Colliders, and the rib cage has a MeshCollider.
4 Development of the Physical Simulator
The manufacturing of organ physical replicas involves rapid prototyping tech-
niques as described in previous works [6,17]. More particularly a 3D printer
Virtual-Physical Simulator for Surgical Robot Movability Evaluation 217
(a) (b) (c)
Fig. 6. Manufacturing of the ascending aorta distal part and aortic valve: (a) 3D virtual
model of the portion to be reproduced, (b) CAD model of designed mold, and (c) CAD
view of the mold inner core with pins for a correct positioning.
(Dimension Elite 3D Printer) is used to turn the 3D virtual model of the patient
bones into tangible 3D synthetic replicas made of acrylonitrile butadiene styrene
(ABS). This plastic is commonly used for the manufacturing of bone replica for
orthopedic surgery simulations, since it sufficiently replicates the mechanical
behavior of the natural tissue [1,16,17].
Soft synthetic replica of the whole or a part of an organ can be manufactured
with casting technique, selecting plastic materials with properties tailored to
the specific application [4]. Injection molds are designed using a computer-aided
design (CAD) software starting from the organ 3D virtual models (Fig. 6).
Figure 7shows the physical simulator developed for cardiac interventions
involving the ascending aorta and the aortic valve, including a replica of:
the rib cage with bones made of ABS, and a portion of costal cartilage made
of a high hardness silicone rubber to reproduce the elastic behavior of the
natural tissue (highlighted in red in Fig. 8b);
the aortic arch (with brachiocephalic, left common carotid, and left subclavian
arteries) made of ABS with a pin to anchor it to a base (Fig. 7b);
the ascending aorta and the aortic valve, which are the anatomical targets of
the intervention, made of soft silicone for a realistic interaction with surgical
instruments (traditional and/or robotic devices);
(a) (b) (c)
Fig. 7. Assembly of the physical simulator: (a) CAD assembly including a portion of
the rib cage, the aortic arch, the ascending aorta, the aortic valve, an aortic valve
support, and a base for a stable positioning of the anatomical parts; (b) the aortic arch
and the aortic valve support with pins; and (c) the CAD assembly of whole simulator.
218 G. Turini et al.
(a) (b)
Fig. 8. Side-by-side views of the virtual and physical anatomies: (a) the virtual
anatomy as imported in Unity for the virtual simulator; and (b) the physical anatomy
built using 3D printing technology and silicones to have flexible and rigid parts. (Color
figure online)
an aortic valve support made of ABS with a pin for anchoring (Fig. 7b);
a base with a grid of holes accommodates the pin of the aortic arch and the
aortic valve support (labeled in green in Fig. 8b).
The assembly of the organ replicas can be customized using the grid of holes
in the base of the physical simulator (see green label in Fig. 8b). This grid allows
apin-hole coupling, constraining 5 DOF, that can be used to simulate different
anatomical configurations by repositioning the aortic arch and the aortic valve.
5 Preliminary Results and Future Work
The virtual surgical simulator was tested on a laptop running Microsoft Windows
7 (Intel Core i7 – 2.80 GHz, 16 GHz RAM, GPU nVidia GeForce GT 650 M),
using a virtual 3D environment including: the patient anatomy composed of
approximately 81k vertices and 150k triangles, and the surgical robot made with
roughly 63k vertices and 64k triangles. The update frequency was ranging from
65 to 75 fps, with the physics engine running at 50 fps (default). The memory
required to run the simulator was about 140 MB.
The physical simulator underwent a qualitative evaluation performed by a
surgeon, and both the geometry and mechanical behavior of all the synthetic
organ replicas were positively evaluated. A quantitative evaluation, considering
all the factors affecting the generation of the 3D organ replicas, estimated an
accuracy of less than 2 mm for the physical anatomy.
The virtual simulator described allows the pre-operative planning to rehearse
the surgical case before the actual intervention, and the evaluation of the surgical
robot during the design-development cycle. The physical simulator presented
enables the evaluation of the surgical robot in a synthetic anatomy, testing the
physical interactions between the robot prototype and the organ replicas. Thus,
we can assess the robot payloa d and compli ance performing a wide range of tasks
(not included in the virtual simulation). Furthermore, the virtual and physical
Virtual-Physical Simulator for Surgical Robot Movability Evaluation 219
surgical simulators can also be efficiently integrated into the clinical context for
teaching and training purposes.
In the future, we plan to combine the virtual and physical simulators into a
single mixed reality system. Additionally, we will also perform validation studies
to test the face validity of the simulator.
Acknowledgments. The research leading to these results has been supported by the
scientific project ValveTech (“Realizzazione di una Valvola Aortica Polimerica di Nuova
Concezione ed Impiantabile Tramite Piattaforma Robotica con Tecniche di Chirurgia
Mininvasiva” 2016–2018) funded by the Tuscany Region (Italy) through the call FAS
SALUTE 2014.
1. Baccarin, L.S., Casarin, R.C.V., Lopes-da Silva, J.V., Passeri, L.A.: Analysis of
mandibular test specimens used to assess a bone fixation system. Craniomaxillo-
facial Trauma Reconstr. 8(03), 171–178 (2015)
2. Bhatt, A.G., Steinberg, J.S.: Robotic-assisted left ventricular lead placement. Card.
Electrophysiol. Clin. 7(4), 649–659 (2015)
3. Bush, B., Nifong, L.W., Alwair, H., Chitwood Jr., W.R.: Video-atlas on robotically
assisted mitral valve surgery. Ann. Cardiothorac. Surg. 2(6), 846–848 (2013)
4. Carbone, M., Condino, S., Mattei, L., Forte, P., Ferrari, V., Mosca, F.: Anthropo-
morphic ultrasound elastography phantoms: Characterization of silicone materials
to build breast elastography phantoms. In: 2012 Annual International Conference
of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 492–494.
IEEE (2012)
5. Celi, S., Berti, S.: In-vivo segmentation and quantification of coronary lesions by
optical coherence tomography images for a lesion type definition and stenosis grad-
ing. Med. Image Anal. 18(7), 1157–1168 (2014)
6. Condino, S., Carbone, M., Ferrari, V., Faggioni, L., Peri, A., Ferrari, M., Mosca,
F.: How to build patient-specific synthetic abdominal anatomies. An innovative
approach from physical toward hybrid surgical simulators. Int. J. Med. Robot.
7(2), 202–213 (2011)
7. De Luca, V., Meo, A., Mongelli, A., Vecchio, P., De Paolis, L.T.: Development
of a virtual simulator for microanastomosis: new opportunities and challenges. In:
Paolis, L.T., Mongelli, A. (eds.) AVR 2016. LNCS, vol. 9769, pp. 65–81. Springer,
Cham (2016). doi:10.1007/978-3-319-40651-0 6
8. Degani, A., Choset, H., Wolf, A., Zenati, M.A.: Highly articulated robotic probe
for minimally invasive surgery. In: Proceedings of the 2006 IEEE International
Conference on Robotics and Automation, ICRA 2006, pp. 4167–4172. IEEE (2006)
9. Folliguet, T.A., Dibie, A., Philippe, F., Larrazet, F., Slama, M.S., Laborde, F.:
Robotically-assisted coronary artery bypass grafting. Cardiol. Res. Pract. 2010
10. Folliguet, T.A., Laborde, F.: Robotic aortic valve replacement. In: Chitwood Jr.,
W.R. (ed.) Atlas of Robotic Cardiac Surgery, pp. 265–269. Springer, London (2014)
11. Gong, W., Cai, J., Wang, Z., Chen, A., Ye, X., Li, H., Zhao, Q.: Robot-assisted
coronary artery bypass grafting improves short-term outcomes compared with min-
imally invasive direct coronary artery bypass grafting. J. Thorac. Dis. 8(3), 459–468
220 G. Turini et al.
12. Iribarne, A., Easterwood, R., Chan, E.Y., Yang, J., Soni, L., Russo, M.J., Smith,
C.R., Argenziano, M.: The golden age of minimally invasive cardiothoracic surgery:
current and future perspectives. Future Cardiol. 7(3), 333–346 (2011)
13. Moglia, A., Turini, G., Ferrari, V., Ferrari, M., Mosca, F.: Patient specific surgical
simulator for the evaluation of the movability of bimanual robotic arms. Stud.
Health Technol. Inf. 163, 379–385 (2011)
14. Neuzil, P., Cerny, S., Kralovec, S., Svanidze, O., Bohuslavek, J., Plasil, P., Jehlicka,
P., Holy, F., Petru, J., Kuenzler, R., et al.: Single-site access robot-assisted
epicardial mapping with a snake robot: preparation and first clinical experience.
J. Robot. Surg. 7(2), 103–111 (2013)
15. Paci, A., Marcutti, S., Ricci, S., Casadio, M., Vercelli, G.V., Marchiol`e, P., Cor-
done, M.: eBSim: development of a low-cost obstetric simulator. In: Paolis, L.T.,
Mongelli, A. (eds.) AVR 2016. LNCS, vol. 9769, pp. 101–110. Springer, Cham
(2016). doi:10.1007/978-3-319-40651-0 9
16. Parchi, P., Condino, S., Carbone, M., Gesi, M., Ferrari, V., Ferrari, M., Lisanti,
M.: Total hip replacement simulators with virtual planning and physical replica for
surgical training and rehearsal. In: Proceedings of the 12th IASTED International
Conference on Biomedical Engineering, BioMed 2016, pp. 97–101 (2016)
17. Parchi, P., Ferrari, V., Piolanti, N., Andreani, L., Condino, S., Evangelisti, G.,
Lisanti, M.: Computer tomography prototyping and virtual procedure simulation
in difficult cases of hip replacement surgery. Surg. Technol. Int. 23, 228–234 (2013)
18. Poffo, R., Toschi, A.P., Pope, R.B., Celullare, A.L., Ben´ıcio, A., Fischer, C.H.,
Vieira, M.L.C., Teruya, A., Hatanaka, D.M., Rusca, G.F., et al.: Robotic surgery
in cardiology: a safe and effective procedure. Einstein (Sao Paulo) 11(3), 296–302
19. Turini, G., Moglia, A., Ferrari, V., Ferrari, M., Mosca, F.: Patient-specific surgical
simulator for the pre-operative planning of single-incision laparoscopic surgery with
bimanual robots. Comput. Aided Surg. 17(3), 103–112 (2012)
20. Wutzler, A., Wolber, T., Haverkamp, W., Boldt, L.H.: Robotic ablation of atrial-
fibrillation. J. Vis. Exp. JoVE 99 (2015)
21. Xu, K., Zheng, X.: Configuration comparison for surgical robotic systems using a
single access port and continuum mechanisms. In: 2012 IEEE International Con-
ference on Robotics and Automation (ICRA), pp. 3367–3374. IEEE (2012)
... rough robotics, further benefits can be reached in terms of telemanipulation, motion scaling, and even smaller incisions [8,9]. Researchers proposed various robotic systems to assist heart surgery [10], allowing in some cases the preoperative planning to test the surgical case before the actual intervention [11]. ...
... ournal of Healthcare Engineering experienced in the AVR procedure via mini-thoracotomy and mini-sternotomy tested the view modalities. Test was performed by using the simulation setup developed in [11], which includes a patient-specific replica of the rib cage, aortic arch, ascending aorta, and the aortic valve, as shown in Figure 9. e aortic arch is made of ABS, and it is provided with a pin to anchor it to a base, while the ascending aorta and the aortic valve are made of soft silicone for a realistic interaction with surgical instruments with casting technique, as described in [29][30][31]. ...
Full-text available
Aortic valve replacement is the only definitive treatment for aortic stenosis, a highly prevalent condition in elderly population. Minimally invasive surgery brought numerous benefits to this intervention, and robotics recently provided additional improvements in terms of telemanipulation, motion scaling, and smaller incisions. Difficulties in obtaining a clear and wide field of vision is a major challenge in minimally invasive aortic valve surgery: surgeon orientates with difficulty because of lack of direct view and limited spaces. This work focuses on the development of a computer vision methodology, for a three-eyed endoscopic vision system, to ease minimally invasive instrument guidance during aortic valve surgery. Specifically, it presents an efficient image stitching method to improve spatial awareness and overcome the orientation problems which arise when cameras are decentralized with respect to the main axis of the aorta and are nonparallel oriented. The proposed approach was tested for the navigation of an innovative robotic system for minimally invasive valve surgery. Based on the specific geometry of the setup and the intrinsic parameters of the three cameras, we estimate the proper plane-induced homographic transformation that merges the views of the operatory site plane into a single stitched image. To evaluate the deviation from the image correct alignment, we performed quantitative tests by stitching a chessboard pattern. The tests showed a minimum error with respect to the image size of 0.46 ± 0.15% measured at the homography distance of 40 mm and a maximum error of 6.09 ± 0.23% at the maximum offset of 10 mm. Three experienced surgeons in aortic valve replacement by mini-sternotomy and mini-thoracotomy performed experimental tests based on the comparison of navigation and orientation capabilities in a silicone aorta with and without stitched image. The tests showed that the stitched image allows for good orientation and navigation within the aorta, and furthermore, it provides more safety while releasing the valve than driving from the three separate views. The average processing time for the stitching of three views into one image is 12.6 ms, proving that the method is not computationally expensive, thus leaving space for further real-time processing.
... A 3D printer (Dimension Elite 3D Printer, with a building volume of 203 × 203 × 305 mm, and a maximum resolution of 0.178 mm) is used to turn the 3D CAD models into tangible 3D synthetic replicas made of acrylonitrile butadiene styrene (ABS). This plastic is commonly used for the manufacturing of bone replicas for orthopaedic surgery simulation, since it quite realistically replicates the mechanical behaviour of the natural tissue [32]. The anatomical parts to be printed are selected each time according to the specific surgical case: the design of the simulator requires the selection of the anatomical parts to be manipulated (those that can provide haptic feedback useful for the comprehensive understanding of the surgical case) and the parts that can be simply visualised in AR. ...
Full-text available
Cryosurgery is a technique of growing popularity involving tissue ablation under controlled freezing. Technological advancement of devices along with surgical technique improvements have turned cryosurgery from an experimental to an established option for treating several diseases. However, cryosurgery is still limited by inaccurate planning based primarily on 2D visualization of the patient’s preoperative images. Several works have been aimed at modelling cryoablation through heat transfer simulations; however, most software applications do not meet some key requirements for clinical routine use, such as high computational speed and user-friendliness. This work aims to develop an intuitive platform for anatomical understanding and pre-operative planning by integrating the information content of radiological images and cryoprobe specifications either in a 3D virtual environment (desktop application) or in a hybrid simulator, which exploits the potential of the 3D printing and augmented reality functionalities of Microsoft HoloLens. The proposed platform was preliminarily validated for the retrospective planning/simulation of two surgical cases. Results suggest that the platform is easy and quick to learn and could be used in clinical practice to improve anatomical understanding, to make surgical planning easier than the traditional method, and to strengthen the memorization of surgical planning.
... The selected anatomy offered a complicated configuration, which would not allow a manual operation in mini-thoracotomy. The stack of medical images was processed using a specific segmentation pipeline developed in VMTK [38], and a tangible replica of the organs was 3D printed in acrylonitrile butadiene styrene (ABS) (rigid anatomical parts) or manufactured via silicone molding (soft tissues). ...
Objective: Aortic valve disease is the most common heart disease in the elderly calling for replacement with an artificial valve. The presented surgical robot aims to provide a highly controllable instrument for efficient delivery of an artificial valve by the help of integrated endoscopic vision. Methods: A robot (called ValveTech), intended for minimally-invasive surgery (MIS) and consisting of a flexible cable driven manipulator, a passive arm, and a control unit has been designed and prototyped. The flexible manipulator has several features (e.g. stabilizing flaps, tiny cameras, dexterous introducer and custom cartridge) to help the proper valve placement. It provides 5 degrees of freedom for reaching the operative site via mini-thoracotomy; it adjusts the valve and expands it at the optimal position. The robot was evaluated by ten cardiac surgeons following a real surgical scenario in artificial chest simulator with an aortic mockup. Moreover, after each delivery, the expanded valve was evaluated objectively in comparison with the ideal position. Results: The robot performances were evaluated positively by surgeons. The trials resulted in faster delivery and an average misalignment distance of 3.8 mm along the aorta axis; 16.3 degrees rotational angle around aorta axis and 8.8 degrees misalignment of the valve commissure plane to the ideal plane were measured. Conclusion: The trials successfully proved the proposed system for valve delivery under endoscopic vision. Significance: The ValveTech robot can be an alternative solution for minimally invasive aortic valve surgery and improve the quality of the operation both for surgeons and patients.
... The EndoCAS Segmentation Pipeline [27], a semi-automatic segmentation tool integrated into the open-source software ITK-SNAP, was used to process the generated DICOM (Digital Imaging and COmmunications in Medicine) dataset. Then, artifacts removal and mesh smoothing stages were performed [28] to optimize the vascular 3D model. Finally, a 3D printer, Objet30Prime (Stratasys, Los Angeles, CA, USA), was used to fabricate a transparent tangible 3D synthetic vascular model (Fig. 5b). ...
Full-text available
Objective: This work aims at providing novel endovascular instrumentation to overcome current technical limitations of in situ endograft fenestration including challenges in targeting the fenestration site under fluoroscopic control and supplying mechanical support during endograft perforation. Technology: Novel electromagnetically trackable instruments were developed to facilitate the navigation of the fenestration device and its stabilization at the target site. In vitro trials were performed to preliminary evaluate the proposed instrumentation for the antegrade in situ fenestration of an aortic endograft, using a laser guidewire designed ad hoc and the sharp-end of a commercial endovascular guidewire. Results: In situ fenestration was successfully performed in 22 trials. A total of two laser tools were employed since an over-bending of laser guidewire tip, due to its manufacturing, caused the damage of the sensor in the first device used. Conclusions: Preliminary in vitro trials demonstrate the feasibility of the proposed instrumentation which could widespread the procedure for in situ fenestration. The results obtained should be validated performing animal studies. Clinical Impact: The proposed instrumentation has the potential to expand indications for standard endovascular aneurysm repair to cases of acute syndromes.
... This completes the preparation phase. The mini thoracotomy location can be defined preoperatively using patient specific surgical simulators [25]. The simulator allows moving a virtual replica of the manipulator in the patient virtual anatomy, in order to evaluate the best location of the mini thoracotomy. ...
Conference Paper
Full-text available
Aortic heart valve replacement is a major surgical intervention, traditionally requiring a large thoracotomy. However, current advances in Minimally Invasive Surgery and Surgical Robotics can offer the possibility to perform the intervention through a narrow mini thoracotomy. The presented surgical robot and proposed surgical scenario aims to provide a highly controllable means for efficiently conducting valve replacement by endoscopic vision. The robot, consisting of a series of joints, is a cable actuated manipulator for reaching the operative site and delivering the valve at the required position. The robot is equipped with endoscopic cameras (to find the hinge points) and three stabilizing flaps (to stabilize the manipulator) for guarantying the proper valve placement. The manipulator is validated by experimental results of flaps' force and camera visions in artificial vessels.
... A 3D printer (Dimension Elite 3D Printer) is used to turn the 3D CAD models into tangible 3D synthetic replicas made of acrylonitrile butadiene styrene (ABS). is plastic is commonly used for the manufacturing of bone replicas for orthopaedic surgery simulation since it adequately approximates the mechanical behaviour of the natural tissue [37]. Finally, silicone mixtures and polyurethane materials are used for the manufacturing of the soft parts. ...
Full-text available
Orthopaedic simulators are popular in innovative surgical training programs, where trainees gain procedural experience in a safe and controlled environment. Recent studies suggest that an ideal simulator should combine haptic, visual, and audio technology to create an immersive training environment. This article explores the potentialities of mixed-reality using the HoloLens to develop a hybrid training system for orthopaedic open surgery. Hip arthroplasty, one of the most common orthopaedic procedures, was chosen as a benchmark to evaluate the proposed system. Patient-specific anatomical 3D models were extracted from a patient computed tomography to implement the virtual content and to fabricate the physical components of the simulator. Rapid prototyping was used to create synthetic bones. The Vuforia SDK was utilized to register virtual and physical contents. The Unity3D game engine was employed to develop the software allowing interactions with the virtual content using head movements, gestures, and voice commands. Quantitative tests were performed to estimate the accuracy of the system by evaluating the perceived position of augmented reality targets. Mean and maximum errors matched the requirements of the target application. Qualitative tests were carried out to evaluate workload and usability of the HoloLens for our orthopaedic simulator, considering visual and audio perception and interaction and ergonomics issues. The perceived overall workload was low, and the self-assessed performance was considered satisfactory. Visual and audio perception and gesture and voice interactions obtained a positive feedback. Postural discomfort and visual fatigue obtained a nonnegative evaluation for a simulation session of 40 minutes. These results encourage using mixed-reality to implement a hybrid simulator for orthopaedic open surgery. An optimal design of the simulation tasks and equipment setup is required to minimize the user discomfort. Future works will include Face Validity, Content Validity, and Construct Validity to complete the assessment of the hip arthroplasty simulator.
... For the aneurysmatic patients, the CT images corresponding to the last radiological follow-up available prior to a recommendation for interventional treatment was considered. The volumetric CT datasets were used to reconstruct the 3D surface models for both the groups by using the workflow presented in Ref. [32]. ...
We present a novel framework for the fluid dynamics analysis of healthy subjects and patients affected by ascending thoracic aorta aneurysm (aTAA). Our aim is to obtain indications about the effect of a bulge on the hemodynamic environment at different enlargements. 3D surface models defined from healthy subjects and patients with aTAA, selected for surgical repair, were generated. A representative shape model for both healthy and pathological groups has been identified. A morphing technique based on radial basis functions (RBF) was applied to mould the shape relative to healthy patient into the representative shape of aTAA dataset to enable the parametric simulation of the aTAA formation. CFD simulations were performed by means of a finite volume solver using the mean boundary conditions obtained from three-dimensional (PC-MRI) acquisition. Blood flow helicity and flow descriptors were assessed for all the investigated models. The feasibility of the proposed integrated approach of RBF morphing technique and CFD simulation for aTAA was demonstrated. Significant hemodynamic changes appear at the 60% of the bulge progression. An impingement of the flow toward the bulge was observed by analyzing the normalized flow eccentricity index.
Conference Paper
Full-text available
The paper deals with the development of virtual surgical simulator for microanastomosis developed within the INTERREG (Italy-Greece) MICRO project. Microanastomosis is a surgical technique that involves, under optical magnification, the conjunction of blood vessels of a few millimeters diameter and is used also to support the surgical treatment of tumours. This manuscript describes the two principal solutions analysed during the progress of the MICRO project. The first step concerns the development of the simulator using the Unity3D engine; the second step describes an evolution of the surgical simulator to support remote control by a haptic interface via web using the WebGL platform based on JavaScript code. For both, a force feedback module has been implemented that reads data coming from the simulator and converts them to generate a servo control action on the haptic interface. For both solutions, some results of the implemented simulator are described .
Conference Paper
Full-text available
This work presents the design a low cost obstetric simulator allowing precise identification of the fetal position in a simulated birth canal. The system consists on a female pelvis, a custom-made fetal mannequin, a virtual 3d representation, a visual display and an user interface to show in real time what happens inside the birth canal. Students are often unable not only to identify correctly the fetal head position, but also to discriminate between the two fontanels, for this reason the simulator eBSim can help them to train this ability and could be an important instrument for the instructors to objectively assess the clinical skill of each student.
Full-text available
The aim of this study was to assess through biomechanical testing if different synthetic materials used to fabricate test specimens have a different biomechanical behavior in comparison with other materials when simulating in vitro load resistance of a fixation method established for sagittal split ramus osteotomy (SSRO). Thirty synthetic and standardized human hemimandible replicas with SSRO were divided into three groups of 10 samples each. Group A-ABS plastic; Group B-polyamide; and Group C-polyurethane. These were fixated with three bicortical position screws (16 mm in length, 2.0-mm system) in an inverted l pattern using perforation guide and 5-mm advancement. Each sample was submitted to linear vertical load, and load strength values were recorded at 1, 3, 5, 7, and 10 mm of displacement. The means and standard deviation were compared using the analysis of variance (p < 0.05) and the Tukey test. A tendency for lower values was observed in Group B in comparison with Groups A and C. At 3 and 5 mm of displacement, a difference between Groups A and C was found in comparison with Group B (p < 0.05). At 7 and 10 mm of displacement, a difference was found among the three groups, in which Group C showed the highest values and Group B the lowest (p < 0.05). Taking into consideration the results obtained and the behavior of each material used as a substrate, significant differences occurred among the materials when compared among them.
Full-text available
Background: Pulmonary vein isolation (PVI) is an established treatment for atrial fibrillation (AF). During PVI an electrical conduction block between pulmonary vein (PV) and left atrium (LA) is created. This conduction block prevents AF, which is triggered by irregular electric activity originating from the PV. However, transmural atrial lesions are required which can be challenging. Re-conduction and AF recurrence occur in 20 - 40% of the cases. Robotic catheter systems aim to improve catheter steerability. Here, a procedure with a new remote catheter system (RCS), is presented. Objective of this article is to show feasibility of robotic AF ablation with a novel system. Materials and methods: After interatrial trans-septal puncture is performed using a long sheath and needle under fluoroscopic guidance. The needle is removed and a guide wire is placed in the left superior PV. Then an ablation catheter is positioned in the LA, using the sheath and wire as guide to the LA. LA angiography is performed over the sheath. A circular mapping catheter is positioned via the long sheath into the LA and a three-dimensional (3-D) anatomical reconstruction of the LA is performed. The handle of the ablation catheter is positioned in the robotic arm of the Amigo system and the ablation procedure begins. During the ablation procedure, the operator manipulates the ablation catheter via the robotic arm with the use of a remote control. The ablation is performed by creating point-by-point lesions around the left and right PV ostia. Contact force is measured at the catheter tip to provide feedback of catheter-tissue contact. Conduction block is confirmed by recording the PV potentials on the circular mapping catheter and by pacing maneuvers. The operator stays out of the radiationfield during ablation. Conclusion: The novel catheter system allows ablation with high stability on low operator fluoroscopy exposure.
Full-text available
To evaluate the short and medium-term outcomes of patients undergoing robotic-assisted minimally invasive cardiac surgery. From March 2010 to March 2013, 21 patients underwent robotic-assisted cardiac surgery. The procedures performed were: mitral valve repair, mitral valve replacement, surgical correction of atrial fibrillation, surgical correction of atrial septal defect, intracardiac tumor resection, totally endoscopic coronary artery bypass surgery and pericardiectomy. The mean age was 48.39±18.05 years. The mean cardiopulmonary bypass time was 151.7±99.97 minutes, and the mean aortic cross-clamp time was 109.94±81.34 minutes. The mean duration of intubation was 7.52±15.2 hours, and 16 (76.2%) patients were extubated in the operating room immediately after the procedure. The mean length of intensive care unit stay was 1.67±1.46 days. There were no conversions to sternotomy. There was no in-hospital death or deaths during the medium-term follow-up. Patients mean follow up time was 684±346 days, ranging from 28 to 1096 days. Robotic-assisted cardiac surgery proved to be feasible, safe and effective and can be applied in the correction of various intra and extracardiac pathologies.
Conference Paper
Live Surgery is today an integral component of surgical training programs and innovative surgical technique dissemination. However, the increasing pressure to maintain efficiency and reduce surgical risks has raised the need for structured training sessions through simulation technologies. Simulation has the potential to overcome several limitations of live surgery, allowing the trainee to gain procedural experience in a safe and controlled environment. In Total Hip Replacement intervention surgical simulation can help not only in training but also in planning the intervention. In this work the authors present the HipSim/PSP-HipSim physical patient specific simulators and 3D Hip Plugin virtual environment (e-SPres3D s.r.l), and their preliminary validation both for training purposes and for surgical rehearsal. 13 othopaedic surgeons participated in this study and answered a questionnaire. Results demonstrate the appropriateness of the HipSim/PSP-HipSim simulators as training instrument and the effectiveness of the PSP-HipSim and 3D Hip Plugin for patient specific surgical planning.
Background: Robot-assisted coronary artery bypass grafts (RACAB) utilizing the da Vinci surgical system are increasingly used and allow the surgeon to conveniently harvest internal mammary arteries (IMAs). The aim of this study was to compare the outcomes of off-pump RACAB and minimally invasive direct coronary artery bypass grafting (MIDCAB) in the short and medium term. Methods: We performed a retrospective review of 132 patients with single- or multiple-vessel coronary artery disease who underwent minimally invasive off-pump CABG (OPCAB) between May 2009 and May 2014. The patients were divided into two groups based on the surgical approach, MIDCAB and RACAB group. The anastomosis of the left internal mammary artery (LIMA) to the left anterior descending artery (LAD) was performed as regular OPCAB through the incision on the beating heart using regular stabilization devices (Genzyme Corporation). The preoperative, intraoperative, postoperative, and follow-up data, including major adverse cardiac and cerebrovascular events (MACCE), were compared. Results: The preoperative data were similar. RACAB significantly shorten the intensive care unit (ICU) stay and postoperative compared with the MIDCAB group (P<0.05). There were 12 (19.7%) patients treated with a two-stage hybrid procedure in the MIDCAB group and 34 (47.9%) patients in the RACAB group (P=0.001). Thirty-day mortality was 1.6% in the MIDCAB group. There were 9 (14.7%) MIDCAB patients and 2 (2.8%) RACAB patients (P=0.013) that developed new arrhythmia. The two groups showed comparable mid-term survival (P=0.246), but the MACCEs were significantly different (P=0.038). Conclusions: RACAB may be a valuable alternative for patients requiring single or simple multi-vessel coronary artery bypass grafting (CABG). Although the mid-term mortality outcomes are similar, RACAB improves short-term outcomes and mid-term MACCE-free survival compared with MIDCAB.
Optical coherence tomography (OCT) is a catheter-based medical imaging technique that produces cross-sectional images of blood vessels. This technique is particularly useful for studying coronary atherosclerosis. In this paper, we present a new framework that allows a segmentation and quantification of OCT images of coronary arteries to define the plaque type and stenosis grading. These analyses are usually carried out on-line on the OCT-workstation where measuring is mainly operator-dependent and mouse-based. The aim of this program is to simplify and improve the processing of OCT images for morphometric investigations and to present a fast procedure to obtain 3D geometrical models that can also be used for external purposes such as for finite element simulations. The main phases of our toolbox are the lumen segmentation and the identification of the main tissues in the artery wall. We validated the proposed method with identification and segmentation manually performed by expert OCT readers. The method was evaluated on ten datasets from clinical routine and the validation was performed on 210 images randomly extracted from the pullbacks. Our results show that automated segmentation of the vessel and of the tissue components are possible off-line with a precision that is comparable to manual segmentation for the tissue component and to the proprietary-OCT-console for the lumen segmentation. Several OCT sections have been processed to provide clinical outcome.
Mitral valve surgery is the most commonly performed robotically-assisted cardiac surgical procedure. The robotic approach evolved from minimally invasive mitral valve surgery, which was performed via right mini-anterolateral thoracotomy either under direct vision or with endoscopic visualization. The da VinciTM Surgical System (Intuitive Surgical Inc., Sunnydale, CA) has been used in several robotic cardiac surgical centers to successfully perform mitral valve surgery. This system uses high-definition three-dimensional camera imaging and EndowristTM (Intuitive Surgical, Sunnydale, CA) instruments, which allow for motion in six degrees of freedom. When compared with minimally invasive mitral valve surgery, the robotic-assisted approach enables unparalleled visualization of the mitral valve apparatus, tremor-free movements, ambidexterity, and avoidance of the fulcrum effect of using long-shafted endoscopic instruments. The 2 to 3 cm lateral working port incision allows for less pain, quicker recovery, and reduced length of stay when compared with sternotomy. This video article provides a detailed description of our current approach to performing complex mitral valve surgery using the da VinciTM system.