Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography

Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, 22 South Greene Street, Baltimore, MD 21201, USA.
Surgical Endoscopy (Impact Factor: 3.31). 02/2010; 24(8):1976-85. DOI: 10.1007/s00464-010-0890-8
Source: PubMed

ABSTRACT Current laparoscopic images are rich in surface detail but lack information on deeper structures. This report presents a novel method for highlighting these structures during laparoscopic surgery using continuous multislice computed tomography (CT). This has resulted in a more accurate augmented reality (AR) approach, termed "live AR," which merges three-dimensional (3D) anatomy from live low-dose intraoperative CT with live images from the laparoscope.
A series of procedures with swine was conducted in a CT room with a fully equipped laparoscopic surgical suite. A 64-slice CT scanner was used to image the surgical field approximately once per second. The procedures began with a contrast-enhanced, diagnostic-quality CT scan (initial CT) of the liver followed by continuous intraoperative CT and laparoscopic imaging with an optically tracked laparoscope. Intraoperative anatomic changes included user-applied deformations and those from breathing. Through deformable image registration, an intermediate image processing step, the initial CT was warped to align spatially with the low-dose intraoperative CT scans. The registered initial CT then was rendered and merged with laparoscopic images to create live AR.
Superior compensation for soft tissue deformations using the described method led to more accurate spatial registration between laparoscopic and rendered CT images with live AR than with conventional AR. Moreover, substitution of low-dose CT with registered initial CT helped with continuous visualization of the vasculature and offered the potential of at least an eightfold reduction in intraoperative X-ray dose.
The authors proposed and developed live AR, a new surgical visualization approach that merges rich surface detail from a laparoscope with instantaneous 3D anatomy from continuous CT scanning of the surgical field. Through innovative use of deformable image registration, they also demonstrated the feasibility of continuous visualization of the vasculature and considerable X-ray dose reduction. This study provides motivation for further investigation and development of live AR.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Autostereoscopic 3D image overlay for augmented reality (AR) based surgical navigation has been studied and reported many times. For the purpose of surgical overlay, the 3D image is expected to have the same geometric shape as the original organ, and can be transformed to a specified location for image overlay. However, how to generate a 3D image with high geometric fidelity and quantitative evaluation of 3D image's geometric accuracy have not been addressed. This paper proposes a graphics processing unit (GPU) based computer-generated integral imaging pipeline for real-time autostereoscopic 3D display, and an automatic closed-loop 3D image calibration paradigm for displaying undistorted 3D images. Based on the proposed methods, a novel AR device for 3D image surgical overlay is presented, which mainly consists of a 3D display, an AR window, a stereo camera for 3D measurement, and a workstation for information processing. The evaluation on the 3D image rendering performance with 2560 × 1600 elemental image resolution shows the rendering speeds of 50-60 frames per second (fps) for surface models, and 5-8 fps for large medical volumes. The evaluation of the undistorted 3D image after the calibration yields sub-millimeter geometric accuracy. A phantom experiment simulating oral and maxillofacial surgery was also performed to evaluate the proposed AR overlay device in terms of the image registration accuracy, 3D image overlay accuracy, and the visual effects of the overlay. The experimental results show satisfactory image registration and image overlay accuracy, and confirm the system usability.
    Computerized Medical Imaging and Graphics 11/2014; DOI:10.1016/j.compmedimag.2014.11.003 · 1.50 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Surgery, interventional radiology, and advanced endoscopy have all developed minimally invasive techniques to effectively treat a variety of diseases with positive impact on patients' postoperative outcomes. However, those techniques are challenging and require extensive training. Robotics and computer sciences can help facilitate minimally invasive approaches. Furthermore, surgery, advanced endoscopy, and interventional radiology could converge towards a new hybrid specialty, hybrid image-guided minimally invasive therapies, in which the three fundamental disciplines could complement one another to maximize the positive effects and reduce the iatrogenic footprint on patients. The present manuscript describes the fundamental steps of this new paradigm shift in surgical therapies that, in our opinion, will be the next revolutionary step in minimally invasive approaches.
    Journal of Pediatric Surgery 10/2014; 50(1). DOI:10.1016/j.jpedsurg.2014.10.022 · 1.31 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Objetivo: estabelecer padrão individual de pressão de manutenção do pneumoperitônio artificial em cirurgia videolaparoscópica. Métodos: estudo clínico prospectivo com trinta pacientes de ambos os sexos submetidos à videolaparoscopia. Foi medida a distância da extremidade inferior do apêndice xifóide à borda superior do púbis, e aferiram-se os valores da pressão intratraqueal, da pressão intraperitoneal e do volume de gás carbônico insuflado durante a criação do pneumoperitônio artificial. Os dados foram coletados nos seguintes momentos: ausência de pneumoperitônio, pneumoperitônio artificial com pressões de 4mmHg, 6mmHg, 8mmHg, 10mmHg, 12mmHg, 14mmHg, 16mmHg, 18mmHg e 20mmHg. Resultados: As medidas da distância xifopúbica mostraram relação direta com as pressões intraperitoneais obtidas na ausência de pneumoperitônio e nos momentos subseqüentes até atingir a pressão intraperitoneal de 20mmHg. As curvas correspondentes a essas variáveis evidenciaram três fases. Na primeira - ausência de pneumoperitônio até o nível de 6mmHg houve pouca alteração da medida xifopúbica. Na segunda - pressão intraperitoneal de 6mmHg a 12mmHg observou-se acentuado aumento da variação da medida xifopúbica. Na terceira fase de 12mmHg a 20mmHg - ocorreu pequeno aumento na medida xifopúbica, apesar de grandes elevações da pressão intraperitoneal. Conclusões: A pressão adequada de manutenção de pneumoperitônio para cirurgia videolaparoscópica será aquela aferida pelo insuflador quando se obtém acréscimo de 8% na medida xifopúbica mensurada com o paciente anestesiado e na ausência de pneumoperitônio
    03/2011, Degree: Mestrado, Supervisor: João Luiz Moreira Coutinho Azevedo