No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Handling and Docking of the Da Vinci Surgical Robot Using Mixed Reality
ResearchGate has not been able to resolve any citations for this publication.
In the world reference context, although virtual reality, augmented reality and mixed reality have been emerging methodologies for several years, only today technological and scientific advances have made them suitable to revolutionize clinical care and medical contexts through the provision of enhanced functionalities and improved health services. This systematic review provides the state-of-the-art applications of the Microsoft® HoloLens 2 in a medical and healthcare context. Focusing on the potential that this technology has in providing digitally supported clinical care, also but not only in relation to the COVID-19 pandemic, studies that proved the applicability and feasibility of HoloLens 2 in a medical and healthcare scenario were considered. The review presents a thorough examination of the different studies conducted since 2019, focusing on HoloLens 2 medical sub-field applications, device functionalities provided to users, software/platform/framework used, as well as the study validation. The results provided in this paper could highlight the potential and limitations of the HoloLens 2-based innovative solutions and bring focus to emerging research topics, such as telemedicine, remote control and motor rehabilitation.
Optical see-through (OST) augmented reality head-mounted displays are quickly emerging as a key asset in several application fields but their ability to profitably assist high precision activities in the peripersonal space is still sub-optimal due to the calibration procedure required to properly model the user's viewpoint through the see-through display. In this work, we demonstrate the beneficial impact, on the parallax-related AR misregistration, of the use of optical see-through displays whose optical engines collimate the computer-generated image at a depth close to the fixation point of the user in the peripersonal space. To estimate the projection parameters of the OST display for a generic viewpoint position, our strategy relies on a dedicated parameterization of the virtual rendering camera based on a calibration routine that exploits photogrammetry techniques. We model the registration error due to the viewpoint shift and we validate it on an OST display with short focal distance. The results of the tests demonstrate that with our strategy the parallax-related registration error is submillimetric provided that the scene under observation stays within a suitable view volume that falls in a ±10 cm depth range around the focal plane of the display. This finding will pave the way to the development of new multi-focal models of OST HMDs specifically conceived to aid high-precision manual tasks in the peripersonal space.
Robot-assisted surgery is assumed to be time consuming partially due to extra time needed in preparing the robot. The objective of this study was to give realistic times in Da Vinci Xi draping and docking and to analyse the learning curve in the transition from the Si to the Xi in an experienced team. This prospective study was held in a hospital with a high volume of robot-assisted surgery in general surgery, urology and gynaecology. Times from the moment patients entered the operating room until the surgeon took place behind console were precisely recorded during the first 6 weeks after the implementation of the Xi. In total, 65 procedures were performed and documented. The learning curve for the process of draping and docking the robot was reached after 21 and 18 cases, respectively. Mean times after completion of the learning curve were 5 min for draping and 7 min for docking and were statistically different from mean times before completion of the learning curve (p values < 0.01). In dedicated teams netto extra time needed for preparing the Xi can even be reduced to just the time needed for docking. Thus, setting up the robot should have limited impact on overall time spent in the operation room.
Mobile augmented reality (mobile AR) enables virtual content such as 3D models, animations and annotations to be placed on top of a real world objects in any context. We applied mobile AR to develop the Calory Battle AR exergame to tackle worldwide childhood obesity. In this game the player finds and defuses virtual calory bombs in a real world environment. Specifically, we present the development of two game versions. First prototype was created without a third party game engine and it led to many challenges. To explore solutions to these challenges, we created a new version of game with the Unity 3D game engine. Using the Unity 3D, the game development process was simplified. A mixed-method usability evaluation on children and university students indicated that especially interaction with AR content and user interface clarity were improved in the Unity 3D version. This study produced three important contributions: 1) a novel mobile AR exergame to motivate children to move; 2) reimplementation of the game using the Unity 3D; and 3) results of a usability evaluation comparing two game versions. We expect that game engines such as the Unity 3D will become essential for AR game development in the future.
Set-up and docking of the da Vinci surgical system are assumed to extend overall operating times. We hypothesized that these tasks could be achieved in adequate times. Therefore, a prospective analysis of set-up and docking times of the da Vinci Surgical System was conducted.
We prospectively analysed set-up and docking times with the da Vinci surgical system in our division.
Ninety-six patients were operated on over 30 months in our institution. Median set-up time was 22 (range 9-50) min and median docking time was 10 (range 2-70) min. Surgeons with previous docking experience were significantly faster than inexperienced surgeons: 8 (range 2-50) vs. 17.5 (range 10-70) min. Both set-up and docking showed a fast learning curve.
The data support the conclusion that both set-up and docking of the robot can be achieved in adequate times and have a low impact on overall operating time.
Laparoscopic surgery has revolutionized the concept of minimally invasive surgery for the last 3 decades. Robotic-assisted surgery is one of the latest innovations in the field of minimally invasive surgery. Already, many procedures have been performed in urology, cardiac surgery, and general surgery. In this article, we attempt to report our preliminary experience with robotic-assisted laparoscopy in a variety of gynecological surgeries. We sought to evaluate the role of robotic-assisted laparoscopy in gynecological surgeries.
The study was a case series of 15 patients who underwent various gynecologic surgeries for combined laparoscopic and robotic-assisted laparoscopic surgery. The da Vinci robot was used in each case at a tertiary referral center for laparoscopic gynecologic surgery. An umbilicus, suprapubic, and 2 lateral ports were inserted. These surgeries were performed both using laparoscopic and robotic-assisted laparoscopic techniques. The assembly and disassembly time to switch from laparoscopy to robotic-assisted surgery was measured. Subjective advantages and disadvantages of using robotic-assisted laparoscopy in gynecological surgeries were evaluated.
Fifteen patients underwent a variety of gynecologic surgeries, such as myomectomies, treatment of endometriosis, total and supracervical hysterectomy, ovarian cystectomy, sacral colpopexy, and Moskowitz procedure. The assembly time to switch from laparoscopy to robotic-assisted surgery was 18.9 minutes (range, 14 to 27), and the disassembly time was 2.1 minutes (range, 1 to 3). Robotic-assisted laparoscopy acts as a bridge between laparoscopy and laparotomy but has the disadvantage of being costly and bulky.
Robotic-assisted laparoscopic surgeries have advantages in providing a 3-dimensional visualization of the operative field, decreasing fatigue and tension tremor of the surgeon, and added wrist motion for improved dexterity and greater surgical precision. The disadvantages include enormous cost and added operating time for assembly and disassembly and the bulkiness of the equipment.
In 1997, Azuma published a survey on augmented reality (AR). Our
goal is to complement, rather than replace, the original survey by
presenting representative examples of the new advances. We refer one to
the original survey for descriptions of potential applications (such as
medical visualization, maintenance and repair of complex equipment,
annotation, and path planning); summaries of AR system characteristics
(such as the advantages and disadvantages of optical and video
approaches to blending virtual and real, problems in display focus and
contrast, and system portability); and an introduction to the crucial
problem of registration, including sources of registration error and
error-reduction strategies
Egocentric augmented reality (AR) interfaces are quickly becoming a key asset for assisting high precision activities in the peripersonal space in several application fields. In these applications, accurate and robust registration of computer-generated information to the real scene is hard to achieve with traditional Optical See-Through (OST) displays given that it relies on the accurate calibration of the combined eye-display projection model. The calibration is required to efficiently estimate the projection parameters of the pinhole model that encapsulate the optical features of the display and whose values vary according to the position of the user's eye. In this work, we describe an approach that prevents any parallax-related AR misregistration at a pre-defined working distance in OST displays with infinity focus; our strategy relies on the use of a magnifier placed in front of the OST display, and features a proper parameterization of the virtual rendering camera achieved through a dedicated calibration procedure that accounts for the contribution of the magnifier. We model the registration error due to the viewpoint parallax outside the ideal working distance. Finally, we validate our strategy on a OST display, and we show that sub-millimetric registration accuracy can be achieved for working distances of ± 100 mm around the focal length of the magnifier.
In recent years optical see-through head-mounted displays (OST-HMDs) have moved from conceptual research to a market of mass-produced devices with new models and applications being released continuously. It remains challenging to deploy augmented reality (AR) applications that require consistent spatial visualization. Examples include maintenance, training and medical tasks, as the view of the attached scene camera is shifted from the user's view. A calibration step can compute the relationship between the HMD-screen and the user's eye to align the digital content. However, this alignment is only viable as long as the display does not move, an assumption that rarely holds for an extended period of time. As a consequence, continuous recalibration is necessary. Manual calibration methods are tedious and rarely support practical applications. Existing automated methods do not account for user-specific parameters and are error prone. We propose the combination of a pre-calibrated display with a per-frame estimation of the user's cornea position to estimate the individual eye center and continuously recalibrate the system. With this, we also obtain the gaze direction, which allows for instantaneous uncalibrated eye gaze tracking, without the need for additional hardware and complex illumination. Contrary to existing methods, we use simple image processing and do not rely on iris tracking, which is typically noisy and can be ambiguous. Evaluation with simulated and real data shows that our approach achieves a more accurate and stable eye pose estimation, which results in an improved and practical calibration with a largely improved distribution of projection error.
Background
The da Vinci robotic surgical telemanipulator has been utilized in several surgical specialties for varied procedures, and the users' experiences have been widely published. To date, no detailed system technical analyses have been performed. MethodsA detailed review was performed of all publications and patents about the technical aspects of the da Vinci robotic system. ResultsPublished technical literature on the da Vinci system highlight strengths and weaknesses of the robot design. While the system facilitates complex surgical operations and has a low malfunction rate, the lack of haptic (especially tactile) feedback and collisions between the robotic arms remain the major limitations of the system. Accurate, preplanned positioning of access ports is essential. Conclusion
Knowledge of the technical aspects of the da Vinci robot is important for optimal use. We confirmed the excellent system functionality and ease of use for surgeons without an engineering background. Research and development of the surgical robot has been predominant in the literature. Future trends address robot miniaturization and intelligent control design. Copyright (c) 2012 John Wiley & Sons, Ltd.
The use of a Unimation Puma 200 robot, properly interfaced with a computerized tomographic (CT) scanner and with a probe guide mounted at its end effector for CT-guided brain tumor biopsis is discussed. Once the target is identified on the CT picture, a simple command allows the robot to move to a position such that the end-effector probe guide points toward the target. This results in a procedure faster than one using a manually adjustable frame. Probably the most important advantage, as is shown, is the improved accuracy that can be achieved by proper calibration of the robot.
To review the history, development, and current applications of robotics in surgery.
Surgical robotics is a new technology that holds significant promise. Robotic surgery is often heralded as the new revolution, and it is one of the most talked about subjects in surgery today. Up to this point in time, however, the drive to develop and obtain robotic devices has been largely driven by the market. There is no doubt that they will become an important tool in the surgical armamentarium, but the extent of their use is still evolving.
A review of the literature was undertaken using Medline. Articles describing the history and development of surgical robots were identified as were articles reporting data on applications.
Several centers are currently using surgical robots and publishing data. Most of these early studies report that robotic surgery is feasible. There is, however, a paucity of data regarding costs and benefits of robotics versus conventional techniques.
Robotic surgery is still in its infancy and its niche has not yet been well defined. Its current practical uses are mostly confined to smaller surgical procedures.
Robotic cholecystectomy is safe, feasible procedure. Initial studies showed significant set up time and operating time but no clear clinical advantage of the robotic involvement. We have investigated the learning curve, advantages and limitation of the procedure.
We reviewed all (n = 51) robotic cholecystectomies performed between July 2004 and December 2005. The surgery was performed using the da Vinci system. We recorded operative time, setup time of robotics instrumentation, conversion to laparoscopic or open cholecystectomy and complication of the procedure.
Forty-eight of the 51 procedures (94%) were completed robotically. We did not experience any significant complications directly related to robotics surgery. The mean +/- SD operating time was 77 +/- 22.3 min. The mean setup time for robotics (from incision until robot was in place, including draping the robot) was 24 +/- 8.8 min. However, the setup time significantly improved as we gained more experience: from 30.6 +/- 10.7 min (first 16 cases) to 18.3 +/- 4.0 min (cases 33-48). The mean robotic time was 34 +/- 16.1 min. We observed no significant improvement in robotic procedure time.
Robotic cholecystectomy offers significant advantages such as three-dimensional view, easier instrument manipulations and possibility of remote site surgery. We observed some shortcomings of robotic surgery such as need for larger and additional ports, and need for undocking the machine in case of cholangiography or change of patient position. Our data shows that the learning curve is between 16 to 32 procedures to significantly decrease the setup time and total operating time.
Recent advances in augmented reality
- R Azuma
- Y Baillot
- R Behringer
- S Feiner
- S Julier
- B Macin-Tyre