Figure 1 - uploaded by Christian Hansen
Content may be subject to copyright.
Pointer Ray. An instrument is virtually extended by a ray pointing at potential collisions along the trajectory. The distance to detected objects is encoded by the line color.
Source publication
Image-guided surgery near anatomical or functional risk structures poses a challenging task for surgeons. To this end, surgical navigation systems that visualize the spatial relation between patient anatomy (represented by 3D images) and surgical instruments have been described. The provided 3D visualizations of these navigation systems are often c...
Context in source publication
Context 1
... Therefore, we implemented a color scale that adapts to the length of the pointer ray itself. Since we were mostly interested in illustrating proximity ranges, we used a discrete scale stretching from green (large distance) over yellow (medium distance) to red (small distance). The corresponding thresholds can be adjusted for each use case. Fig. 1 shows the concept with a virtual instrument pointing towards a target structure. The ray changes in appearance with respect to the distance to the structure. ...
Similar publications
Purpose
Minimally invasive alternatives are now available for many complex surgeries. These approaches are enabled by the increasing availability of intra-operative image guidance. Yet, fluoroscopic X-rays suffer from projective transformation and thus cannot provide direct views onto anatomy. Surgeons could highly benefit from additional informati...
Citations
... However, when assistive visualizations were included, spatial perception was superior for some digital configurations compared to the real world [67]. Approaches to improve spatial cognition are mostly based on improvements in visual keys [27]. Within our study we are interested if visualization can have similar improvements for distance and shape estimation. ...
Figure 1: Example of the shape estimation task with the vascular model and Phong visualization. The hand is about to grab and align the arrow. ABSTRACT Since for most consumers the Virtual Reality (VR) experience exceeds that of desktop applications, an increasing number of applications is being transferred from desktop to VR. Industrial and entertainment applications primarily expect for a richer consumer experience, while others, such as surgical applications, seek for improved precision over their desktop counterparts. One way to improve the performance of precision-based VR applications is to provide suitable visualizations. Today, these "suitable" visualizations are mostly transferred from desktop to VR without considering their spatial and temporal performance might change in VR. This may not lead to an optimal solution, which can be crucial for precision-based tasks. Misinterpretation of shape or distance in a surgical or pre-operative simulation can affect the chosen treatment and thus directly impact the outcome. Therefore, we evaluate the performance differences of multiple visualizations for 3D surfaces based on their shape and distance estimation for desktop and VR applications. We conducted a quantitative user study with 56 participants evaluating seven visualizations (Phong, Toon, Fresnel, Pseudo-Chromadepth, Heatmap, Isolines, and Arrow Glyphs). Our results show that the performance of each visualization varies depending on the task, system , and surface type, with VR generally providing improved results. While Isolines are able to improve distance estimation, Phong and Heatmaps are beneficial for shape estimation.
... Visualisations have consisted of 2D and 3D content. A challenge with 3D content is the complexity of the model, which can increase the mental load of the surgeon [6]. ...
Augmented reality (AR) has shown much potential when applied in surgical settings, which can help guide surgeons through complex procedures, train students, and provide heads-up and hands-free spatial information. In this position paper, we discuss some of the current use cases of AR in surgical practice, evaluation measures, challenges and potential directions for future research. The aim of this paper is to start important discussion to improve future research and outcomes for system implementations for surgery.
... Winne et al. [42] used a guiding line during endoscopy, indicating the distance of the target and pointer instrument by changing its colour every millimetre. Heinrich et al. [15] also introduced a colour-coded pointer ray from the tip of the instrument aligned with the tool axis. To detect surrounding structures, a side-looking radar, rotating around the instrument's tip, was developed. ...
Purpose
The treatment of cerebral aneurysms shifted from microsurgical to endovascular therapy. But for some difficult aneurysm configurations, e.g. wide neck aneurysms, microsurgical clipping is better suited. From this combination of limited interventions and the complexity of these cases, the need for improved training possibilities for young neurosurgeons arises.
Method
We designed and implemented a clipping simulation that requires only a monoscopic display, mouse and keyboard. After a virtual craniotomy, the user can apply a clip at the aneurysm which is deformed based on a mass–spring model. Additionally, concepts for visualising distances as well as force were implemented. The distance visualisations aim to enhance spatial relations, improving the navigation of the clip. The force visualisations display the force acting on the vessel surface by the applied clip. The developed concepts include colour maps and visualisations based on rays, single objects and glyphs.
Results
The concepts were quantitatively evaluated via an online survey and qualitatively evaluated by a neurosurgeon. Regarding force visualisations, a colour map is the most appropriate concept. The necessity of distance visualisations became apparent, as the expert was unable to estimate distances and to properly navigate the clip. The distance rays were the only concept supporting the navigation appropriately.
Conclusion
The easily accessible surgical training simulation for aneurysm clipping benefits from a visualisation of distances and simulated forces.
... Purely mid-air 6-DoF alignment does not allow for pure separation of these two phases and so novel forms of visualization are needed. Another aspect of alignment involving physical surfaces is that they allow for the use of projector-based AR [19][20][21]. We use a similar concept of projection in some of our interfaces by projecting a virtual light from the hand-held object onto a virtual plane floating in space. ...
... However, such interfaces rely on rendering virtual reflections of objects that are near the instrument, which is less applicable to mid-air alignment in open space. Another approach uses animations to show depth [11,21]. However, this requires the user to wait while the animation plays to receive guidance, which can increase physical exertion. ...
... One method is to illustrate distances from the instrument tip to its surroundings by connecting lines [Choi et al. 2016;De Paolis and De Luca 2018]. More concepts were investigated in preliminary work, i.e., a virtual instrument extension to detect distances ahead of the instrument, a radar line scanning for nearby objects and regularly lighting up lines connecting the instrument tip with selected points of interest [Heinrich et al. 2019]. These advances were shown to be beneficial for the estimation of spatial relations but were mostly displayed on monitors. ...
... The first concept was adapted from preliminary work [Heinrich et al. 2019]. The Pointer is a virtual extension of the instrument and indicates the distance of the tool to the closest anatomical structure along its trajectory. ...
The estimation of distances and spatial relations between surgical instruments and surrounding anatomical structures is a challenging task for clinicians in image-guided surgery. Using augmented reality (AR), navigation aids can be displayed directly at the intervention site to support the assessment of distances and reduce the risk of damage to healthy tissue. To this end, four distance-encoding visualisation concepts were developed using a head-mounted optical see-through AR setup and evaluated by conducting a comparison study. Results suggest the general advantage of the proposed methods compared to a blank visualisation providing no additional information. Using a Distance Sensor concept signalising the proximity of nearby structures resulted in the least time the instrument was located below 5mm to surrounding risk structures and yielded the least amount of collisions with them.
... In Vorarbeiten von Heinrich et al. wurden verschiedene Konzepte zur Verbesserung der räumlichen Wahrnehmung von Instrumenten und umliegenden Strukturen innerhalb von 3D-Darstellungen entwickelt und untersucht [25]. Die Konzepte konzentrierten sich auf die Hinzugabe von visuellen Hinweisen durch farbkodierte Linien zwischen Instrument und umliegende Strukturen. ...
... Die vorgestellten Navigationskonzepte kombinieren zuvor untersuchte Konzepte, passen sie an oder ergänzen sie. Die Konzepte Pointer und Radar beschreiben in ihrer originalen Form stark reduzierte Visualisierungen [25]. ...
... Damit verhindert werden kann, dass Strukturen ungewollt mit der Spitze verletzt werden, ist es wichtig, dass der Benutzer die Entfernung zwischen Werkzeugspitze und Umgebung einschätzen kann. Ein Konzept für diesen Einsatzbereich wurde von Heinrich et al. bereits in einer Monitor-gestützten Anwendung untersucht[25]. Bei diesem Konzept wird das verwendete Instrument entlang der Spitze künstlich mit Hilfe eines Strahls verlängert, bis dieser auf umliegende Strukturen trifft. Um diesen Abstand besser einschätzen zu können wurde der Strahl zusätzlich in diskreten Stufen farbkodiert. ...
The assessment of distances and spatial relationships between medical instruments and surrounding risk structures is a challenging task for a surgeon in image-guided surgery. Using augmented reality, navigation instructions with suitable visualizations can be displayed directly in the surgeon's field of view to support a better hand-eye coordination. For this purpose, various instrument-based visualizations were developed within this thesis in order to be able to better assess distances and spatial relations. These were then implemented as part of a prototype augmented reality application.
In an evaluation two concepts could be identified which allow a maximization of the distance to risk structures and a reduction of collisions. Furthermore, it could be shown that the developed concepts fulfil basic requirements for usability, though a shorter processing time could only be achieved to a limited extent within the scope of this work. The developed augmented reality prototype shows first improvement approaches, which set a base for further investigations with more sophisticated hardware.
Augmented reality (AR) devices are gaining popularity in industrial development and healthcare as they provide information that would not be accessible in a rapid and intuitive way. Head-mounted displays dominate in this field and are currently being comprehensively tested. Alongside its informative function, AR can be used to steer the user's actions to aid complex or high precision tasks. This is the case in surgery, which is recently seeing the development of ad-hoc head-mounted displays to meet the requirements of safety, ergonomics, and reliability. However, head-mounted displays are subject to perceptual problems that can affect their use in delicate and demanding tasks. This article aims to evaluate projected AR as an alternative to head mounted displays (HMDs) when accurate guidance on the surface is needed. We directly compare them in a user study and evaluate both user accuracy and user perception to assess whether projected AR can be a practical and useful paradigm for precision manual tasks. Ten users performed tracing trajectory tasks under the guidance of an HMD and a projected AR device. Three accuracy levels were quantitatively tested: 0.5, 1, and 2 mm. Statistical analysis showed no significant difference in the accuracy of the two AR visualization modes, whereas the user perception assessment revealed statistical differences in virtual-to-real perception and visual discomfort. The quantitative results of this article proved that both technologies can guide manual precision tasks with the same accuracy, but projected AR features some perceptual advantages.
