Marta A. Kersten-Oertel

Concordia University Montreal · Department of Computer Science and Software Engineering

Augmented reality has the potential to aid surgeons with particular surgical tasks in image-guided surgery. In augmented reality (AR) visualization for neurosurgery, the live view of the surgical scene is merged with preoperative patient data, aiding the surgeon in mapping patient images from the image guidance system to the real patient. Furthermo...

The aim of this report is to present a prototype augmented reality (AR) intra-operative brain imaging system. We present our experience of using this new neuronavigation system in neurovascular surgery and discuss the feasibility of this technology for aneurysms, arteriovenous malformations (AVMs), and arteriovenous fistulae (AVFs). We developed an...

Cerebral vascular images obtained through angiography are used by neurosurgeons for diagnosis, surgical planning, and intraoperative guidance. The intricate branching of the vessels and furcations, however, make the task of understanding the spatial three-dimensional layout of these images challenging. In this paper, we present empirical studies on...

This paper presents a review of the state of the art of visualization in mixed reality image guided surgery (IGS). We used the DVV (data, visualization processing, view) taxonomy to classify a large unbiased selection of publications in the field. The goal of this work was not only to give an overview of current visualization methods and techniques...

Mixed reality visualizations are increasingly studied for use in image guided surgery (IGS) systems, yet few mixed reality systems have been introduced for daily use into the operating room (OR). This may be the result of several factors: the systems are developed from a technical perspective, are rarely evaluated in the field, and/or lack consider...

In breast reconstruction after a single mastectomy, the surgeon must choose from hundreds of implants to select the one that best replicates the patient’s natural breast. Due to a lack of measurement tools, the surgeon must depend on their previous experience to visually choose the best implant, leading them to compare and use numerous implants to...

Clinical assessments are an integral part of the care and management of Parkinson’s disease, but full spectral assessments are difficult to obtain consistently, especially at follow-up visits. To better understand the etiology and pathogenesis of the disease and to offer accurate prognosis and tailored treatment plans, data-driven computational met...

In this paper, we evaluate the effect of depth cues on the perception of three-dimensional cerebral angiographic data in a virtual reality (VR) environment. Specifically, a user study was conducted to evaluate the effectiveness of shading, pseudo-chromadepth and aerial perspective, both with and without a dynamic component, where the volume renderi...

Deep brain stimulation (DBS) is an effective surgical treatment for Parkinson’s disease. The procedure requires precise placement of a stimulation electrode into the therapeutic target while avoiding vital anatomies (e.g. blood vessels) to prevent surgical risks. Therefore, multi-contrast imaging data are often employed to capture full anatomical d...

The evolution of consumer-grade hardware components (e.g., trackers, portable ultrasound probes) has opened the door for the development of low cost systems. We evaluated different low-cost tracking alternatives on the accuracy of 3D freehand ultrasound reconstruction in the context of image-guided neurosurgery. Specifically, we compared two low-co...

Federated Learning using the Federated Averaging algorithm has shown great advantages for large-scale applications that rely on collaborative learning, especially when the training data is either unbalanced or inaccessible due to privacy constraints. We hypothesize that Federated Averaging underestimates the full extent of heterogeneity of data whe...

One of the main challenges of gaze-based interactions is the ability to distinguish normal eye function from a deliberate interaction with the computer system, commonly referred to as ‘Midas touch’. In this paper we propose EyeTAP (Eye tracking point-and-select by Targeted Acoustic Pulse) a contact-free multimodal interaction method for point-and-s...

Neuronavigation using pre-operative imaging data for neurosurgical guidance is a ubiquitous tool for the planning and resection of oncologic brain disease. These systems are rendered unreliable when brain shift invalidates the patient-image registration. Our previous review in 2015, Brain shift in neuronavigation of brain tumours: A review offered...

In this paper, we explore how new technologies can be used to improve Multiple Sclerosis (MS) treatments. Treatment of MS most often includes self-injecting medicine into the subcutis, the tissue layer between the skin and the muscle. The injections can make a patient’s skin sore, red, itchy, and even cause pain, thus many patients suffer from inje...

Assessment of collateral circulation in ischemic stroke, which can identify patients for the most appropriate treatment strategies, is currently conducted with visual inspection by a radiologist. Yet numerous studies have shown that visual inspection suffers from inter and intra-rater variability. We present an automatic evaluation of collaterals u...

PurposeSufficient collateral blood supply is crucial for favorable outcomes with endovascular treatment. The current practice of collateral scoring relies on visual inspection and thus can suffer from inter and intra-rater inconsistency. We present a robust and automatic method to score cerebral collateral blood supply to aid ischemic stroke treatm...

Purpose: Neuronavigation systems making use of augmented reality (AR) have been the focus of much research in the last couple of decades. In recent years, there has been considerable interest in using mobile devices for AR in the operating room (OR). We propose a complete system that performs real-time AR video augmentation on a mobile device in t...

One of the main challenges of gaze-based interactions is the ability to distinguish normal eye function from a deliberate interaction with the computer system, commonly referred to as 'Midas touch'. In this paper we propose, EyeTAP (Eye tracking point-and-select by Targeted Acoustic Pulse) a hands-free interaction method for point-and-select tasks....

Stroke is the second leading cause of disability worldwide. In order to minimize disability, the goal of stroke treatment is to preserve tissue in the area where blood supply is decreased but sufficient to stave off cell death. Thrombectomy has been shown to offer fast and efficient reperfusion with high recanalization rates. However, due to the in...

In breast reconstruction following a single mastectomy, the surgeon needs to choose between tens of available implants to find the one that can reproduce the symmetry of the patient's breasts. However, due to the lack of measurement tools this decision is made purely visually, which means the surgeon has to order multiple implants to confirm the si...

We propose a decentralized conceptual marketplace model for IoT generated personal data. Our model is based on a thorough analysis of personal data in a marketplace context, with specific focus on the challenges presented by commercializing IoT generated personal data. Our model introduces a novel perspective on the commercialization of personal da...

Purpose The combination of data visualization and auditory display (e.g., sonification) has been shown to increase accuracy, and reduce perceived difficulty, within 3D navigation tasks. While accuracy within such tasks can be measured in real time, subjective impressions about the difficulty of a task are more elusive to obtain. Prior work utilizin...

Ultrasound (US) is a popular medical imaging technique in the clinic due to its low cost, high portability, and real-time diagnostic value. A special type of ultrasound technique, transcranial Doppler (TCD) ultrasound can be used to measure blood flows in cerebral blood vessels through acoustic bone windows of the intact human skull. Although TCD u...

In image-guided neurosurgery, a registration between the patient and their pre-operative images and the tracking of surgical tools enables GPS-like guidance to the surgeon. However, factors such as brainshift, image distortion, and registration error cause the patient-to-image alignment accuracy to degrade throughout the surgical procedure no longe...

Recent developments pertaining to ear-mounted wearable computer interfaces (i.e., “hearables”) offer a number of distinct affordances over other wearable devices in ambient and ubiquitous computing systems. This paper provides a survey of hearables and the possibilities that they offer as computer interfaces. Thereafter, these affordances are exami...

We present our work investigating the feasibility of combining intraoperative ultrasound for brain shift correction and augmented reality (AR) visualization for intraoperative interpretation of patient-specific models in image-guided neurosurgery (IGNS) of brain tumors. We combine two imaging technologies for image-guided brain tumor neurosurgery....

Stroke is the second leading cause of disability worldwide. Thrombectomy has been shown to offer fast and efficient reperfusion with high recanalization rates and thus improved patient outcomes. One of the most important indicators to identify patients amenable to thrombectomy is evidence of good collateral circulation. Currently, methods for evalu...

Image-guided neurosurgery, or neuronavigation, has been used to visualize the location of a surgical probe by mapping the probe location to pre-operative models of a patient's anatomy. One common limitation of this approach is that it requires the surgeon to divert their attention away from the patient and towards the neuronavigation system. In ord...

Image-guided surgery (IGS) has allowed for more minimally invasive procedures where smaller incisions are used, leading to better patient outcomes, a reduced risk of infection, less pain, shorter hospital stays and faster recoveries. One drawback that has emerged with image-guided surgery techniques, is that the surgeon must shift their attention f...

Stroke is the second leading cause of disability worldwide. In order to minimize disability, the goal of stroke treatment is to preserve tissue in the area where blood supply is decreased but sufficient to stave off cell death. Thrombectomy has been shown to offer fast and efficient reperfusion with high recanalization rates. However, due to the in...

Purpose: Navigation systems commonly used in neurosurgery suffer from two main drawbacks: (1) their accuracy degrades over the course of the operation and (2) they require the surgeon to mentally map images from the monitor to the patient. In this paper, we introduce the Intraoperative Brain Imaging System (IBIS), an open-source image-guided neuro...

Purpose: Neuronavigation based on preoperative imaging data is a ubiquitous tool for image guidance in neurosurgery. However, it is rendered unreliable when brain shift invalidates the patient-to-image registration. Many investigators have tried to explain, quantify, and compensate for this phenomenon to allow extended use of neuronavigation system...

Augmented reality has been proposed as a solution to overcome some of the current shortcomings of image-guided neurosurgery. In particular, it has been used to merge patient images, surgical plans, and the surgical field of view into a comprehensive visualization. In this paper we explore the use of augmented reality for planning craniotomies in im...

One of the main challenges facing members of the medical imaging community is the lack of real clinical cases and ground truth datasets with which to validate new registration, segmentation, and other image processing algorithms. In this work we present a collection of data from tumour patients acquired at the Montreal Neurological Institute and Ho...

We present our work to combine intraoperative ultrasound imaging and augmented reality visualization to improve the use of patient specific models throughout image-guided neurosurgery in the context of tumour resections. Preliminary results in a study of 3 patients demonstrate the successful combination of the two technologies as well as improved a...

In image-guided neurosurgery the patient is registered with the reference of a tracking system and preoperative data before sterile draping. Due to several factors extensively reported in the literature, the accuracy of this registration can be much deteriorated after the initial phases of the surgery. In this paper, we present a simple method that...

In neurovascular surgery, the surgeon must navigate among eloquent areas, through complex neurovascular anatomy to a particular vascular malformation or anomaly. Augmented reality (AR) visualization may be used to show vessels not visible when looking at the brain surface and to aid navigation by bringing into spatial alignment pre-operative vascul...

pWe present empirical studies that consider the effects of stereopsis and simulated aerial perspective on depth perception in translucent volumes. We consider a purely absorptive lighting model, in which light is not scattered or reflected, but is simply absorbed as it passes through the volume. A purely absorptive lighting model is used, for examp...

In neurovascular image-guided surgery, surgeons use pre-operative vascular data sets (from angiography) to guide them. They map information from angiography images onto the patient on the operating room table to localize important vessels. This spatial mapping is complex, time consuming and prone to error. We’ve developed an augmented reality (AR)...

Cerebral arteriovenous malformations (AVMs) are a type of vascular anomaly consisting of large intertwined vascular growth (the nidus) that are prone to serious hemorrhaging and can result in patient death if left untreated. Intervention through surgical clipping of feeding and draining vessels to the nidus is a common treatment. However, identific...

In neurovascular surgery, and in particular surgery for arteriovenous malformations (AVMs), the surgeon maps pre-operative images to the patient on the operating table to aid in vessel localization and resection. This type of spatial mapping is not trivial, is time consuming, and may be prone to error. Using augmented reality (AR) we can register t...

In a mixed reality visualization, physical and virtual environments are merged to produce new visualizations where both real and virtual objects are displayed together. In image guided surgery (IGS), surgical tools and data sets are fused into a mixed reality visualization providing the surgeon with a view beyond the visible anatomical surface of t...

We introduce the DVV taxonomy (Data, Visualization processing, View) which defines each of the major components of a mixed reality image-guided surgery (IGS) system. We propose that these components should be considered and in turn validated for acceptance of a system into the operating room. A taxonomy of IGS visualization systems is a step toward...

The Mobile Emergency Triage (MET) system is a clinical triage support system that aids physicians in making triage decisions as to whether a child presenting in the Emergency Department of a hospital with a specific pain complaint should be discharged to a family physician, needs to be admitted for further investigation/observation, or requires an...

We present empirical studies that consider the effects of stereopsis and simulated aerial perspective on depth perception in translucent volumes. We consider a purely absorptive lighting model, in which light is not scattered or reflected, but is simply absorbed as it passes through the volume. A purely absorptive lighting model is used, for exampl...

Previous CT-based methods of measuring acetabular coverage of the femoral head have either been labor-intensive or have required extensive preprocessing of the data prior to visualization. We propose a method of measuring acetabular coverage using stereoscopic digitally reconstructed radiographs that required very little labor or image preprocessin...