Fabrizio Cutolo

University of Pisa | UNIPI · Department of Information Engineering

Conventional orthopaedic navigation systems depend on marker-based tracking, which may introduce additional skin incisions, increase the risk and discomfort for the patient, and entail increased workflow complexity. The guidance is conveyed via 2D monitors, which may distract the surgeon and increase the cognitive burden. This study presents an Ar...

Head-mounted displays (HMDs) are hands-free devices particularly useful for guiding near-field tasks such as manual surgical procedures. See-through HMDs do not significantly alter the user’s direct view of the world, but the optical merging of real and virtual information can hinder their coherent and simultaneous perception. In particular, the co...

For an optical see-through (OST) augmented reality (AR) head-mounted display (HMD) to assist in performing high-precision activities in the peripersonal space, a fundamental requirement is the correct spatial registration between the virtual information and the real environment. This registration can be achieved through a calibration procedure invo...

Recent research studies reported that the employment of wearable augmented reality (AR) systems such as head‐mounted displays for the in situ visualisation of ultrasound (US) images can improve the outcomes of US‐guided biopsies through reduced procedure completion times and improved accuracy. Here, the authors continue in the direction of recent d...

Achieving and maintaining proper image registration accuracy is an open challenge of image-guided surgery. This work explores and assesses the efficacy of a registration sanity check method for augmented reality-guided navigation (AR-RSC), based on the visual inspection of virtual 3D models of landmarks. We analyze the AR-RSC sensitivity and specif...

Robotic surgery has reached a very high level of advancement, in particular, Da Vinci Xi robotic system allows the execution of surgical tasks of the highest precision. It offers advantages in terms of reduced blood loss, reduced postoperative hospital stay, and less postoperative pain. The disadvantage the surgeon has found is progressive tactile...

The growing interest in augmented reality applications has led to an in-depth look at the performance of head-mounted displays and their testing in numerous domains. Other devices for augmenting the real world with virtual information are presented less frequently and usually focus on the description of the device rather than on its performance ana...

The adoption of extended reality solutions is growing rapidly in the healthcare world. Augmented reality (AR) and virtual reality (VR) interfaces can bring advantages in various medical-health sectors; it is thus not surprising that the medical MR market is among the fastest-growing ones. The present study reports on a comparison between two of the...

In the original article, there was a mistake in the content of Table 2 page 8 column 1 as published. The values of the mean and standard deviation of the virtual-to-real overlay error in visual angles, which are reported for different checkerboard distances, are to be corrected. Due to a typing error within the data analysis code, we mistakenly con...

The growing interest in augmented reality applications has led to an in-depth look at the performance of head-mounted displays and their testing in numerous domains. Other devices for augmenting the real world with virtual information are presented less frequently and usually focus on the description of the device rather than on its performance ana...

In the field of image-guided surgery, Augmented Reality wearable displays are a widely studied and documented technology for their ability to provide egocentric vision together with the overlap between real and virtual content. In particular, optical see-through (OST) displays have the advantage of maintaining visual perception of the real world. H...

Patient-specific templates (PST) have become a useful tool for guiding osteotomy in complex surgical scenarios such as pelvic resections. The design of the surgical template results in sharper, less jagged resection margins than freehand cuts. However, their correct placement can become difficult in some anatomical regions and cannot be verified du...

Augmented Reality (AR) can avoid some of the drawbacks of Minimally Invasive Surgery and may provide opportunities for developing innovative tools to assist surgeons. In laparoscopic surgery, the achievement of easy and sufficiently accurate registration is an open challenge. This is particularly true in procedures, such as laparoscopic abdominal S...

[This corrects the article DOI: 10.3389/frobt.2020.572001.].

In recent years, huge progress has been made in the management of brain tumors, due to the availability of imaging devices, which provide fundamental anatomical and pathological information not only for diagnostic purposes [...]

In the context of a European project, we identified over 150 requirements for the development of an augmented reality (AR) head-mounted display (HMD) specifically tailored to support highly challenging manual surgical procedures. The requirements were established by surgeons from different specialties and by industrial players working in the surgic...

In the context of image-guided surgery, augmented reality (AR) represents a ground-breaking enticing improvement, mostly when paired with wearability in the case of open surgery. Commercially available AR head-mounted displays (HMDs), designed for general purposes, are increasingly used outside their indications to develop surgical guidance applica...

Augmented reality (AR) devices are gaining popularity in industrial development and healthcare as they provide information that would not be accessible in a rapid and intuitive way. Head-mounted displays dominate in this field and are currently being comprehensively tested. Alongside its informative function, AR can be used to steer the user's acti...

Nowadays, ultrasound (US) is increasingly being chosen as imaging modality for both diagnostic and interventional applications, owing to its positive characteristics in terms of safety, low footprint and low cost. The combination of this imaging modality with wearable augmented reality (AR) systems, such as the head-mounted displays (HMD), comes fo...

Visual augmented reality (AR) headsets have the potential to enhance surgical navigation by providing physicians with an egocentric visualization interface capable of seamlessly blending the virtual navigation aid with the real surgical scenario. However, technological and human-factor limitations still hinder the routine use of commercial AR heads...

The inaccessibility of user-perceived reality remains an open issue in pursuing the accurate calibration of optical see-through (OST) head-mounted displays (HMDs). Manual user alignment is usually required to collect a set of virtual-to-real correspondences, so that a default or an offline display calibration can be updated to account for the user’...

Background: This report discusses the utility of a wearable augmented reality platform in neurosurgery for parasagittal and convexity en plaque meningiomas with bone flap removal and custom-made cranioplasty. Methods: A real patient with en plaque cranial vault meningioma with diffuse and extensive dural involvement, extracranial extension into...

Augmented Reality (AR) headsets have become the most ergonomic and efficient visualization devices to support complex manual tasks performed under direct vision. Their ability to provide hands-free interaction with the augmented scene makes them perfect for manual procedures such as surgery. This study demonstrates the reliability of an AR head-mou...

The inaccessibility of user-perceived reality remains an open issue in pursuing the accurate calibration of optical see-through (OST) head-mounted displays (HMDs). Manual user alignment is usually required to collect a set of virtual-to-real correspondences, so that a default or an offline display calibration can be updated to account for the user'...

Wearable Video See-Through (VST) devices for Augmented Reality (AR) and for obtaining a Magnified View are taking hold in the medical and surgical fields. However, these devices are not yet usable in daily clinical practice, due to focusing problems and a limited depth of field. This study investigates the use of liquid-lens optics to create an aut...

Today, neuronavigation is widely used in daily clinical routine to perform safe and efficient surgery. Augmented reality (AR) interfaces can provide anatomical models and preoperative planning contextually blended with the real surgical scenario, overcoming the limitations of traditional neuronavigators. This study aims to demonstrate the reliabili...

Cryosurgery is a technique of growing popularity involving tissue ablation under controlled freezing. Technological advancement of devices along with surgical technique improvements have turned cryosurgery from an experimental to an established option for treating several diseases. However, cryosurgery is still limited by inaccurate planning based...

Visual augmented reality (AR) has the potential to improve the accuracy, efficiency and reproducibility of computer-assisted orthopaedic surgery (CAOS). AR Head-mounted displays (HMDs) further allow non-eye-shift target observation and egocentric view. Recently, a markerless tracking and registration (MTR) algorithm was proposed to avoid the artifi...

Despite the growing availability of self-contained augmented reality head-mounted displays (AR HMDs) based on optical see-through (OST) technology, their potential applications across highly challenging medical and industrial settings are still hampered by the complexity of the display calibration required to ensure the locational coherence between...

Optical see-through (OST) augmented reality head-mounted displays are quickly emerging as a key asset in several application fields but their ability to profitably assist high precision activities in the peripersonal space is still sub-optimal due to the calibration procedure required to properly model the user's viewpoint through the see-through d...

Background: In the context of guided surgery, augmented reality (AR) represents a groundbreaking improvement. The Video and Optical See-Through Augmented Reality Surgical System (VOSTARS) is a new AR wearable head-mounted display (HMD), recently developed as an advanced navigation tool for maxillofacial and plastic surgery and other non-endoscopic...

In recent years, the spreading of visual augmented reality as an effective tool in image-guided surgery, has stimulated the research community to investigate the use of commercial augmented reality headsets a broad range of potential applications. This aroused enthusiasm among clinicians for the potential of augmented reality, but also revealed som...

Egocentric augmented reality (AR) interfaces are quickly becoming a key asset for assisting high precision activities in the peripersonal space in several application fields. In these applications, accurate and robust registration of computer-generated information to the real scene is hard to achieve with traditional Optical See-Through (OST) displ...

The simplified, manual calibration of commercial Optical See-Through Head-Mounted Displays (OST-HMDs) is neither accurate nor convenient for medical applications. An interaction-free calibration method that can be easily implemented in commercial headsets is thus desired. State-of-the-art automatic calibrations simplify the eye-screen system as a p...

Augmented reality (AR) Head-Mounted Displays (HMDs) are emerging as the most efficient output medium to support manual tasks performed under direct vision. Despite that, technological and human-factor limitations still hinder their routine use for aiding high-precision manual tasks in the peripersonal space. To overcome such limitations, in this wo...

The increasing capability of computing power and mobile graphics has made possible the release of self-contained augmented reality (AR) headsets featuring efficient head-anchored tracking solutions. Ego motion estimation based on well-established infrared tracking of markers ensures sufficient accuracy and robustness. Unfortunately, wearable visibl...

In recent years, there has been an increasing interest towards the augmented reality as applied to the surgical field. We conducted a systematic review of literature classifying the augmented reality applications in oral and cranio-maxillofacial surgery (OCMS) in order to pave the way to future solutions that may ease the adoption of AR guidance in...

In recent years, the entry into the market of self contained optical see-through headsets with integrated multi-sensor capabilities has led the way to innovative and technology driven augmented reality applications and has encouraged the adoption of these devices also across highly challenging medical and industrial settings. Despite this, the disp...

The growing availability of self-contained and affordable augmented reality headsets such as the Microsoft HoloLens is encouraging the adoption of these devices also in the healthcare sector. However, technological and human-factor limitations still hinder their routine use in clinical practice. Among them, the major drawbacks are due to their gene...

In non-orthostereoscopic video see-through (VST) head-mounted displays (HMDs), the perception of the three-dimensional space is negatively altered by geometrical aberrations, which may lead to perceptual errors, problems of hand-eye coordination, and discomfort for the user. Parallax-free VST HMDs have been proposed, yet their embodiments are gener...

Augmented Reality (AR) based on head-mounted displays (HMDs) represent the most ergonomic and efficient solution for aiding complex manual tasks performed under direct vision and it is considered an enabling technology of the fourth industrial revolution (Industry 4.0). This work intends to give a harmonized and consistent overview of the workflow...

Aortic valve replacement is the only definitive treatment for aortic stenosis, a highly prevalent condition in elderly population. Minimally invasive surgery brought numerous benefits to this intervention, and robotics recently provided additional improvements in terms of telemanipulation, motion scaling, and smaller incisions. Difficulties in obta...

An open discussion concerning some of the technical and human-factor flaws that still hinder the adoption of commercial OST head-mounted displays (e.g. Microsoft Hololens) in image-guided surgery.

Hybrid surgical simulators based on Augmented Reality (AR) solutions benefit from the advantages of both the box trainers and the Virtual Reality simulators. This paper reports on the results of a long development stage of a hybrid simulator for laparoscopic cholecystectomy that integrates real and the virtual components. We first outline the speci...

In the realm of wearable augmented reality (AR) systems, stereoscopic video see-through displays raise issues related to the user’s perception of the three-dimensional space. This paper seeks to put forward few considerations regarding the perceptual artefacts common to standard stereoscopic video see-through displays with fixed camera convergence....

In mini-invasive surgery and in endoscopic procedures, the surgeon operates without a direct visualization of the patient’s anatomy. In image-guided surgery, solutions based on wearable augmented reality (AR) represent the most promising ones. The authors describe the characteristics that an ideal Head Mounted Display (HMD) must have to guarantee s...

Augmented reality (AR) has been successfully providing surgeons an extensive visual information of surgical anatomy to assist them throughout the procedure. AR allows surgeons to view surgical field through the superimposed 3D virtual model of anatomical details. However, open surgery presents new challenges. This study provides a comprehensive ove...

In non-orthoscopic video see-through (VST) head-mounted displays (HMDs), depth perception through stereopsis is adversely affected by sources of spatial perception errors. Solutions for parallax-free and orthoscopic VST HMDs were considered to ensure proper space perception but at expenses of an increased bulkiness and weight. In this work, we pres...

Neuronavigation has become an essential neurosurgical tool in pursuing minimal invasiveness and maximal safety, even though it has several technical limitations. Augmented reality (AR) neuronavigation is a significant advance, providing a real-time updated 3D virtual model of anatomical details, overlaid on the real surgical field. Currently, only...

To assist surgeons in the acquisition of the required skills for the proper execution of the laparoscopic procedure, surgical simulators are used. During training with simulators it is useful to provide a surgical performance quantitative evaluation. Recent research works showed that such evaluation can be obtained by tracking the laparoscopic inst...

In the context of Augmented Reality (AR), industrial pioneers and early adopters have considered the wearable optical see-through (OST) displays as proper and effective tools in applications spanning from manufacturing and maintenance up to the entertainment field and the medical area, because they provide the user with an egocentric viewpoint main...

Aim: Navigation technology has given surgeons the capacity to know precisely where their instruments are during surgery, and simulation-guided navigation is a surgical method which is based on the use of navigation technology linked to 3D virtual surgery simulation. The aim of the present study was to evaluate the absolute accuracy of simulation-g...

ABSTRACT Purpose: Benefits of minimally invasive neurosurgery mandate the development of ergonomic paradigms for neuronavigation. Augmented Reality (AR) systems can overcome the shortcomings of commercial neuronavigators. The aim of this work is to apply a novel AR system, based on a head-mounted stereoscopic video see-through display, as an aid in...

We present an intuitive and ergonomic AR strategy to be coupled with a standard external fixation system aimed at aiding the accurate closed reduction of long-bone shaft fractures. The correct six DOF alignment between the bone fragments can be retrieved by manually repositioning a pair of reference frames constrained to the two extremities of the...

In the context of surgical navigation systems based on augmented reality (AR), the key challenge is to ensure the highest degree of realism in merging computer-generated elements with live views of the surgical scene. This paper presents an algorithm suited for wearable stereoscopic augmented reality video see-through systems for use in a clinical...

Augmented Reality (AR) has already proven its worth in various applications in the medical domain. However, most of the solutions proposed were bound to specific hardware or software configurations, and/or their application was limited to specific cases, thus lacking in flexibility. In this paper, we present a software framework suitable for AR vid...

In mini-invasive surgery, the surgeon operates without a direct visualization of the patient’s anatomy. In image-guided surgery, solutions based on augmented reality (AR) represent the most promising ones. The aim of this study was to evaluate the efficacy of a new wearable AR system as aid in the performance of percutaneous procedures in spine sur...

The most challenging step of cholecystectomy is the identification of Calot’s triangle. Adequate training by the surgical operators may reduce intraoperative risks due to anatomical misinterpretation. In this context, augmented reality (AR) can offer an excellent contribution to traditional education, by allowing the implementation of hybrid simula...

The human perception of the three-dimensional world is influenced by the mutual integration of physiological and psychological depth cues, whose complexity is still an unresolved issue per se. Even more so if we wish to mimic the perceptive efficiency of the human visual system within augmented reality (AR) based surgical navigation systems. In thi...

Surgical simulation based on augmented reality (AR), mixing the benefits of physical and virtual simulation, represents a step forward in surgical training. However, available systems are unable to update the virtual anatomy following deformations impressed on actual anatomy. A proof-of-concept solution is described providing AR visualization of hi...