Figure - uploaded by Hongen Liao
Content may be subject to copyright.
Augmented reality system configuration, which consists of an IV display device, 3D data source collection equipment (computed tomography), 3D optical tracking system, and a computer. A half-silvered mirror is attached to the IV display. Through the half mirror, surgeons can see the IV image without the need for special glasses. (a) Tracker attached to the teeth; (b) tracker attached to the surgical instrument. 3D, three-dimensional; IV, integral videographic.
Source publication
To evaluate the feasibility and accuracy of a three-dimensional augmented reality system incorporating integral videography for imaging oral and maxillofacial regions, based on preoperative computed tomography data. Three-dimensional surface models of the jawbones, based on the computed tomography data, were used to create the integral videography...
Contexts in source publication
Context 1
... configuration of the AR system is shown in Figure 1. The IV image display, which has been developed at The University of Tokyo and previously described, [6][7][8] consists of a high-density rear liquid crystal display and microarray glass. ...Context 2
... system measured the position and direction of the subject's movements in real time. The tracking marker was noninva- sively attached to the subject's tooth by using a dental splint ( Figure 1a). The coordinate system of the IV images was obtained by measuring the position of the characteristics of these images in space. ...Context 3
... integrate the coordinate systems between the object and IV images, the Polaris Spectra optical tracking system was used. However, in this case, the tracking marker was attached to the surgical instrument (Figure 1b). The coordinate system of the IV images was obtained by measuring the position of the cha- racteristics of these images in space. ...Context 4
... this system, the surgeon could view the internal structures in the 3D for- mat, the data for which were initially obtained from the preoperative CT data and superimposed onto the actual subject's anatomy through a half-silvered mirror. Supplementary Movies 1-3 show the surgical sites, as viewed by the surgeon. The movies demonstrate that by using the AR technique, stereoscopy is possible from every position and that the stereoscopic images are accurately displayed across different view- ing angles. ...Similar publications
There is no apical morphological data being available for mandibular first or second premolars in the Turk-ish population. The aims of the study were (I) to assess apical morphological data of mandibular first and second premolars in a Turkish population at a young-adult age range (II) to analyze potential correlations between the size and position...
Citations
... It is the most malignant and harmful tumor of the head and neck, accounting for about 50% of the incidence of the head and neck squamous cell carcinoma [2]. Due to the rich blood flow and complex anatomical structure of oral and maxillofacial region, OSCC surgery often cannot completely remove the tumor [3]. At the same time, OSCC is prone to lymph node metastasis and postoperative recurrence, so its prognosis is poor [4]. ...
Objective:
N7-methylguanosine modification-related lncRNAs (m7G-related lncRNAs) are involved in progression of many diseases. This study was aimed at revealing the risk correlation between N7-methylguanosine modification-related lncRNAs and survival prognosis of oral squamous cell carcinoma.
Methods:
In the present study, coexpression network analysis and univariate Cox analysis were used to obtained 31 m7G-related mRNAs and 399 m7G-related lncRNAs. And the prognostic risk score model of OSCC patients was evaluated and optimized through cross-validation.
Results:
Through the coexpression analysis and risk assessment analysis of m7G-related prognostic mRNAs and lncRNAs, it was found that six m7G-related prognostic lncRNAs (AC005332.6, AC010894.1, AC068831.5, AL035446.1, AL513550.1, and HHLA3) were high-risk lncRNAs. Three m7G-related prognostic lncRNAs (AC007114.1, HEIH, and LINC02541) were protective lncRNAs. Then, survival curves were drawn by comparing the survival differences between patients with high and low expression of each m7G-related prognostic lncRNA in the prognostic risk score model. Further, risk curves, scatter plots, and heat maps were drawn by comparing the survival differences between high-risk and low-risk OSCC patients in the prognostic model. Finally, forest maps and the ROC curve were generated to verify the predictive power of the prognostic risk score model. Our results will help to find early and accurate prognostic risk markers for OSCC, which could be used for early prediction and early clinical intervention of survival, prognosis, and disease risk of OSCC patients in the future.
... Nevertheless, the registration and implant accuracy are improved this method as a marker-based. Suenaga et al. [22] has improved on the visualisation of the system by proposing a stereo vision for tracking and marker-less registration on the state of the art solution in [23]. They established a new AR marker-less registration system which does not need any additional display device for surgeons to see the superimposed 3D image. ...
... Accuracy and processing time results for mandible (adult[20][21][22][23][24][25][26][27][28][29][30] ...
Over time, Augmented Reality (AR) based technology becomes not being properly to implement with oral and maxillofacial surgery to visualise the narrow area spot in jaw surgery as blood vassals and root canals in these types of surgeries. Image registration is considered the major limitation of using the AR in these types of surgeries and reduces the accuracy of visualising the narrow areas. In this research, we propose a Correntropy based scale ICP algorithm as a solution to improve the image registration during jaw surgery. Correntropy is considered here to minimise the error metric of the ICP algorithm instead of the Euclidean distance measurement compared to the state-of-the-art solution. This led to decrease the registration error, increase the video accuracy and reduce the processing time simultaneously. The proposed system consists of Enhanced Tracking Learning Detection (TLD), which is used as an occlusion removal featured algorithm in the intra-operative stage of the AR-based jaw surgery system. In this research, a Modified Correntropy-based enhanced ICP (MCbeICP) algorithm is proposed for the system’s pose-refinement phase. Moreover, this proposed algorithm (MCbeICP) has a new function to process the point set registration with great noises and outliers. It eliminates the poor performance of the ICP algorithm of the noisy point set. Furthermore, the ICP algorithm considers the scale factor to register the point with different scales of the real-time video and the sample models. Additionally, this method improves the result of the pose refinement stage in terms of registration accuracy and processing time. By this method, the pose refinement stage gives an improved result in terms of registration accuracy and processing time. The samples, which were taken from the upper (maxillary) and the lower (mandible) jaw bone show that the proposed algorithm provides a significant accuracy improvement in alignment to 0.21- 0.29 mm from 0.23 to 0.35 mm and an increment in processing time from 8 to 12 frames per second (fs/s) to 10-14 fs/s compared to the result provided by state of the art. The proposed augmented reality (AR) system is focused on the overlay accuracy and processing time. Finally, this study addressed the limitation of Image registration with AR using modified Correntropy-based enhanced ICP algorithm to implement oral and maxillofacial surgery successfully.
... AR has been studied and used in surgery, where preoperative data are superimposed on the surgical site to enhance the perception of the physical environment. 12 The experiments of Jud 13 et al. and Porpiglia 14 et al. proved that in surgery, the application of AR in surgery can significantly improve success rates. AR is completed through the following steps. ...
In modern radiotherapy, error reduction in the patients' daily setup error is important for achieving accuracy. In our study, we proposed a new approach for the development of an assist system for the radiotherapy position setup by using augmented reality (AR). We aimed to improve the accuracy of the position setup of patients undergoing radiotherapy and to evaluate the error of the position setup of patients who were diagnosed with head and neck cancer, and that of patients diagnosed with chest and abdomen cancer. We acquired the patient's simulation CT data for the three-dimensional (3D) reconstruction of the external surface and organs. The AR tracking software detected the calibration module and loaded the 3D virtual model. The calibration module was aligned with the Linac isocenter by using room lasers. And then aligned the virtual cube with the calibration module to complete the calibration of the 3D virtual model and Linac isocenter. Then, the patient position setup was carried out, and point cloud registration was performed between the patient and the 3D virtual model, such the patient's posture was consistent with the 3D virtual model. Twenty patients diagnosed with head and neck cancer and 20 patients diagnosed with chest and abdomen cancer in the supine position setup were analyzed for the residual errors of the conventional laser and AR-guided position setup. Results show that for patients diagnosed with head and neck cancer, the difference between the two positioning methods was not statistically significant (P > 0.05). For patients diagnosed with chest and abdomen cancer, the residual errors of the two positioning methods in the superior and inferior direction and anterior and posterior direction were statistically significant (t = -5.80, -4.98, P < 0.05). The residual errors in the three rotation directions were statistically significant (t = -2.29 to -3.22, P < 0.05). The experimental results showed that the AR technology can effectively assist in the position setup of patients undergoing radiotherapy, significantly reduce the position setup errors in patients diagnosed with chest and abdomen cancer, and improve the accuracy of radiotherapy.
... The third component is a display system that allows virtual and 3D objects to be viewed in the real world. Finally, a tracking system is required to complete the registration phase, which involves a constant tracking of the user during the procedure to allow for real-time visualization [10], [11]. Marker-free registration, such as laser skin surface scanning, and markerbased registration, such as anatomical landmarks, bone screws, and skin adhesive markers, are the two primary forms of registration techniques [12], [13]. ...
... In see-through display and projection-based AR, Virtual objects are projected on transparent silver mirrors, see-through devices, and projectors. The virtual objects are projected on a device positioned between the patient and the operator in the operating area [10], [12], [16]- [20]. 3D image display technologies vary; among them, we have stereoscopy, integral imaging (or integral photography), and holography. ...
... In AR Systems, the virtual objects can be viewed from multiple angles and follow the patient and the operator's movements using tracking systems [10], [19]. Tracking is accomplished using two methods. ...
This literature review aims to discuss augmented reality systems and provide an update on the most recent technological developments and applications in the dental field. The studies that met the inclusion criteria in the last 20 years, from 2000 to 5 May 2020, in the PubMed database were included. The search resulted in n=72 articles, in which n=40 included and n=32 excluded. AR systems are still being tested as there are still some limitations that limit the adoption of this technology in the dental sector. Several studies have resulted in a device appropriate for clinical use, yet no regular clinical application was recorded.
... This allows the surgeons to focus their attention on the patient and the surgical site alone, and they do not have to mentally map information between two different sites. Several experimental AR systems to support complex interventions have been experimentally trailed, in example for tumor resection [41][42][43][44], pedicle screw placement [45][46][47] (as shown exemplary in Fig. 5.5E), and surgical drilling [48,49]. Even though IGS is the most common procedure supported by medical AR, it is also the most challenging. ...
... The stationary setup between physician and patient might, however, limit the workspace of the surgeons, and the response time of the system was still an area of improvement. The same setup was later used by Suenaga et al. [49] for displaying surface renderings of the maxillary and mandibular jaw on a human volunteer. ...
... The AR-based guidance of procedures such as bone sectioning [10], bone drilling [11] or the identification of correct entry points and trajectories of surgical instruments [12] requires high accuracy. Certain types of surgery, e.g. the excision of small tumours [13] or otologic surgery [14], require submillimetre accuracy. ...
Emerging holographic headsets can be used to register patient-specific virtual models obtained from medical scans with the patient’s body. Maximising accuracy of the virtual models’ inclination angle and position (ideally, ≤ 2° and ≤ 2 mm, respectively, as in currently approved navigation systems) is vital for this application to be useful. This study investigated the accuracy with which a holographic headset registers virtual models with real-world features based on the position and size of image markers.
HoloLens® and the image-pattern-recognition tool Vuforia Engine™ were used to overlay a 5-cm-radius virtual hexagon on a monitor’s surface in a predefined position. The headset’s camera detection of an image marker (displayed on the monitor) triggered the rendering of the virtual hexagon on the headset’s lenses. 4 × 4, 8 × 8 and 12 × 12 cm image markers displayed at nine different positions were used. In total, the position and dimensions of 114 virtual hexagons were measured on photographs captured by the headset’s camera.
Some image marker positions and the smallest image marker (4 × 4 cm) led to larger errors in the perceived dimen-sions of the virtual models than other image marker positions and larger markers (8 × 8 and 12 × 12 cm). ≤ 2° and ≤ 2 mm errors were found in 70.7% and 76% of cases, respectively.
Errors obtained in a non-negligible percentage of cases are not acceptable for certain surgical tasks (e.g. the identification of correct trajectories of surgical instruments). Achieving sufficient accuracy with image marker sizes that meet surgical needs and regardless of image marker position remains a challenge.
... Thirdly, a display system to display virtual and 3D objects in the real world. Lastly, a tracking device is needed to accomplish the registration phase, which is a phase that is needed to continuously track the user during the procedure to allow for real-time visualization [6]. Registration techniques can be categorized into two main groups: marker-free registration, such as laser skin surface scanning, and marker-based registration, such as anatomical landmarks, bone screws, and skin adhesive markers [7,8]. ...
... Registration techniques can be categorized into two main groups: marker-free registration, such as laser skin surface scanning, and marker-based registration, such as anatomical landmarks, bone screws, and skin adhesive markers [7,8]. The virtual objects can be viewed from multiple angles and follow the patient and the operator's movements by the usage of tracking systems [6][7][8][9]. ...
... See-through display and projection-based AR uses translucent silver mirrors, see-through devices, and projectors. Those devices are placed between the operator and the patient; to allow the projection of the virtual objects [6,[13][14][15][16]. Multiple researchers have proved the effectiveness of the AR simulators in assisting dentists by showing and displaying virtual models in the operating field. ...
... Thirdly, a display system to display virtual and 3D objects in the real world. Lastly, a tracking device is needed to accomplish the registration phase, which is a phase that is needed to continuously track the user during the procedure to allow for real-time visualization [6]. Registration techniques can be categorized into two main groups: marker-free registration, such as laser skin surface scanning, and marker-based registration, such as anatomical landmarks, bone screws, and skin adhesive markers [7,8]. ...
... Registration techniques can be categorized into two main groups: marker-free registration, such as laser skin surface scanning, and marker-based registration, such as anatomical landmarks, bone screws, and skin adhesive markers [7,8]. The virtual objects can be viewed from multiple angles and follow the patient and the operator's movements by the usage of tracking systems [6][7][8][9]. ...
... See-through display and projection-based AR uses translucent silver mirrors, see-through devices, and projectors. Those devices are placed between the operator and the patient; to allow the projection of the virtual objects [6,[13][14][15][16]. Multiple researchers have proved the effectiveness of the AR simulators in assisting dentists by showing and displaying virtual models in the operating field. ...
Introduction: With all the advancements that technology has reached, Dentistry can't be left behind. In the past few years, researchers have focused on emerging technologies like Virtual and Augmented Reality with clinical practice. Objectives: This literature review aims to provide an update on the latest technological applications and development in augmented reality in the dental field. Methods: The PubMed database was reviewed, and the studies that fulfilled the inclusion criteria in the last 20 years, from 2000 to 5 May 2020, were included. Results: The search results revealed a total of 72 articles, 32 were excluded, while 40 articles were included. It’s been observed that augmented reality application is still under testing, as certain drawbacks still tie the spread of this technology in the dental field. Multiple studies have resulted in a system that is suitable for clinical use. Yet no routine clinical application has been reported. Conclusion: The research department has already covered more advanced technologies like mixed reality. Therefore, a question arises, whether augmented realty will continue to grow independently or will mixed reality dominate the field.
... Thirdly, a display system to display virtual and 3D objects in the real world. Lastly, a tracking device is needed to accomplish the registration phase, which is a phase that is needed to continuously track the user during the procedure to allow for real-time visualization [6]. Registration techniques can be categorized into two main groups: marker-free registration, such as laser skin surface scanning, and marker-based registration, such as anatomical landmarks, bone screws, and skin adhesive markers [7,8]. ...
... Registration techniques can be categorized into two main groups: marker-free registration, such as laser skin surface scanning, and marker-based registration, such as anatomical landmarks, bone screws, and skin adhesive markers [7,8]. The virtual objects can be viewed from multiple angles and follow the patient and the operator's movements by the usage of tracking systems [6][7][8][9]. ...
... See-through display and projection-based AR uses translucent silver mirrors, see-through devices, and projectors. Those devices are placed between the operator and the patient; to allow the projection of the virtual objects [6,[13][14][15][16]. Multiple researchers have proved the effectiveness of the AR simulators in assisting dentists by showing and displaying virtual models in the operating field. ...
Introduction: With all the advancements that technology has reached, Dentistry can't be left behind. In the past few years, researchers have focused on emerging technologies like Virtual and Augmented Reality with clinical practice. Objectives: This literature review aims to provide an update on the latest technological applications and development in augmented reality in the dental field. Methods: The PubMed database was reviewed, and the studies that fulfilled the inclusion criteria in the last 20 years, from 2000 to 5 May 2020, were included. Results: The search results revealed a total of 72 articles, 32 were excluded, while 40 articles were included. It’s been observed that augmented reality application is still under testing, as certain drawbacks still tie the spread of this technology in the dental field. Multiple studies have resulted in a system that is suitable for clinical use. Yet no routine clinical application has been reported. Conclusion: The research department has already covered more advanced technologies like mixed reality. Therefore, a question arises, whether augmented realty will continue to grow independently or will mixed reality dominate the field.
... Augmented reality (AR) technology can be used to generate 3-dimensional (3D) images from preoperative imaging data and superimpose the reconstructed 3D images on the surgical field during the intraoperative period [3]. The superimposition of the "transparent" images of anatomical structures on the surgical field assists the surgeon in determining the location of the corresponding unseen structures lying beneath the visible surfaces of organs and dissected tissues view by the surgeon [4]. The use of AR technology to assist in the identification of the inferior pancreaticoduodenal artery (IPDA) in the artery-first approach for PD has been previously reported [5]. ...
Introduction:
Pancreaticoduodenectomy (PD) with superior mesenteric vein (SMV) reconstruction are often required to achieve complete (R0) resection for pancreatic head cancer (PHC) with tumor invasion of the SMV. Augmented reality (AR) technology can be used to assist in determining the extent of SMV involvement by superimposing virtual 3-dimensional (3D) images of the pancreas and regional vasculature on the surgical field.
Materials and methods:
Three patients with PHC and tumor invasion of the SMV underwent AR-assisted PD with SMV resection and reconstruction following preoperative computed tomography scanning. Preoperative imaging data were used to reconstruct 3D images of anatomical structures, including the tumor, portal vein (PV), SMV, and splenic vein (SV). Using AR software installed on a smart phone, the reconstructed 3D images were superimposed on the surgical field as viewed in a smart phone display to provide intermittent navigational assistance to the surgeon in identifying the boundaries of PHC tumor invasion for resection of the vessels involved.
Result:
All patients successfully completed the operation. Intraoperative AR applications displayed virtual images of the pancreas, SMV, bile duct, common hepatic artery (CHA), and superior mesenteric artery (SMA). Two patients required end-to-end anastomosis for reconstruction of the SMV. One patient required allogenic vascular bypass to reconstruct the SMV-PV juncture with concomitant reconstruction of the SV-SMV confluence by end-to-side anastomosis of the SV and bypass vessel. Postoperative pathology confirmed R0 resections for all patients.
Conclusion:
AR navigation technology based on preoperative CT image data can assist surgeons performing PD with SMV resection and reconstruction.
